From yohuang at nvidia.com  Thu Dec 12 10:04:07 2019
From: yohuang at nvidia.com (Justin Huang)
Date: Thu Dec 12 10:39:18 2019
Subject: [odb-users] Options to speed up object persist to sqlite3 DB?
Message-ID: <MN2PR12MB3024933FB6B58A5BFBF4684ECB550@MN2PR12MB3024.namprd12.prod.outlook.com>

Hi,

I used VTune to profile my program and it showed odb::database::persist (more specifically odb::sqlite::insert_statement::execute) is the bottle neck. It takes around 3 seconds to persist 7k simple object (I am not quite sure if this is normal or not).

In my program, I iterate all objects and call db->persist(object) one by one. Are there faster ways to persist objects?

Thanks,
Justin

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
From boris at codesynthesis.com  Mon Dec 16 06:49:13 2019
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Mon Dec 16 06:52:24 2019
Subject: [odb-users] Options to speed up object persist to sqlite3 DB?
In-Reply-To: <MN2PR12MB3024933FB6B58A5BFBF4684ECB550@MN2PR12MB3024.namprd12.prod.outlook.com>
References: <MN2PR12MB3024933FB6B58A5BFBF4684ECB550@MN2PR12MB3024.namprd12.prod.outlook.com>
Message-ID: <boris.20191216134348@codesynthesis.com>

Justin Huang <yohuang@nvidia.com> writes:

> I used VTune to profile my program and it showed odb::database::persist
> (more specifically odb::sqlite::insert_statement::execute) is the bottle
> neck. It takes around 3 seconds to persist 7k simple object (I am not
> quite sure if this is normal or not).
> 
> In my program, I iterate all objects and call db->persist(object) one
> by one. Are there faster ways to persist objects?

Normally, for best performance, you would want to batch perists into
several transactions. That is, persist N objects per transaction,
commit it, start the next transaction, persist the next N objects,
and so on. You will need to experiment to find the best N (it would
depend on the size/shape of your objects, etc).

Beyond that, SQLite includes a lot of tunable options that allow you
to speed things up, usually at the expense for consistency and/or
reliability. I suggest that you google for these since they are
not ODB-specific.

From aviad.danieli at morphisec.com  Tue Dec 17 04:59:14 2019
From: aviad.danieli at morphisec.com (Aviad Danieli)
Date: Tue Dec 17 07:13:54 2019
Subject: [odb-users] Newbie call for aid
Message-ID: <CAN0dy+eV-fWQWyeNYgLs6qhTToZSTQh_zeP1Ub-H8Z85CynojQ@mail.gmail.com>

Hi all,

I wish to use ODB to connect my project to a SQLite database.
I managed to write the required header files and got the hxx and cxx files
generated by ODB. However, I don't seem to manage to compile the project.

I went back to the published example projects, but going over the
instructions in the README files it seems like I need to run ./configure
--with-database=<database> but I don't seem to fine the configure file or a
way to generate it anywhere. All documentation seems to assume I can run it
from the project's root.

I know it may sound like a silly question, but can anyone point me in
the right direction here?
How do I find\generate the configure file required to compile a project
with ODB artifacts in it?

Much appreciated,

-- 

*    Aviad Danieli*

-- 
CONFIDENTIALITY NOTICE: The contents of this email message and any 
attachments are intended solely for the addressee(s) and may contain 
confidential and/or privileged information and?their further?disclosure?may 
be contractually or legally restricted. If you are not the intended 
recipient, please do not use or disclose the contents of this email or its 
attachments to anyone and please delete it immediately and notify the 
sender.
From boris at codesynthesis.com  Tue Dec 17 07:38:16 2019
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Tue Dec 17 07:41:31 2019
Subject: [odb-users] Newbie call for aid
In-Reply-To: <CAN0dy+eV-fWQWyeNYgLs6qhTToZSTQh_zeP1Ub-H8Z85CynojQ@mail.gmail.com>
References: <CAN0dy+eV-fWQWyeNYgLs6qhTToZSTQh_zeP1Ub-H8Z85CynojQ@mail.gmail.com>
Message-ID: <boris.20191217142846@codesynthesis.com>

Aviad Danieli <aviad.danieli@morphisec.com> writes:

> I managed to write the required header files and got the hxx and cxx files
> generated by ODB. However, I don't seem to manage to compile the project.

What build system are you using for your project and what exactly didn't
work?

Generally, the recommended sequence of steps to add ODB to a new or
existing project:

1. Build the ODB compiler and ODB runtime libraries as described in:

   https://codesynthesis.com/products/odb/doc/install-build2.xhtml

2. Compile your header file with the ODB compiler produced on step (1).

3. Take the *-odb.* header/source files produced on step (2) and add
   them to your project like any other header/source file.

4. Link to the ODB runtime libraries produced on step (1).


> I went back to the published example projects, but going over the
> instructions in the README files it seems like I need to run ./configure
> --with-database=<database> but I don't seem to fine the configure file or a
> way to generate it anywhere.

I don't think this will help your much unless you are using autotools
to build your project.

From javier.gutierrez at web.de  Sun Dec 29 15:32:53 2019
From: javier.gutierrez at web.de (Javier Gutierrez)
Date: Sun Dec 29 15:37:05 2019
Subject: [odb-users] Generate "Valite_to" from next "valid_from"
Message-ID: <0f7401d5be87$2c7d4fd0$8577ef70$@web.de>

Hi there,



I have a table only with a valid_from column, I need to make a range out of
it, so the valid_to for current row is the valid_from of the next row. I am
talking about timestamps if it makes any difference.

E.g.

ID, ITEM_ID,   VALID_FROM      -> VALID_TO

1,   1,         2019.01.01      -> 2019.02.15

2,   1,         2019.02.15      -> 2019.03.07

3,   1,         2019.03.07      -> 2100.01.01

4,   2,         2019.01.20      -> 2019.03.01

5,   2,         2019.03.01      -> 2100.01.01



I need to generate this column on the fly because I am querying against it.
Additionally I need this to work on any database.

Any ideas how to do this using ODB or if it is possible at all ?



Thanks a lot in advance,

Javier

From boris at codesynthesis.com  Mon Dec 30 09:22:40 2019
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Mon Dec 30 09:26:36 2019
Subject: [odb-users] Generate "Valite_to" from next "valid_from"
In-Reply-To: <0f7401d5be87$2c7d4fd0$8577ef70$@web.de>
References: <0f7401d5be87$2c7d4fd0$8577ef70$@web.de>
Message-ID: <boris.20191230161434@codesynthesis.com>

Javier Gutierrez <javier.gutierrez@web.de> writes:

> I have a table only with a valid_from column, I need to make a range out of
> it, so the valid_to for current row is the valid_from of the next row. I am
> talking about timestamps if it makes any difference.
> 
> E.g.
> 
> ID, ITEM_ID,   VALID_FROM      -> VALID_TO
> 
> 1,   1,         2019.01.01      -> 2019.02.15
> 
> 2,   1,         2019.02.15      -> 2019.03.07
> 
> 3,   1,         2019.03.07      -> 2100.01.01 
> 
> I need to generate this column on the fly because I am querying against it.

You need to generate the *column* (as in, "ALTER TABLE ... ADD COLUMN ...")
or the *value in the column*? I am guessing it's the latter but your
imprecise language makes it hard to understand what exactly you mean (what
does "I am qurying against it" has to do with anything then).


> Additionally I need this to work on any database.
> 
> Any ideas how to do this using ODB or if it is possible at all ?

If it's the column value that you need to generate, I don't see a
difficulty: load/query the previous row and copy its VALID_TO value to
the VALID_FROM of the new row?