From boris at codesynthesis.com  Wed Mar  1 06:47:57 2023
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Wed Mar  1 06:40:31 2023
Subject: [odb-users] Creating virtual tables
In-Reply-To: <CANOaDUvDWNrcVoxMaaiSMX5eG4dR4JbqWaXngjPcpNCihGjd0A@mail.gmail.com>
References: <CANOaDUtfOVh-iRGQXoA=WNgZisogBdzDQNCg5mgnxNZXa-3-9Q@mail.gmail.com>
	<boris.20230227142840@codesynthesis.com>
	<CANOaDUvDWNrcVoxMaaiSMX5eG4dR4JbqWaXngjPcpNCihGjd0A@mail.gmail.com>
Message-ID: <boris.20230301134423@codesynthesis.com>

Irvin Josue Hernandez Rivera <iherna31@asu.edu> writes:

> The underlying DB I am using is Sqlite DB and I am using this db to store
> user positions as a Lat Lon. I wanted to add support to the DB for ranged
> queries an rtree https://www.sqlite.org/rtree.html
> In order for sqlite to create the rtree module the table instantiation need
> to be as follows:
> 
> CREATE VIRTUAL TABLE *<name>* USING rtree(*<column-names>*);
>
> I don't think the options pragma allows for this type of table creation

Right, AFAICS, this is an ad hoc statement that doesn't follow the
standard CREATE TABLE syntax:

https://www.sqlite.org/lang_createtable.html

I think your best bet is to create it manually by executing the
necessary statement directly.

From becoggins at hotmail.com  Fri Mar 10 11:00:01 2023
From: becoggins at hotmail.com (Brian E. Coggins)
Date: Fri Mar 10 10:52:53 2023
Subject: [odb-users] Query Language and Polymorphic Types
Message-ID: <MW4PR01MB62272ACCFA34A219FBDED6B7B0BA9@MW4PR01MB6227.prod.exchangelabs.com>

Hi Boris and Friends,

Imagine one has a polymorphic hierarchy like the following (all of these as database classes):

struct base
{
	?
};

struct derived : public base
{
	...
	int data;
};

And this is connected by relationship to another class (also a database class):

struct owner
{
	...
	std::shared_ptr< base > owned;
};

Now imagine I?m querying for owner instances where I already know that owned is of type derived and I?d like to include derived.data as a query field.  Does ODB?s query language offer any kind of trick for including the derived member in the query?  I?ve been messing around with odb::query<owner>::owned but can?t figure out any obvious way to poke into the derived type from within the query language.

I?d love to avoid having to load every owned, dynamic_cast each of them to derived one-at-a-time, and manually inspect each data member.

Of course I could also just hand-code a SELECT?JOIN query and make a custom view class, and maybe that?s the best answer.  But it would be nifty if the query language had a solution.

Thanks,
Brian



From boris at codesynthesis.com  Mon Mar 13 08:29:35 2023
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Mon Mar 13 08:22:01 2023
Subject: [odb-users] Query Language and Polymorphic Types
In-Reply-To: <MW4PR01MB62272ACCFA34A219FBDED6B7B0BA9@MW4PR01MB6227.prod.exchangelabs.com>
References: <MW4PR01MB62272ACCFA34A219FBDED6B7B0BA9@MW4PR01MB6227.prod.exchangelabs.com>
Message-ID: <boris.20230313142525@codesynthesis.com>

Brian E. Coggins <becoggins@hotmail.com> writes:

> Now imagine I?m querying for owner instances where I already know
> that owned is of type derived and I?d like to include derived.data
> as a query field.  Does ODB?s query language offer any kind of trick
> for including the derived member in the query?

I think your best bet is to try to express this assumption (i.e., that
the pointer to the base object is actually always the derived object)
as an object loading view:

https://www.codesynthesis.com/products/odb/doc/manual.xhtml#10.2

From becoggins at hotmail.com  Mon Mar 13 16:16:01 2023
From: becoggins at hotmail.com (Brian E. Coggins)
Date: Mon Mar 13 16:08:51 2023
Subject: [odb-users] Query Language and Polymorphic Types
In-Reply-To: <boris.20230313142525@codesynthesis.com>
References: <MW4PR01MB62272ACCFA34A219FBDED6B7B0BA9@MW4PR01MB6227.prod.exchangelabs.com>
	<boris.20230313142525@codesynthesis.com>
Message-ID: <MW4PR01MB6227FE5D76243F6C597991CBB0B99@MW4PR01MB6227.prod.exchangelabs.com>



> On 13 Mar 2023, at 08:29, Boris Kolpackov <boris@codesynthesis.com> wrote:
> 
> I think your best bet is to try to express this assumption (i.e., that
> the pointer to the base object is actually always the derived object)
> as an object loading view:
> 
> https://www.codesynthesis.com/products/odb/doc/manual.xhtml#10.2

Ah, that makes sense as a way to do it.  Thank you!

Brian

From lloydkl.tech at gmail.com  Tue Mar 21 05:59:40 2023
From: lloydkl.tech at gmail.com (Lloyd)
Date: Tue Mar 21 05:52:20 2023
Subject: [odb-users] Slow select in SQLite
Message-ID: <CAA10ZTqb1-Pp3=fMCUmH8spSvbJa44d9_4Pp-W6q0u0ZMCqdkg@mail.gmail.com>

Hi,


We have the following two tables created through ODB in an SQLite database


CREATE TABLE "InfoTable" ("ID" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"Name" TEXT NOT NULL,
"ImageID" INTEGER NOT NULL,
CONSTRAINT "ImageID_fk" FOREIGN KEY("ImageID") REFERENCES
"AddInfoTable"("ImageFileId") ON DELETE CASCADE DEFERRABLE INITIALLY
DEFERRED
);


CREATE TABLE "AddInfoTable" ("ImageFileId" INTEGER NOT NULL PRIMARY KEY
AUTOINCREMENT,
"ImageFormat" TEXT NOT NULL,
"ImageSize" INTEGER NOT NULL,
"DataTree" TEXT NOT NULL );//JSON - represents a tree


DataTree column of AddInfoTable stores a JSON (As plain text) representing
a hierarchical data structure.  The table contains 0.25 million records.


The following ODB query is used to retrieve the data. It takes almost an
hour to fetch all the data from the table.

typedef odb::query<InfoTable> query;
result r = db.query<InfoTable>();


We have tried the following scenarios to identify the problem


1) Using a DB having only the InfoTable containing 0.25 million records. It
takes less than 1 sec to fetch all the data from the db.

2) The DB consists of 2 tables, InfoTable and AddInfoTable. InfoTable
consists of 3 fields and one of them is the foreign key of AddInfoTable and
the AddInfoTable has 5 fields but no fields holding JSON type value. This
too retrieves the data from the table very fast.

3)Used a JSON of size only 400 bytes (DataTree) and tried to retrieve, it
takes less than 1 sec. only.

4) The query is very slow when the AddInfoTable has the JSON (Data Tree
Column) data of size 10kb/raw on average.

May I know the reason for this slow query? (The same tables and query works
fast on ODB PostgreSQL)

We are using ODB 2.4 on Windows 10 with SQLite version 3.27.2

Thanks a lot in advance,
  Lloyd
From boris at codesynthesis.com  Wed Mar 22 08:11:34 2023
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Wed Mar 22 08:03:52 2023
Subject: [odb-users] Slow select in SQLite
In-Reply-To: <CAA10ZTqb1-Pp3=fMCUmH8spSvbJa44d9_4Pp-W6q0u0ZMCqdkg@mail.gmail.com>
References: <CAA10ZTqb1-Pp3=fMCUmH8spSvbJa44d9_4Pp-W6q0u0ZMCqdkg@mail.gmail.com>
Message-ID: <boris.20230322140000@codesynthesis.com>

Lloyd <lloydkl.tech@gmail.com> writes:

> 1) Using a DB having only the InfoTable containing 0.25 million records. It
> takes less than 1 sec to fetch all the data from the db.
> 
> 2) The DB consists of 2 tables, InfoTable and AddInfoTable. InfoTable
> consists of 3 fields and one of them is the foreign key of AddInfoTable and
> the AddInfoTable has 5 fields but no fields holding JSON type value. This
> too retrieves the data from the table very fast.
> 
> 3)Used a JSON of size only 400 bytes (DataTree) and tried to retrieve, it
> takes less than 1 sec. only.
> 
> 4) The query is very slow when the AddInfoTable has the JSON (Data Tree
> Column) data of size 10kb/raw on average.


So the database size in case (4) is 250,000 * 10KB = ~2.5GB. It shouldn't
cause any issues on moderns hardware (with SSDs, etc), but it's not exactly
nothing. But still, 1h sounds abnormal.


> May I know the reason for this slow query? (The same tables and query works
> fast on ODB PostgreSQL)
>
> We are using ODB 2.4 on Windows 10 with SQLite version 3.27.2

I am 99% certain this is not about ODB but about SQLite and/or Windows.
One thing that I would try is make the column just TEXT, not JSON, to
eliminate any (remote) possibility of some JSON validation overhead.

Another thing worth trying would be to switch to an in-memory SQLite
database and see what kind of numbers you get. If it's fast, then it's
definitely not ODB and I would suggest asking on the SQLite mailing
lists/forums.

From Rohith.Tenneti2 at carrier.com  Mon Mar 27 12:49:23 2023
From: Rohith.Tenneti2 at carrier.com (Tenneti, Rohith)
Date: Tue Mar 28 06:34:06 2023
Subject: [odb-users] New distribution pkgs for ODB
In-Reply-To: <MWHPR06MB2368B3AAAD5C33D7CF7580F2FB849@MWHPR06MB2368.namprd06.prod.outlook.com>
References: <MWHPR06MB2368B3AAAD5C33D7CF7580F2FB849@MWHPR06MB2368.namprd06.prod.outlook.com>
Message-ID: <BYAPR06MB38957B31E1ECA38B3FAD4A76BC8B9@BYAPR06MB3895.namprd06.prod.outlook.com>

Hello,

Could you share the distribution packages for odb, libodb, and libodb-sqlite for 2.5.0-b.23. We ran into a few compile issues and would like to use the official distribution packages.



Even though I followed the INSTALL-GIT and INSTALL instructions on each package I was not able to 'make dist' for any of the relevant interested pkgs ( odb, libodb, libodb-sqlite ).

This is what worked:

  *   Installed all dependencies as per INSTALL-GIT ( ie. build-0.3, cli, libcutl plus all other autotools related pkgs ).

This is what DID NOT work:

  *   Create a 'make dist' starting with the odb compiler package. After going through the initial config and choosing installed deps for cli and libcutl ( /usr/local ).

I kept getting errors in the make starting with this one "../odb/odb/options.cli

unknown option '--generate-specifier'

make: *** [~/Downloads/odb/build/import/cli/cli-cxx.make:27: ~/Downloads/odb/odb/options.hxx] Error 1"

What we need:
Distribution pkgs for each of the following components from .git tag or latest master if stable.
( we are looking for odb dist source and libs that support -std=c++17 )
libodb_2.5.0-b.23.tar.bz2
libodb-sqlite_2.5.0-b.23.tar.bz2
odb_2.5.0-b.23.tar.bz2 ( or already built binary pkg for odb compiler ).

BR,
Antonio Montero
Staff Engineer, Gen Engrg, Panels NPD
Robust Design Lead (RDL) Systems,
Commercial Fire - Edwards&Barcelona
8985 Town Center Parkway
Bradenton, Florida 34202
email: antonio.montero@carrier.com<mailto:antonio.montero@carrier.com>

[cid:image001.jpg@01D960A8.EE552530]
http://edwardsfiresafety.com<http://edwardsfiresafety.com/>



-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 586841 bytes
Desc: image001.jpg
Url : https://codesynthesis.com/pipermail/odb-users/attachments/20230327/f6aa0303/image001-0001.jpg
From boris at codesynthesis.com  Tue Mar 28 06:59:14 2023
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Tue Mar 28 06:51:27 2023
Subject: [odb-users] New distribution pkgs for ODB
In-Reply-To: <BYAPR06MB38957B31E1ECA38B3FAD4A76BC8B9@BYAPR06MB3895.namprd06.prod.outlook.com>
References: <MWHPR06MB2368B3AAAD5C33D7CF7580F2FB849@MWHPR06MB2368.namprd06.prod.outlook.com>
	<BYAPR06MB38957B31E1ECA38B3FAD4A76BC8B9@BYAPR06MB3895.namprd06.prod.outlook.com>
Message-ID: <boris.20230328125104@codesynthesis.com>

Tenneti, Rohith <Rohith.Tenneti2@carrier.com> writes:

> Could you share the distribution packages for odb, libodb, and
> libodb-sqlite for 2.5.0-b.23.
> 
> Even though I followed the INSTALL-GIT and INSTALL instructions
> on each package I was not able to 'make dist' for any of the
> relevant interested pkgs ( odb, libodb, libodb-sqlite ).

The recommended way to build ODB from source is described here:

https://codesynthesis.com/products/odb/doc/install-build2.xhtml


> What we need:
>
> Distribution pkgs for each of the following components from .git tag
> or latest master if stable. ( we are looking for odb dist source and
> libs that support -std=c++17 )

If you would like to build libodb and libodb-sqlite with -std=c++17
(by default they are built with the latest available standard supported
by the compiler that you are using), you can add config.cxx.std=17 to
the `bpkg create` command on the "Building ODB Runtime Libraries" step
in the above instructions.

Let me know if you run into any issues.

From Rohith.Tenneti2 at carrier.com  Tue Mar 28 15:48:28 2023
From: Rohith.Tenneti2 at carrier.com (Tenneti, Rohith)
Date: Wed Mar 29 03:38:57 2023
Subject: [External]Re: [odb-users] New distribution pkgs for ODB
In-Reply-To: <boris.20230328125104@codesynthesis.com>
References: <MWHPR06MB2368B3AAAD5C33D7CF7580F2FB849@MWHPR06MB2368.namprd06.prod.outlook.com>
	<BYAPR06MB38957B31E1ECA38B3FAD4A76BC8B9@BYAPR06MB3895.namprd06.prod.outlook.com>
	<boris.20230328125104@codesynthesis.com>
Message-ID: <BYAPR06MB3895564F2273C863D2B3DF9FBC889@BYAPR06MB3895.namprd06.prod.outlook.com>


How would one specify a prefix in the build2.config options when cross compiling for another target so that the build/compile is able to locate the headers and libs for the cross compile of libodb and libodb-sqlite?

Thanks,
Rohith



-----Original Message-----
From: Boris Kolpackov <boris@codesynthesis.com> 
Sent: Tuesday, March 28, 2023 6:59 AM
To: Tenneti, Rohith <Rohith.Tenneti2@carrier.com>
Cc: odb-users@codesynthesis.com; Montero, Antonio <antonio.montero@carrier.com>
Subject: [External]Re: [odb-users] New distribution pkgs for ODB

[Some people who received this message don't often get email from boris@codesynthesis.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Tenneti, Rohith <Rohith.Tenneti2@carrier.com> writes:

> Could you share the distribution packages for odb, libodb, and 
> libodb-sqlite for 2.5.0-b.23.
>
> Even though I followed the INSTALL-GIT and INSTALL instructions on 
> each package I was not able to 'make dist' for any of the relevant 
> interested pkgs ( odb, libodb, libodb-sqlite ).

The recommended way to build ODB from source is described here:

https://codesynthesis.com/products/odb/doc/install-build2.xhtml


> What we need:
>
> Distribution pkgs for each of the following components from .git tag 
> or latest master if stable. ( we are looking for odb dist source and 
> libs that support -std=c++17 )

If you would like to build libodb and libodb-sqlite with -std=c++17 (by default they are built with the latest available standard supported by the compiler that you are using), you can add config.cxx.std=17 to the `bpkg create` command on the "Building ODB Runtime Libraries" step in the above instructions.

Let me know if you run into any issues.

From boris at codesynthesis.com  Wed Mar 29 03:55:48 2023
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Wed Mar 29 03:48:02 2023
Subject: [External]Re: [odb-users] New distribution pkgs for ODB
In-Reply-To: <BYAPR06MB3895564F2273C863D2B3DF9FBC889@BYAPR06MB3895.namprd06.prod.outlook.com>
References: <MWHPR06MB2368B3AAAD5C33D7CF7580F2FB849@MWHPR06MB2368.namprd06.prod.outlook.com>
	<BYAPR06MB38957B31E1ECA38B3FAD4A76BC8B9@BYAPR06MB3895.namprd06.prod.outlook.com>
	<boris.20230328125104@codesynthesis.com>
	<BYAPR06MB3895564F2273C863D2B3DF9FBC889@BYAPR06MB3895.namprd06.prod.outlook.com>
Message-ID: <boris.20230329094714@codesynthesis.com>

Tenneti, Rohith <Rohith.Tenneti2@carrier.com> writes:

> How would one specify a prefix in the build2.config options when
> cross compiling for another target so that the build/compile is
> able to locate the headers and libs for the cross compile of libodb
> and libodb-sqlite?

There is a note on cross-compiling at the beginning of the instructions.
If cross-compiling requires that you specify additional header and/or
library search paths, then you can specify them with config.cc.poptions
and config.cc.loptions, respectively. For example:

$ bpkg create -d libodb-gcc-aarch64 cc          \
  config.cxx=aarch64-linux-gnu-g++              \
  config.cc.coptions=-O3                        \
  config.cc.poptions=-I/opt/gcc-aarch64/include \
  config.cc.loptions=-L/opt/gcc-aarch64/lib     \
  config.install.root=/opt/gcc-aarch64          \
  config.install.sudo=sudo

You can specify multiple search paths like this:

config.cc.poptions="-I/opt/gcc-common/include -I/opt/gcc-aarch64/include"

From antonio.montero at carrier.com  Wed Mar 29 17:48:03 2023
From: antonio.montero at carrier.com (Montero, Antonio)
Date: Thu Mar 30 06:47:35 2023
Subject: [External]Re: [odb-users] New distribution pkgs for ODB
In-Reply-To: <boris.20230329094714@codesynthesis.com>
References: <MWHPR06MB2368B3AAAD5C33D7CF7580F2FB849@MWHPR06MB2368.namprd06.prod.outlook.com>
	<BYAPR06MB38957B31E1ECA38B3FAD4A76BC8B9@BYAPR06MB3895.namprd06.prod.outlook.com>
	<boris.20230328125104@codesynthesis.com>
	<BYAPR06MB3895564F2273C863D2B3DF9FBC889@BYAPR06MB3895.namprd06.prod.outlook.com>
	<boris.20230329094714@codesynthesis.com>
Message-ID: <MWHPR06MB23685DEC9618B6655B738797FB899@MWHPR06MB2368.namprd06.prod.outlook.com>

Thank you for the response. I was able to build (cross compile) the corresponding pkgs.
Couple of items:

1- I was not successful in configuring the libodb-sqlite build to use an unpackaged version of libsqlite3 rather than pull the repo source and build from source.
It was not obvious in the documentation how to cleanly accomplish this. I tried with ?sys however that at the moment appears to default to think it is a system provided pkg and adds /usr/local paths to the compiler which then causes issues when cross compiling as it tries to pick up wchar.h from x86_64 when cross compiling for armv7. Per the documentation I attempted to specify the ?sys:sqlite3/<version> but that did not work either. It complained that it did not satisfied the dependency. I just ended up building using the build from source option which at the moment is good enough. Our application build system is Yocto based and sqlite3 is being built through yocto recipe and its different version hence why I wanted to link libodb-sqlite with that already installed version.

2- Is there a way to strip .so libs via build2 configuration? I could not find an option to cause 'strip --strip-all' to be called on the created '.so' libs

BR,
Antonio Montero
Staff Engineer, Gen Engrg, Panels NPD
Robust Design Lead (RDL) Systems,
Commercial Fire - Edwards&Barcelona
8985 Town Center Parkway
Bradenton, Florida 34202
email: antonio.montero@carrier.com


http://edwardsfiresafety.com




-----Original Message-----
From: Boris Kolpackov <boris@codesynthesis.com> 
Sent: Wednesday, March 29, 2023 3:56 AM
To: Tenneti, Rohith <Rohith.Tenneti2@carrier.com>
Cc: odb-users@codesynthesis.com; Montero, Antonio <antonio.montero@carrier.com>
Subject: Re: [External]Re: [odb-users] New distribution pkgs for ODB

[You don't often get email from boris@codesynthesis.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Tenneti, Rohith <Rohith.Tenneti2@carrier.com> writes:

> How would one specify a prefix in the build2.config options when cross 
> compiling for another target so that the build/compile is able to 
> locate the headers and libs for the cross compile of libodb and 
> libodb-sqlite?

There is a note on cross-compiling at the beginning of the instructions.
If cross-compiling requires that you specify additional header and/or library search paths, then you can specify them with config.cc.poptions and config.cc.loptions, respectively. For example:

$ bpkg create -d libodb-gcc-aarch64 cc          \
  config.cxx=aarch64-linux-gnu-g++              \
  config.cc.coptions=-O3                        \
  config.cc.poptions=-I/opt/gcc-aarch64/include \
  config.cc.loptions=-L/opt/gcc-aarch64/lib     \
  config.install.root=/opt/gcc-aarch64          \
  config.install.sudo=sudo

You can specify multiple search paths like this:

config.cc.poptions="-I/opt/gcc-common/include -I/opt/gcc-aarch64/include"

From boris at codesynthesis.com  Thu Mar 30 07:05:06 2023
From: boris at codesynthesis.com (Boris Kolpackov)
Date: Thu Mar 30 06:57:17 2023
Subject: [External]Re: [odb-users] New distribution pkgs for ODB
In-Reply-To: <MWHPR06MB23685DEC9618B6655B738797FB899@MWHPR06MB2368.namprd06.prod.outlook.com>
References: <MWHPR06MB2368B3AAAD5C33D7CF7580F2FB849@MWHPR06MB2368.namprd06.prod.outlook.com>
	<BYAPR06MB38957B31E1ECA38B3FAD4A76BC8B9@BYAPR06MB3895.namprd06.prod.outlook.com>
	<boris.20230328125104@codesynthesis.com>
	<BYAPR06MB3895564F2273C863D2B3DF9FBC889@BYAPR06MB3895.namprd06.prod.outlook.com>
	<boris.20230329094714@codesynthesis.com>
	<MWHPR06MB23685DEC9618B6655B738797FB899@MWHPR06MB2368.namprd06.prod.outlook.com>
Message-ID: <boris.20230330125602@codesynthesis.com>

Montero, Antonio <antonio.montero@carrier.com> writes:

> Per the documentation I attempted to specify the ?sys:sqlite3/<version> [...]

That should be ?sys:libsqlite3. There is actually an example of the
correct command line shown in the instructions:

$ bpkg build libodb-sqlite ?sys:libsqlite3

If the location of the libsqlite3 header/library that you would like to
use is not in the compiler's default search paths (or the ones you have
already added with -I/-L), then you will need to add the suitable -I/-L
options; build2 will only look for the library in the compiler's build-in
search paths plus what you have specified with -L. One thing that may
throw things off is a broken pkg-config (.pc) file that has something
like -I/usr/include hardcoded in it. So you may want to just removing it
(libsqlite3 should link fine without it).


> 2- Is there a way to strip .so libs via build2 configuration? I
> could not find an option to cause 'strip --strip-all' to be called
> on the created '.so' libs

No, there is no way to cause this. But perhaps adding the following
will have the same effect:

config.cc.loptions=-Wl,--strip-all