From alain.mari2 at free.fr Thu Jan 10 16:44:37 2019 From: alain.mari2 at free.fr (alain.mari2@free.fr) Date: Fri Jan 11 08:42:12 2019 Subject: [odb-users] Unable to generate .sql files with odb and sqlite database In-Reply-To: <463742892.142430684.1547155946031.JavaMail.root@zimbra33-e6.priv.proxad.net> Message-ID: <71054920.142549215.1547156677567.JavaMail.root@zimbra33-e6.priv.proxad.net> Hello, When using odb with SQL Server, everything is fine when compiling C++ headers with odb options : odb.exe --std c++11 --database mssql --generate-schema --generate-query --table-prefix mytab_ myheader.hxx The .sql files are correctly generated, and I can create my schema. But when I am using odb with sqlite and this command : odb.exe --std c++11 --database sqlite --generate-schema --generate-query --table-prefix mytab_ myheader.hxx No .sql files are generated ! How can I do to create .sql files and create tables is my sqlite db file (I don't want to use sqlite 'in memory') ? .sql created with SQL Server are not compatible with sqlite ... Regards. Alain. From boris at codesynthesis.com Fri Jan 11 08:30:55 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Jan 11 08:44:28 2019 Subject: [odb-users] Unable to generate .sql files with odb and sqlite database In-Reply-To: <71054920.142549215.1547156677567.JavaMail.root@zimbra33-e6.priv.proxad.net> References: <463742892.142430684.1547155946031.JavaMail.root@zimbra33-e6.priv.proxad.net> <71054920.142549215.1547156677567.JavaMail.root@zimbra33-e6.priv.proxad.net> Message-ID: alain.mari2@free.fr writes: > But when I am using odb with sqlite and this command : > > [...] > > No .sql files are generated ! See the --generate-schema option documentation in odb(1): https://codesynthesis.com/products/odb/doc/odb.xhtml From alain.mari at se.com Mon Jan 14 11:04:14 2019 From: alain.mari at se.com (ALAIN MARI) Date: Tue Jan 15 05:05:30 2019 Subject: [odb-users] Change-tracking map, list and set containers Message-ID: Hello ! We are currently evaluating odb (either with SQlite or MSSQL as backend) as a replacement of our object database for our software in Schneider Electric. We started to make many small examples that work really fine. But our main concern is performances ! In particular, we have many classes with huge std::map containers inside. I saw in the SQL execution plan that an update of such a class required the full rebuilt of the entire jointure table in SQL db. Is there some kind of change-tracking or smart map container in a future version of ODB (same as vector) ? A request had been made in 2014 but apparently not followed up. We mainly use std::vector, std::map, std::list and std::set in our software, but std::map and std::list are the most urgent. Best regards. Alain. From boris at codesynthesis.com Tue Jan 15 06:48:27 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Jan 15 07:02:13 2019 Subject: [odb-users] Change-tracking map, list and set containers In-Reply-To: References: Message-ID: ALAIN MARI writes: > In particular, we have many classes with huge std::map containers inside. > I saw in the SQL execution plan that an update of such a class required > the full rebuilt of the entire jointure table in SQL db. > > Is there some kind of change-tracking or smart map container in a future > version of ODB (same as vector)? We looked into it but change-tracking containers are not exactly trivial to implement, especially if you want to do this efficiently. Look at the vector's implementation to get a sense of the complexity involved. On the other hand, if you make some application-specific assumptions (e.g., elements are never removed) or accept some inefficiency, then an application-specific implementation can be provided relatively easily. Another interesting possibility is to come up with some kind of a hybrid container that uses the change-tracking vector for database interfacing and std::map as its API (from the database storage point of view, map is just a sequence). How well this will work depends on how you use your maps (again, application-specific knowledge). For example, if all the modifications go through your object's API, then you could just add the vector data member next to the map and maintain both in parallel. From alain.mari at se.com Tue Jan 15 08:46:43 2019 From: alain.mari at se.com (ALAIN MARI) Date: Tue Jan 15 09:02:05 2019 Subject: [odb-users] Change-tracking map, list and set containers In-Reply-To: References: Message-ID: Thanks Boris for your answer. I will ask our engineers to see how containers are used in our software (5 million loc ...) and I will revert back to you if we have special needs. Sometimes, maps can be advantageously replaced by sorted vectors. Besides, I will try to maintain a vector/map in parallel in some of our persistent objects but I'm afraid of memory consumption in that case. I will also look inside your odb::vector implementation to get a sense of the complexity, and to see I we can adapt it to std::list. Regards. Alain. ALAIN MARI writes: > In particular, we have many classes with huge std::map containers inside. > I saw in the SQL execution plan that an update of such a class > required the full rebuilt of the entire jointure table in SQL db. > > Is there some kind of change-tracking or smart map container in a > future version of ODB (same as vector)? We looked into it but change-tracking containers are not exactly trivial to implement, especially if you want to do this efficiently. Look at the vector's implementation to get a sense of the complexity involved. On the other hand, if you make some application-specific assumptions (e.g., elements are never removed) or accept some inefficiency, then an application-specific implementation can be provided relatively easily. Another interesting possibility is to come up with some kind of a hybrid container that uses the change-tracking vector for database interfacing and std::map as its API (from the database storage point of view, map is just a sequence). How well this will work depends on how you use your maps (again, application-specific knowledge). For example, if all the modifications go through your object's API, then you could just add the vector data member next to the map and maintain both in parallel. From boris at codesynthesis.com Tue Jan 15 09:04:01 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Jan 15 09:17:48 2019 Subject: [odb-users] Change-tracking map, list and set containers In-Reply-To: References: Message-ID: ALAIN MARI writes: > Sometimes, maps can be advantageously replaced by sorted vectors. True. Or even unsorted (if you normally have just a handful of elements, linear search over an unsorted vector can be faster that map lookup due to data locality). > Besides, I will try to maintain a vector/map in parallel in some of > our persistent objects but I'm afraid of memory consumption in that > case. You can always get clever and maintain a pointer to the map's element in the vector or some such. It will require a bit of extra mapping effort for ODB, but otherwise should be doable. > I will also look inside your odb::vector implementation to get a > sense of the complexity, and to see I we can adapt it to std::list. Yes, I think std::list we can handle using odb::vector's implementation (vector_impl); off the top of my head I don't see any good reasons to replicate the list's storage model for the change state. From alain.mari at se.com Tue Jan 15 11:35:27 2019 From: alain.mari at se.com (ALAIN MARI) Date: Wed Jan 16 08:48:20 2019 Subject: [odb-users] Change-tracking map, list and set containers In-Reply-To: References: Message-ID: Actually, we have big maps (up to 1 million elements) in which we make lots of lookups. So it is mandatory to have sorted vector (persistent) to mimic the map behavior. Unfortunately, when I want to apply standard algorithm std::sort, on odb::vector, compilation fails with this message : 1>c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\algorithm(3835): error C3892: '_First': you cannot assign to a variable that is const 1>c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\algorithm(4023): note: see reference to function template instantiation '_BidIt std::_Insertion_sort_unchecked<_RanIt,_Pr>(_BidIt,const _BidIt,_Pr)' being compiled 1> with 1> [ 1> _BidIt=odb::vector_iterator>,std::_Vector_iterator>>>, 1> _RanIt=odb::vector_iterator>,std::_Vector_iterator>>>, 1> _Pr=std::less 1> ] -----Message d'origine----- De?: Boris Kolpackov [mailto:boris@codesynthesis.com] Envoy??: mardi 15 janvier 2019 15:04 ??: ALAIN MARI Cc?: odb-users@codesynthesis.com Objet?: Re: [odb-users] Change-tracking map, list and set containers [External email: Use caution with links and attachments] ________________________________ ALAIN MARI writes: > Sometimes, maps can be advantageously replaced by sorted vectors. True. Or even unsorted (if you normally have just a handful of elements, linear search over an unsorted vector can be faster that map lookup due to data locality). > Besides, I will try to maintain a vector/map in parallel in some of > our persistent objects but I'm afraid of memory consumption in that > case. You can always get clever and maintain a pointer to the map's element in the vector or some such. It will require a bit of extra mapping effort for ODB, but otherwise should be doable. > I will also look inside your odb::vector implementation to get a sense > of the complexity, and to see I we can adapt it to std::list. Yes, I think std::list we can handle using odb::vector's implementation (vector_impl); off the top of my head I don't see any good reasons to replicate the list's storage model for the change state. ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________ From alain.mari at se.com Thu Jan 17 03:26:14 2019 From: alain.mari at se.com (ALAIN MARI) Date: Thu Jan 17 04:45:57 2019 Subject: [odb-users] Change-tracking map, list and set containers In-Reply-To: References: Message-ID: Sorry, but I didn't see that the answer to that question is in the documentation ! -----Message d'origine----- De?: odb-users-bounces@codesynthesis.com [mailto:odb-users-bounces@codesynthesis.com] De la part de ALAIN MARI Envoy??: mardi 15 janvier 2019 17:35 ??: odb-users@codesynthesis.com Objet?: RE: [odb-users] Change-tracking map, list and set containers [External email: Use caution with links and attachments] ________________________________ Actually, we have big maps (up to 1 million elements) in which we make lots of lookups. So it is mandatory to have sorted vector (persistent) to mimic the map behavior. Unfortunately, when I want to apply standard algorithm std::sort, on odb::vector, compilation fails with this message : 1>c:\program files (x86)\microsoft visual 1>studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\algorithm(3835): error C3892: '_First': you cannot assign to a variable that is const c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\algorithm(4023): note: see reference to function template instantiation '_BidIt std::_Insertion_sort_unchecked<_RanIt,_Pr>(_BidIt,const _BidIt,_Pr)' being compiled 1> with 1> [ 1> _BidIt=odb::vector_iterator>,std::_Vector_iterator>>>, 1> _RanIt=odb::vector_iterator>,std::_Vector_iterator>>>, 1> _Pr=std::less 1> ] -----Message d'origine----- De : Boris Kolpackov [mailto:boris@codesynthesis.com] Envoy? : mardi 15 janvier 2019 15:04 ? : ALAIN MARI Cc : odb-users@codesynthesis.com Objet : Re: [odb-users] Change-tracking map, list and set containers [External email: Use caution with links and attachments] ________________________________ ALAIN MARI writes: > Sometimes, maps can be advantageously replaced by sorted vectors. True. Or even unsorted (if you normally have just a handful of elements, linear search over an unsorted vector can be faster that map lookup due to data locality). > Besides, I will try to maintain a vector/map in parallel in some of > our persistent objects but I'm afraid of memory consumption in that > case. You can always get clever and maintain a pointer to the map's element in the vector or some such. It will require a bit of extra mapping effort for ODB, but otherwise should be doable. > I will also look inside your odb::vector implementation to get a sense > of the complexity, and to see I we can adapt it to std::list. Yes, I think std::list we can handle using odb::vector's implementation (vector_impl); off the top of my head I don't see any good reasons to replicate the list's storage model for the change state. ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________ ______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. ______________________________________________________________________ From sharperm at 163.com Sun Jan 20 21:50:28 2019 From: sharperm at 163.com (sharper) Date: Mon Jan 21 07:05:51 2019 Subject: [odb-users] "MYSQL_BIND" :redefined; Message-ID: <2d5b5e46.4c7d.1686e50fd90.Coremail.sharperm@163.com> hi, I just begin use the odb library with mysql. I have successfully compiled odb library, when I compile the libodb-mysql there comes some error, like this:"MYSQL_BIND":redefine and "my_bool *" can not cast to "bool *". i locate the error file is "mysql-types.hxx", I used the odblib version is 2.4.0, and mysql version is 8.0.12. can you help me to solve these problems. thanks!!! -- ------------------------------------------------------------------------------------------------------------------------------------------------ Name? Sharperm Time?2019?1?21? ------------------------------------------------------------------------------------------------------------------------------------------------ From boris at codesynthesis.com Mon Jan 21 06:54:22 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Jan 21 07:08:29 2019 Subject: [odb-users] "MYSQL_BIND" :redefined; In-Reply-To: <2d5b5e46.4c7d.1686e50fd90.Coremail.sharperm@163.com> References: <2d5b5e46.4c7d.1686e50fd90.Coremail.sharperm@163.com> Message-ID: sharper writes: > when I compile the libodb-mysql there comes some error, > like this:"MYSQL_BIND":redefine and "my_bool *" can not cast to "bool *". > i locate the error file is "mysql-types.hxx", > I used the odblib version is 2.4.0, and mysql version is 8.0.12. Can you try the latest pre-release version by following these build instructions and letting me know if you have the same issue: https://codesynthesis.com/products/odb/doc/install-build2.xhtml If the issue is still there, make sure to paste the exact error message with location information, etc. From sharperm at 163.com Mon Jan 21 21:07:00 2019 From: sharperm at 163.com (sharper) Date: Tue Jan 22 04:03:47 2019 Subject: [odb-users] "MYSQL_BIND" :redefined; In-Reply-To: References: <2d5b5e46.4c7d.1686e50fd90.Coremail.sharperm@163.com> Message-ID: <13644b78.22b1.168734f905b.Coremail.sharperm@163.com> Hi Thanks for your resolution for my issue. I have compiled successfully in mysql version 5.7.x. After compared mysql 5.7.x with mysql 8.0.12, I find that "my_bool" is no longer exist in the file of mysql.h at mysql version 8.0.12, And "MYSQL_BIND" had defined in the file of mysql.h at mysql version 8.0.12, not defined in mysql version 5.7.x. I will also try the mothed that you gived for me. thanks again. -- ------------------------------------------------------------------------------------------------------------------------------------------------ Name? Sharperm Time?2019?1?22? ------------------------------------------------------------------------------------------------------------------------------------------------ At 2019-01-21 19:54:22, "Boris Kolpackov" wrote: >sharper writes: > >> when I compile the libodb-mysql there comes some error, >> like this:"MYSQL_BIND":redefine and "my_bool *" can not cast to "bool *". >> i locate the error file is "mysql-types.hxx", >> I used the odblib version is 2.4.0, and mysql version is 8.0.12. > >Can you try the latest pre-release version by following these build >instructions and letting me know if you have the same issue: > >https://codesynthesis.com/products/odb/doc/install-build2.xhtml > >If the issue is still there, make sure to paste the exact error >message with location information, etc. From carlo.canobbio at hsi.ch Mon Jan 21 02:11:44 2019 From: carlo.canobbio at hsi.ch (Carlo CANOBBIO) Date: Tue Jan 22 04:03:47 2019 Subject: [odb-users] ODBC Generic Message-ID: <3930bda3b98d45b690fa2fbb023b14f4@hsi.ch> Hi, great work, thanks. I have seen that the support of MS SQL is made through "SQL Server Native Client" ODBC driver. I have tried to use it with Pervasive SQL but the MS SQL syntax is not compatible with it. You think that is it a good idea to develop a generic ODBC support with personalizable syntax ? Thank you Carlo CANOBBIO Software Engineer HSI SIRIO SA Via Carlo Maderno 42a 6850 Mendrisio Switzerland +41 91 612 2121 (telephone) +41 91 612 2122 (telefax) carlo,canobbio@hsi.ch www.hsi.ch From boris at codesynthesis.com Tue Jan 22 08:15:54 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Jan 22 08:30:03 2019 Subject: [odb-users] ODBC Generic In-Reply-To: <3930bda3b98d45b690fa2fbb023b14f4@hsi.ch> References: <3930bda3b98d45b690fa2fbb023b14f4@hsi.ch> Message-ID: Carlo CANOBBIO writes: > I have seen that the support of MS SQL is made through "SQL Server > Native Client" ODBC driver. I have tried to use it with Pervasive > SQL but the MS SQL syntax is not compatible with it. Right, even though the API is ODBC-based, it is very specific to MSSQL. > You think that is it a good idea to develop a generic ODBC support > with personalizable syntax ? I don't believe this will work out very well. Experience shows that anything beyond basic functionality requires tight knowledge/integration with the database in question, especially if the goal is to achieve optimal performance. From sharperm at 163.com Wed Jan 23 02:11:43 2019 From: sharperm at 163.com (sharper) Date: Wed Jan 23 03:54:27 2019 Subject: [odb-users] Data too long for column "XXX" at row 1. Message-ID: <537789f5.8c5f.168798ce40a.Coremail.sharperm@163.com> Hi I defined a std::vector data type as a blob,the maximum memory size of the data i need is 16384 * sizeof(float). when i execute the code "db->persist(db_object )",it throw an error "throw database_exception (e, sqlstate, msg);" the value of "e" is 1406, the value of "sqlstate" is 22001, the value of "msg" is Data too long for column "XXX" at row 1. Could you help me to solve this problem? -- ------------------------------------------------------------------------------------------------------------------------------------------------ Name? Sharperm Time?2019?1?22? ------------------------------------------------------------------------------------------------------------------------------------------------ From troy.heron at hixxy.org Wed Jan 23 02:21:52 2019 From: troy.heron at hixxy.org (Troy Heron) Date: Wed Jan 23 03:54:27 2019 Subject: [odb-users] Relationships and constraints without pointers Message-ID: Hello, Let's say I have these 2 hypothetical objects: struct Object1 { std::set values; }; struct Object2 { std::set values; }; Is it possible for me to create a relationship and constraints using pragmas and no additional objects or pointers such that the values in Object2::values can only be those in Object1::values and that Object2::values is updated when a value is removed from Object1::values or when the value is changed? Regards, Troy From boris at codesynthesis.com Wed Jan 23 08:21:04 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Jan 23 08:35:17 2019 Subject: [odb-users] Relationships and constraints without pointers In-Reply-To: References: Message-ID: Troy Heron writes: > struct Object1 > { > std::set values; > }; > > struct Object2 > { > std::set values; > }; > > Is it possible for me to create a relationship and constraints using > pragmas and no additional objects or pointers [...] In 2.5.0[1] there is the new points_to pragma for that. For example: struct Object1 { #pragma db points_to(Object2) std::string value; }; It does not yet support containers of pointers, however. > [...] such that the values in Object2::values can only be those in > Object1::values and that Object2::values is updated when a value is > removed from Object1::values or when the value is changed? I assume you are alluding to the ON DELETE/ON UPDATE mechanism. There is the on_delete pragma but there is no on_update. [1] https://codesynthesis.com/products/odb/doc/install-build2.xhtml From troy.heron at hixxy.org Thu Jan 31 00:48:38 2019 From: troy.heron at hixxy.org (Troy Heron) Date: Thu Jan 31 07:15:55 2019 Subject: [odb-users] Relationships and constraints without pointers In-Reply-To: References: Message-ID: Thank you. Do you know when the new version will be ready for release? On Wed, 23 Jan 2019 at 23:21, Boris Kolpackov wrote: > Troy Heron writes: > > > struct Object1 > > { > > std::set values; > > }; > > > > struct Object2 > > { > > std::set values; > > }; > > > > Is it possible for me to create a relationship and constraints using > > pragmas and no additional objects or pointers [...] > > In 2.5.0[1] there is the new points_to pragma for that. For example: > > struct Object1 > { > #pragma db points_to(Object2) > std::string value; > }; > > It does not yet support containers of pointers, however. > > > > [...] such that the values in Object2::values can only be those in > > Object1::values and that Object2::values is updated when a value is > > removed from Object1::values or when the value is changed? > > I assume you are alluding to the ON DELETE/ON UPDATE mechanism. There > is the on_delete pragma but there is no on_update. > > > [1] https://codesynthesis.com/products/odb/doc/install-build2.xhtml > From boris at codesynthesis.com Thu Jan 31 07:11:25 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Jan 31 07:26:03 2019 Subject: [odb-users] Relationships and constraints without pointers In-Reply-To: References: Message-ID: Troy Heron writes: > Do you know when the new version will be ready for release? Yes, the release is overdue. The reason for the delay is that we are moving to a uniform build/packaging toolchain as well as more of "continuous delivery" model in the future. I would recommend that you do not wait for the official release and start using the new setup as described here: https://codesynthesis.com/products/odb/doc/install-build2.xhtml At this stage in the development cycle, it is as stable as 2.4.0, we are using it ourselves in multiple projects, we have migrated a number of users with very good results, and it is fully supported (i.e., we provide the same level of support as for 2.4.0).