From rccraigb at gmail.com Sun Jun 7 17:09:05 2020 From: rccraigb at gmail.com (Craig Burton) Date: Sun Jun 7 17:21:46 2020 Subject: [odb-users] ODB exception on insert (ODB 2.4.0) Message-ID: Hi, I am encountering the following exception upon attempting to insert a derived class that inherits from two classes, one defined as polymorphic, the other as abstract: 1136 (21S01): Column count doesn't match value count at row 1 The insert statement is reproducible and is generated as follows: const char access::object_traits_impl< ::PersistentDerived, id_mysql >::persist_statement[] = "INSERT INTO `PersistentDerived` " "(`id`, " "`mDerived`) " "VALUES " "(?, ?, ?)"; Perhaps I am defining the persistence mapping incorrectly, or this issue has been addressed in 2.5, but I thought I would check. The class definitions that reproduce this behavior are as follows: (file abstract.hxx) =================== #include #include #pragma db object polymorphic class PersistentBase { public: PersistentBase() {} virtual ~PersistentBase() {} protected: friend class odb::access; #pragma db id auto unsigned long id_; std::string mBase; }; #pragma db object abstract class AbstractBase { public: AbstractBase() {} virtual ~AbstractBase() {} protected: friend class odb::access; std::string mAbstractBase; }; #pragma db object class PersistentDerived : public PersistentBase, AbstractBase { public: PersistentDerived() {} ~PersistentDerived() {} protected: friend class odb::access; std::string mDerived; }; This command was used to compile: odb --database mysql --generate-schema --generate-query --generate-session abstract.hxx Thank you in advance for taking a look. Craig From boris at codesynthesis.com Mon Jun 8 09:42:48 2020 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Jun 8 09:55:12 2020 Subject: [odb-users] ODB exception on insert (ODB 2.4.0) In-Reply-To: References: Message-ID: Craig Burton writes: > I am encountering the following exception upon attempting to insert a > derived class that inherits from two classes, one defined as polymorphic, > the other as abstract: [...] I've checked and this bug is still present in 2.5.0. However, before fixing it, we would need to understanding whether mixing the two inheritance styles is even something that makes sense and that we want to support. And the answer is likely "no" (this stuff is already complex enough). You can also achieve what you (probably) want with a transient base and a virtual data member: //#pragma db object transient class AbstractBase { ... std::string mAbstractBase; }; #pragma db object class PersistentDerived : public PersistentBase, AbstractBase { ... #pragma db member(mAbstractBase_) virtual(std::string) access(mAbstractBase) std::string mDerived; }; From rccraigb at gmail.com Tue Jun 9 17:27:18 2020 From: rccraigb at gmail.com (Craig Burton) Date: Tue Jun 9 17:40:06 2020 Subject: [odb-users] ODB exception on insert (ODB 2.4.0) In-Reply-To: References: Message-ID: Thank you for looking into this, Boris. I am able to work around this issue with only a little effort, I just wanted to make sure I was attempting to do it correctly. Thank you (and others at Code Synthesis) for a fantastic product! Craig On Mon, Jun 8, 2020 at 6:42 AM Boris Kolpackov wrote: > Craig Burton writes: > > > I am encountering the following exception upon attempting to insert a > > derived class that inherits from two classes, one defined as polymorphic, > > the other as abstract: [...] > > I've checked and this bug is still present in 2.5.0. > > However, before fixing it, we would need to understanding whether > mixing the two inheritance styles is even something that makes > sense and that we want to support. And the answer is likely "no" > (this stuff is already complex enough). > > You can also achieve what you (probably) want with a transient base > and a virtual data member: > > //#pragma db object transient > class AbstractBase > { > ... > > std::string mAbstractBase; > }; > > #pragma db object > class PersistentDerived > : public PersistentBase, AbstractBase > { > ... > > #pragma db member(mAbstractBase_) virtual(std::string) > access(mAbstractBase) > > std::string mDerived; > }; > > From seu.hanbin at gmail.com Wed Jun 10 09:30:19 2020 From: seu.hanbin at gmail.com (=?UTF-8?B?6Z+p5b2s?=) Date: Thu Jun 11 09:46:56 2020 Subject: [odb-users] Global Custom Session in Multi-Threading Application Message-ID: Hello, We are building our application based on odb-2.4.0 and PostgreSQL (with ibodb-pgsql-2.4.0), and we are trying to implement a *global* *thread-safe custom session rather than the default thread-local-session* in our application according to ODB manual 11.2 and the custom session example in odb-tests. Basically our custom session uses a thread-safe tbb::cocurrent_hash_map as object map like this, > template > struct object_map: object_map_base, > tbb::cocurrent_hash_map::id_type, > object_data > > { }; > > And I also add a global mutex on each access operation to the custom session, like this. So the access to our custom session and object map are guarantee to be thread-safe. > static std::mutex _mtx; _cache_insert (){ std::lock_guard _lock(_mtx); //.... } _cache_find () { std::lock_guard _lock((_mtx) ; //.... } _cache_erase () > { std::lock_guard _lock(_mtx); //... } In our application, we have only *one global session *and *multiple threads. *In each thread , we have *one transaction *to* load DATA * from PostgreSQL. Those *DATA* *may have a shared_ptr point to the same object. But we found that sometimes the shared_ptr will be initialized with some weird number instead of the real obj in db. * *db pragma ...* *struct DATA{* * //...* * std::shared_ptr obj;* *};* *class Application{* * session s;* * void parallel_update_db() {* * tbb::parallel_for(* * //....* * transaction t;* * // load DATA* * );* * }* *};* We noticed that after ODB calls the* _cache_insert *operation in our session, *the inserted object is still not initialized with the real data in db, and according to line 89 in *the load method in libodb-pgsql *, actually the inserted object will actually be initialized after the _cache_insert operation. So given that we have two threads load and insert the same object to our object map, would one thread end up with an invalid object pointer? Or are there any possible issue with my implementation and usage of custom session?* From boris at codesynthesis.com Thu Jun 11 09:52:45 2020 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Jun 11 10:05:20 2020 Subject: [odb-users] Global Custom Session in Multi-Threading Application In-Reply-To: References: Message-ID: ?? writes: > We noticed that after ODB calls the _cache_insert operation in our > session, the inserted object is still not initialized with the real data > in db, and according to line 89 in the load method in libodb-pgsql > , > actually the inserted object will actually be initialized after the > _cache_insert operation. So given that we have two threads load and insert > the same object to our object map, would one thread end up with an invalid > object pointer? Or are there any possible issue with my implementation and > usage of custom session? Section 11.2, "Custom Sessions" in the manual has some hints on how this can be implemented. Specifically: "The _cache_insert() function shall add the object into the object cache and return its position. The _cache_find() function looks an object up in the object cache given its id. It returns a NULL pointer if the object is not found. The _cache_erase() cache management function shall remove the object from the cache. It is called if the database operation that caused the object to be inserted (for example, load) failed. Note also that after insertion the object state is undefined. You can only access the object state (for example, make a copy or clear a flag) from one of the notification functions discussed below. The notification functions are called after an object has been persisted, loaded, updated, or erased, respectively." This suggests that you need to hold the lock until one of the notification functions is called (the custom cache_position type is there exactly so that you could propagate something like a lock from _cache_insert() to one of the notification functions). Note also that if you use a single mutex for this, you will probably end up with pretty poor performance and also probably won't be able to have any object pointers (because their loading would trigger a call to _cache_insert() and a recursive attempt to acquire a lock). So I would look into a mutex for the map itself and then a pool of mutexes that are used to protect an object between _cache_insert() and a notification function call. As you can see, this is quite tricky. From Tschoche.Jonas at ddv-mediengruppe.de Tue Jun 16 07:17:20 2020 From: Tschoche.Jonas at ddv-mediengruppe.de (Tschoche.Jonas@ddv-mediengruppe.de) Date: Tue Jun 16 07:35:22 2020 Subject: [odb-users] Schema Migration not working correctly In-Reply-To: References: <5D8FC02A-5DE4-46C2-92D8-231D0D84E76F@ddv-mediengruppe.de> Message-ID: <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> Hello, Sorry for the late response. Also I know that the odb compiler use and maintain the changelog, but when I delete database and the changelog and compile the project first time, no entries are added. When I edit the table schema (adding a new field in C++ Object Model) then a add-table entry appairs in odb changelog ? I think on first build it should have all table definitions and when I adding a field then should it make a alter-table in changelog. I build each with following options: Example for one file: cd PROJDIR && /usr/local/bin/odb -d mysql --generate-query --generate-schema --schema-format separate --std c++17 --include-regex "%(.*)/(.+)%\$2%" --fkeys-deferrable-mode not_deferrable --profile boost/date-time --profile boost/optional --profile boost/uuid --changelog odb-changelog.log --changelog-dir PROJDIR --default-pointer std::shared_ptr --output-dir PROJDIR/cmake-build-debug/odb_gen --hxx-suffix .hpp --ixx-suffix _inline.hpp --cxx-suffix .cpp --odb-file-suffix _odb -I/usr/local/include -I/usr/local/include I tested it many times remove all I added a new field for a table and the table is dedected, but I don?t like to write the initial xml file by hand to make this working. I don?t see the issue. Step by Step: 1. I cleaned the database and schmea_versions table. 2. Reset CMake Cache 3. Delete odb_changelog 4. Complete recompile 5. Look in odb_changelog -> Result: no entries 6. Make a change in a c++ object model 7. Set new versionsnumber (I use a base class for all tables ? see bottom) 1, 1 -> 1, 2 8. Recompile 9. Entry for Table definition is added 10. Run Programm -> Crash on Schema migration, because table exist. BaseClass: #pragma db model version(1, 1) #pragma db object abstract class BaseEntity {} Object: #pragma db object table("TABLE") class TABLE : public BaseEntity {} Von: Paul Stath Datum: Donnerstag, 30. April 2020 um 18:40 An: "odb-users@codesynthesis.com" , "Tschoche, Jonas" Cc: Boris Kolpackov Betreff: RE: [odb-users] Schema Migration not working correctly This message has been archived. View the original item As Boris responded, a small example will go a long way towards allowing others to help. I had a similar experience when I first started using schema migration in my project, maybe you are having the same issue? In the "C++ Object Persistence with ODB" manual in the "Object Model Version and Changelog" section (https://www.codesynthesis.com/products/odb/doc/manual.xhtml#13.1) is a very important paragraph. (capitalization added for emphasis) "The changelog is maintained by the ODB compiler. Specifically, you do not need to make any manual changes to this file. You will, however, need to KEEP IT AROUND from one invocation of the ODB compiler to the next. In other words, the changelog file is BOTH the INPUT and the OUTPUT of the ODB compiler. This, for example, means that if your project's source code is stored in a version control repository, then you will most likely want to store the changelog there as well. If you delete the changelog, From boris at codesynthesis.com Tue Jun 16 08:29:27 2020 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Jun 16 08:42:17 2020 Subject: [odb-users] Schema Migration not working correctly In-Reply-To: <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> References: <5D8FC02A-5DE4-46C2-92D8-231D0D84E76F@ddv-mediengruppe.de> <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> Message-ID: Tschoche.Jonas@ddv-mediengruppe.de writes: > BaseClass: > #pragma db model version(1, 1) > #pragma db object abstract > class BaseEntity {} > > Object: > > #pragma db object table("TABLE") > class TABLE : public BaseEntity {} At version 1 does your TABLE class have any data members? Try something like this: #pragma db object table("TABLE") class TABLE : public BaseEntity { public: #pragma db id int id; }; Also, if you look into the generated .sql file (again, corresponding to version 1), do you see any 'CREATE TABLE' statements? From Tschoche.Jonas at ddv-mediengruppe.de Tue Jun 16 08:51:02 2020 From: Tschoche.Jonas at ddv-mediengruppe.de (Tschoche.Jonas@ddv-mediengruppe.de) Date: Tue Jun 16 09:15:54 2020 Subject: [odb-users] Schema Migration not working correctly In-Reply-To: References: <5D8FC02A-5DE4-46C2-92D8-231D0D84E76F@ddv-mediengruppe.de> <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> Message-ID: <7F67352D-0F5D-4EEB-9F38-978D7421BDF7@ddv-mediengruppe.de> Yes it was a small exampe the baseentry has id and some other fields and are many other tables with datamembers. I use the 'embedded' (separate option) so I have no sql files - I tried to compile with format option sql and all sql with the right content are generated. When I use the embedded - it creates the database - all it working fine, but not the schmema mirgrations - because of false written odb_changelog. After compiling the code I looked in odb_changelog, it has following content: When I change to #pragma db model version(1, 2) And recompile - it is the same result - only when I add a data member to object then apairs to the odb_changelog After second compile: When adding a new data member: All other Members (exclude MYNEWDATAMEMBER) was available on 1. And 2. Compile. Why it don't reconized the old members on initial generate? I cleaned for this all cache data. ?Am 16.06.20, 14:29 schrieb "Boris Kolpackov" : Tschoche.Jonas@ddv-mediengruppe.de writes: > BaseClass: > #pragma db model version(1, 1) > #pragma db object abstract > class BaseEntity {} > > Object: > > #pragma db object table("TABLE") > class TABLE : public BaseEntity {} At version 1 does your TABLE class have any data members? Try something like this: #pragma db object table("TABLE") class TABLE : public BaseEntity { public: #pragma db id int id; }; Also, if you look into the generated .sql file (again, corresponding to version 1), do you see any 'CREATE TABLE' statements? From boris at codesynthesis.com Tue Jun 16 09:21:00 2020 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Jun 16 09:33:54 2020 Subject: [odb-users] Schema Migration not working correctly In-Reply-To: <7F67352D-0F5D-4EEB-9F38-978D7421BDF7@ddv-mediengruppe.de> References: <5D8FC02A-5DE4-46C2-92D8-231D0D84E76F@ddv-mediengruppe.de> <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> <7F67352D-0F5D-4EEB-9F38-978D7421BDF7@ddv-mediengruppe.de> Message-ID: Try the following minimal test (it's always a good idea to try to reproduce the problem on a minimal test). It works for me and if it works for you, try to see what's different compared to your project. $ cd /tmp $ cat <test.hxx #pragma db model version(1, 1) #pragma db object struct test { #pragma db id auto unsigned long id_; }; EOF $ mkdir -p PROJDIR/cmake-build-debug/odb_gen $ odb -d mysql --generate-query --generate-schema --schema-format separate \ --std c++17 --changelog odb-changelog.log --changelog-dir PROJDIR \ --output-dir PROJDIR/cmake-build-debug/odb_gen test.hxx PROJDIR/odb-changelog.log: info: initializing changelog with base version 1 $ cat PROJDIR/odb-changelog.log
$ cat <test.hxx #pragma db model version(1, 2) #pragma db object struct test { #pragma db id auto unsigned long id_; int data; }; EOF $ odb -d mysql --generate-query --generate-schema --schema-format separate \ --std c++17 --changelog odb-changelog.log --changelog-dir PROJDIR \ --output-dir PROJDIR/cmake-build-debug/odb_gen test.hxx $ cat PROJDIR/odb-changelog.log
From Tschoche.Jonas at ddv-mediengruppe.de Tue Jun 16 12:44:37 2020 From: Tschoche.Jonas at ddv-mediengruppe.de (Tschoche.Jonas@ddv-mediengruppe.de) Date: Wed Jun 17 05:50:02 2020 Subject: [odb-users] Schema Migration not working correctly In-Reply-To: References: <5D8FC02A-5DE4-46C2-92D8-231D0D84E76F@ddv-mediengruppe.de> <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> <7F67352D-0F5D-4EEB-9F38-978D7421BDF7@ddv-mediengruppe.de> Message-ID: I tried it also with my project files. I deleted the odb_changelog and run the same command that is executed by cmake and the table is in changelog, but when I run the same command with the next table header and the odb_changelog is exist - it will nothing append to it. 1. For users: odb -d mysql --generate-query --generate-schema --schema-format separate --std c++17 --include-regex "%(.*)/(.+)%\$2%" --fkeys-deferrable-mode not_deferrable --profile boost/date-time --profile boost/optional --profile boost/uuid --changelog odb-changelog.log --changelog-dir PROJDIR --default-pointer std::shared_ptr --output-dir PROJDIR/cmake-build-debug/odb_gen --hxx-suffix .hpp --ixx-suffix _inline.hpp --cxx-suffix .cpp --odb-file-suffix _odb -I/usr/local/include PROJDIR/src/datamodel/entities/Users.hpp PROJDIR/odb-changelog.log: info: initializing changelog with base version 1 2. For madantor: odb -d mysql --generate-query --generate-schema --schema-format separate --std c++17 --include-regex "%(.*)/(.+)%\$2%" --fkeys-deferrable-mode not_deferrable --profile boost/date-time --profile boost/optional --profile boost/uuid --changelog odb-changelog.log --changelog-dir PROJDIR --default-pointer std::shared_ptr --output-dir PROJDIR/cmake-build-debug/odb_gen --hxx-suffix .hpp --ixx-suffix _inline.hpp --cxx-suffix .cpp --odb-file-suffix _odb -I/usr/local/include PROJDIR/src/datamodel/entities/Madantor.hpp So I tried with: (I deleted the changelog again) odb -d mysql --generate-query --generate-schema --schema-format separate --std c++17 --include-regex "%(.*)/(.+)%\$2%" --fkeys-deferrable-mode not_deferrable --profile boost/date-time --profile boost/optional --profile boost/uuid --changelog odb-changelog.log --changelog-dir PROJDIR --default-pointer std::shared_ptr --output-dir PROJDIR/cmake-build-debug/odb_gen --hxx-suffix .hpp --ixx-suffix _inline.hpp --cxx-suffix .cpp --odb-file-suffix _odb -I/usr/local/include PROJDIR /src/datamodel/entities/Users.hpp PROJDIR/src/datamodel/entities/Madantor.hpp But only the last one is in changelog (Madantor) ?Am 16.06.20, 15:21 schrieb "Boris Kolpackov" : odb -d mysql --generate-query --generate-schema --schema-format separate \ --std c++17 --changelog odb-changelog.log --changelog-dir PROJDIR \ --output-dir PROJDIR/cmake-build-debug/odb_gen test.hxx From boris at codesynthesis.com Fri Jun 19 07:56:57 2020 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Jun 19 08:09:55 2020 Subject: [odb-users] Schema Migration not working correctly In-Reply-To: References: <5D8FC02A-5DE4-46C2-92D8-231D0D84E76F@ddv-mediengruppe.de> <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> <7F67352D-0F5D-4EEB-9F38-978D7421BDF7@ddv-mediengruppe.de> Message-ID: Tschoche.Jonas@ddv-mediengruppe.de writes: > I tried it also with my project files. I deleted the odb_changelog and > run the same command that is executed by cmake and the table is in > changelog, but when I run the same command with the next table header > and the odb_changelog is exist - it will nothing append to it. That is your problem: you are trying to share the same changelog between multiple header files. You will either need to use separate changelog files or, if you want it combined, then you will need to generate the schema for all your headers at once; search for the --at-once option in the ODB manual for details and examples. From Tschoche.Jonas at ddv-mediengruppe.de Mon Jun 22 09:15:48 2020 From: Tschoche.Jonas at ddv-mediengruppe.de (Tschoche.Jonas@ddv-mediengruppe.de) Date: Tue Jun 23 03:11:28 2020 Subject: [odb-users] Schema Migration not working correctly In-Reply-To: References: <5D8FC02A-5DE4-46C2-92D8-231D0D84E76F@ddv-mediengruppe.de> <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> <7F67352D-0F5D-4EEB-9F38-978D7421BDF7@ddv-mediengruppe.de> Message-ID: Ah okay, thanks. I tried it and it worked, but when i use --at-once it generates one header file, I like to use a single changelog file and several header files. ?Am 19.06.20, 13:57 schrieb "Boris Kolpackov" : Tschoche.Jonas@ddv-mediengruppe.de writes: > I tried it also with my project files. I deleted the odb_changelog and > run the same command that is executed by cmake and the table is in > changelog, but when I run the same command with the next table header > and the odb_changelog is exist - it will nothing append to it. That is your problem: you are trying to share the same changelog between multiple header files. You will either need to use separate changelog files or, if you want it combined, then you will need to generate the schema for all your headers at once; search for the --at-once option in the ODB manual for details and examples. From boris at codesynthesis.com Wed Jun 24 05:23:47 2020 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Jun 24 05:37:02 2020 Subject: [odb-users] Schema Migration not working correctly In-Reply-To: References: <5D8FC02A-5DE4-46C2-92D8-231D0D84E76F@ddv-mediengruppe.de> <3C6E466E-FF87-48B3-9A15-90A6A8B73606@ddv-mediengruppe.de> <7F67352D-0F5D-4EEB-9F38-978D7421BDF7@ddv-mediengruppe.de> Message-ID: Tschoche.Jonas@ddv-mediengruppe.de writes: > I tried it and it worked, but when i use --at-once it generates one > header file, I like to use a single changelog file and several header > files. To achieve this you will need split the generation of the database support code (header/source files) and schema into separate invocations of the ODB compiler. For the schema generation (which is where the changelog is maintained), you can use the --at-once mode. Again, if you search for --at-once in the ODB manual, you will see examples of how to do this.