From boris at codesynthesis.com Fri Mar 1 08:40:03 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Mar 1 08:56:15 2019 Subject: [xsd-users] Segmentation fault in application when parsing large XML with default (8MiB) stack size In-Reply-To: References: Message-ID: Tempelaar E. (Erik) writes: > Our app crashes with segfault when it tries to parse a rather large > XML-file (close to the stacksize of 8MiB). As a workaround the stack > for the application has been increased. > > #5 0x08224055 in xercesc_3_2::RegularExpression::matchUnion(xercesc_3_2::RegularExpression::Context*, xercesc_3_2::Op const*, unsigned int) const () > #6 0x08221097 in xercesc_3_2::RegularExpression::match(xercesc_3_2::RegularExpression::Context*, xercesc_3_2::Op const*, unsigned int) const () > #7 0x08224090 in xercesc_3_2::RegularExpression::matchUnion(xercesc_3_2::RegularExpression::Context*, xercesc_3_2::Op const*, unsigned int) const () > > ... this continues ... I don't think it's the size of the XML document (most of the content should be allocated on the heap) but rather the regex implementation in Xerces-C++ which is apparently recursive/stack-based. Other than re-implementing the regex support in Xerces-C++, the only viable workaround is to reimplement/remove the regex constraint in your schema that causes this (or, disable XML Schema validation altogether). I would first try to come up with a "better" regex that doesn't trigger this. From Erik.Tempelaar at vanoord.com Fri Mar 8 03:09:55 2019 From: Erik.Tempelaar at vanoord.com (Tempelaar E. (Erik)) Date: Fri Mar 8 08:08:06 2019 Subject: [xsd-users] Segmentation fault in application when parsing large XML with default (8MiB) stack size In-Reply-To: References: Message-ID: That is indeed the case. Disabling the regex in the schema resolved the error. I'll try to come up with a better one. Thank you, much appreciated! -----Original Message----- From: Boris Kolpackov Sent: 01 March 2019 14:40 To: Tempelaar E. (Erik) Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] Segmentation fault in application when parsing large XML with default (8MiB) stack size Tempelaar E. (Erik) writes: > Our app crashes with segfault when it tries to parse a rather large > XML-file (close to the stacksize of 8MiB). As a workaround the stack > for the application has been increased. > > #5 0x08224055 in > xercesc_3_2::RegularExpression::matchUnion(xercesc_3_2::RegularExpress > ion::Context*, xercesc_3_2::Op const*, unsigned int) const () > #6 0x08221097 in > xercesc_3_2::RegularExpression::match(xercesc_3_2::RegularExpression:: > Context*, xercesc_3_2::Op const*, unsigned int) const () > #7 0x08224090 in > xercesc_3_2::RegularExpression::matchUnion(xercesc_3_2::RegularExpress > ion::Context*, xercesc_3_2::Op const*, unsigned int) const () > > ... this continues ... I don't think it's the size of the XML document (most of the content should be allocated on the heap) but rather the regex implementation in Xerces-C++ which is apparently recursive/stack-based. Other than re-implementing the regex support in Xerces-C++, the only viable workaround is to reimplement/remove the regex constraint in your schema that causes this (or, disable XML Schema validation altogether). I would first try to come up with a "better" regex that doesn't trigger this. Disclaimer: This mail transmission and any attached files are confidential and are intended for the addressee only. If you are not the person or organization to whom it is addressed, you must not copy, disclose, distribute or take any action in reliance upon it. If you have received this message in error, please contact the sender by email and delete all copies of this message and all copies of any attached files. From donaldhpalmer at yahoo.com Sat Mar 9 22:16:07 2019 From: donaldhpalmer at yahoo.com (Donald Palmer) Date: Sat Mar 9 22:32:44 2019 Subject: [xsd-users] Xerces-C++ issues installed on Linux Mint References: <229530472.2908553.1552187767423.ref@mail.yahoo.com> Message-ID: <229530472.2908553.1552187767423@mail.yahoo.com> hello all,trying to get through the Hello XML example but have not gotten it to compile. I am using QT Creator. Have been able to generate the hello.pxx and hello.hxx files OK.I'm not real experienced at installing software that must be configured and compiled as Xerces-C++ must be. Current version does not have binaries available.I think I did get it configured and installed but QT does not seem to fined everything it needs to compile the driver.cxx code.I do know the Xerces-C++ files are at /usr/local/include/xerceses/uitl/ and my XSD files are at usr/include/xsd/cxx/I have been able to use external build for the hello.xsd file and generate the hello.cxx and h.xx files.Thanks in advance From jimson.james at intellitix.com Wed Mar 13 15:26:31 2019 From: jimson.james at intellitix.com (jimson.james@intellitix.com) Date: Wed Mar 13 15:43:33 2019 Subject: [xsd-users] ACE CDR Constructor for deserialization. Message-ID: <01d901d4d9d2$aa730050$ff5900f0$@intellitix.com> Hi XSD users, In the ACE CDR example, the deserialization uses the constructor. Didn't find any functions to do something like below, std::auto_ptr copy (catalog_(icdr, ::xml_schema::Flags::dont_validate)); Any idea? Regards, From jimson.james at intellitix.com Fri Mar 15 01:33:41 2019 From: jimson.james at intellitix.com (jimson.james@intellitix.com) Date: Fri Mar 15 01:50:47 2019 Subject: [xsd-users] XSD_CXX_TREE_DECIMAL_FIXED functionality. Message-ID: <008a01d4daf0$a70efad0$f52cf070$@intellitix.com> Guys, Any simple way to ensure the 'decimal' presion is kept even when the decimal is 0.00? Right now I have patched the decimal serialization like below. template std::basic_string insert (const as_decimal& d) { std::basic_ostringstream os; os.imbue (std::locale::classic ()); std::streamsize prec; const facet* f = d.facets ? facet::find (d.facets, facet::fraction_digits) : 0; ... // Remove the trailing zeros and the decimal point if necessary. // typename std::basic_string::size_type size (r.size ()), n (size); if (prec != 0) { for (; n > 0 && cr[n - 1] == '0'; --n)/*noop*/; if (n > 0 && cr[n - 1] == '.') #if defined(XSD_CXX_TREE_DECIMAL_FIXED) n+=2; #else --n; #endif ... Is it fair to fix it like this, or do I really need to use the customized type functionality? Thx From boris at codesynthesis.com Mon Mar 18 08:50:42 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Mar 18 09:07:46 2019 Subject: [xsd-users] ACE CDR Constructor for deserialization. In-Reply-To: <01d901d4d9d2$aa730050$ff5900f0$@intellitix.com> References: <01d901d4d9d2$aa730050$ff5900f0$@intellitix.com> Message-ID: jimson.james@intellitix.com writes: > In the ACE CDR example, the deserialization uses the constructor. > > Didn't find any functions to do something like below, > > std::auto_ptr copy (catalog_(icdr, > ::xml_schema::Flags::dont_validate)); Loading from a binary representation is done via constructors (note that there is no schema validation so the flags do not apply). See Section 5.2, "Binary Serialization" for details: https://codesynthesis.com/projects/xsd/documentation/cxx/tree/manual/#5.2 From boris at codesynthesis.com Mon Mar 18 08:59:29 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Mar 18 09:16:35 2019 Subject: [xsd-users] XSD_CXX_TREE_DECIMAL_FIXED functionality. In-Reply-To: <008a01d4daf0$a70efad0$f52cf070$@intellitix.com> References: <008a01d4daf0$a70efad0$f52cf070$@intellitix.com> Message-ID: jimson.james@intellitix.com writes: > Any simple way to ensure the 'decimal' presion is kept even when the decimal > is 0.00? > > [...] > > Is it fair to fix it like this, or do I really need to use the customized > type functionality? Using type customization is the clean way to do it (see the cxx/tree/custom/double example which does pretty much this but for the xs:double type). Alternatively, if you always want two fractional digits, then you can compile with the XSD_CXX_TREE_DECIMAL_PRECISION macro defines to 2. From rjl at third-monday.com Thu Mar 28 12:17:19 2019 From: rjl at third-monday.com (rjl@third-monday.com) Date: Thu Mar 28 12:35:05 2019 Subject: [xsd-users] Issue with VS2017 Message-ID: <56da6799477a4cd1f0e3ce3450f9de34@third-monday.com> Bear with me here, this is a loooong email... I'm using XSD 3.3.0 and Xerces 3.1 on Windows. I have a Qt-based project that I have been building and using Visual Studio 2008 for about 8 years. I build and use both 'Debug' and 'Release' build configurations. This project uses XSD Tree utility for XML serialization/deserialization. Works great! I'm migrating to VS2017 and now I'm getting some rather strange LNK2005 errors on std::string methods and operators. The code base is broken into two libraries and a bunch of UI directories. In Debug build configuration the two libraries are built into DLLs and then must be specified in the PATH when running one of the UIs. In Release mode everything is compiled statically into a single executable. That builds and works great in the old and new environments. Note the 'Debug' configuration is only used for internal development/debugging. We only ship statically linked Release configuration. The two libraries are 'model' and 'engine'. The 'model' is built first and the 'engine' is then built. Code in the 'engine' library uses code in the 'model' library. In 'Debug' configuration, when the 'engine' is linked, I get a bunch of the following errors, all dealing with std::string: link /NOLOGO /DYNAMICBASE /NXCOMPAT /DEBUG /NODEFAULTLIB:LIBCMT /NODEFAULTLIB:LIBCMTD /NODEFAULTLIB:LIBCPMT /NODEFAULTLIB:LIBCPMTD /NODEFAULTLIB:MSVCRT /NODEFAULTLIB:MSVCPRT /NODEFAULTLIB:LIBVCRUNTIME /NODEFAULTLIB:LIBVCRUNTIMED /NODEFAULTLIB:VCRUNTIME /DLL /SUBSYSTEM:WINDOWS /MANIFEST:embed /OUT:lib.debug\flowEngine.dll @C:\Users\rjl\AppData\Local\Temp\nmF07.tmp model.lib(model.dll) : error LNK2005: "public: __thiscall std::basic_string,class std::allocator >::basic_string,class std::allocator >(class std::basic_string,class std::allocator > const &)" (??0?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@QAE@ABV01@@Z) already defined in NamedObject.obj model.lib(model.dll) : error LNK2005: "public: class std::basic_string,class std::allocator > & __thiscall std::basic_string,class std::allocator >::operator=(class std::basic_string,class std::allocator > const &)" (??4?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@QAEAAV01@ABV01@@Z) already defined in NamedObject.obj (NamedObject is in 'engine' library.) These look like 'std::string (const std::string & other)' and 'std::string & operator=(const std::string & other)'. I did a dumpbin /all model.dll on both versions (VS2008 and VS2017) and found that, indeed, the VS2017 version included std::string implementations, but the VS2008 did not. If I don't include my XSD stuff (which requires Xerces) in the 'model' DLL, I don't have the problem. I verified that the Xerces (v3.1.1) is compiled with /MDd. I had been using Xerces in 'static debug' with using the '-DXERCES_STATIC_LIBRARY' option at compile time and in 'dynamic debug' mode (not using the '-DXERCES_STATIC_LIBRARY' compile time option). Both versions fail the same (std::string stuff is in the model.dll in the VS2017 build. I did a 'dumpbin' on the Xerces 3.1 debug DLL and it did NOT include the std::string class/functions. Help? From boris at codesynthesis.com Fri Mar 29 10:32:44 2019 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Mar 29 10:50:27 2019 Subject: [xsd-users] Issue with VS2017 In-Reply-To: <56da6799477a4cd1f0e3ce3450f9de34@third-monday.com> References: <56da6799477a4cd1f0e3ce3450f9de34@third-monday.com> Message-ID: rjl@third-monday.com writes: > I'm using XSD 3.3.0 and Xerces 3.1 on Windows. > > [...] > > I'm migrating to VS2017 [...] As a first step, I would suggest that you upgrade to at least XSD 4.0.0, or even to the next version pre-release: https://codesynthesis.com/~boris/tmp/xsd/4.1.0.a11/ > I'm getting some rather strange LNK2005 errors on std::string methods > and operators. In a nutshell, the reason for these errors is: 1. XSD implements certain basic types by deriving from std::string (because they have the 'is-a' relationship to xsd:string in XML Schema). 2. MSVC automatically applies the dllimport/dllexport of a derived class to a base class that doesn't have its own (and which std::string doesn't). 3. In MSVC 2017 (but not 2008) the implementation of std::string now contains non-inline member functions (or inline ones that the compiler chooses not to inline in Debug). 4. If you have two or more DLLs that include the generated code, then you may end up with duplicate std::string symbols because they both export them (see 2 above). There are various ways to solve this problem with the easiest being to place all the generated code into a single DLL. For other ways see this earlier post: http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003011.html With this email containing more analysis/information on the problem: http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003019.html From rjl at third-monday.com Fri Mar 29 10:57:09 2019 From: rjl at third-monday.com (rjl@third-monday.com) Date: Fri Mar 29 11:14:48 2019 Subject: [xsd-users] Issue with VS2017 In-Reply-To: References: <56da6799477a4cd1f0e3ce3450f9de34@third-monday.com> Message-ID: <31838d9de74fe3d8f46565eb640bbbda@third-monday.com> Thanks for the response. Yes, I can upgrade to the latest XSD. Will that require an upgrade to Xerces, as well? All of the XSD Tree generated code is linked into the 'model' DLL/library. The 'engine' DLL/library and all of the apps use those structures from the model library. VS2017 complains mightly about classes in a DLL library with 'dllexport' that contain 'std::string' attributes (e.g., 'private: std::string _name;' in the class). That's what that 'NamedObject' class looks like as I referenced in my original msg. I like using DLLs for development because they simplify building as I'm working - no need to re-link apps when some lines in a library changed. However, I think I may have made progress by doing away with DLLs during development. I think I can build 'static debug' libraries and apps. That'll increase development time (e.g,. must wait for apps to re-link) but it'll save the rest of the hair on my head as I move forward. I'll let you know what I find after upgrading and reading the discussions listed below. Thanks again, r On 2019-03-29 09:32, Boris Kolpackov wrote: > rjl@third-monday.com writes: > >> I'm using XSD 3.3.0 and Xerces 3.1 on Windows. >> >> [...] >> >> I'm migrating to VS2017 [...] > > As a first step, I would suggest that you upgrade to at least XSD > 4.0.0, > or even to the next version pre-release: > > https://codesynthesis.com/~boris/tmp/xsd/4.1.0.a11/ > > >> I'm getting some rather strange LNK2005 errors on std::string methods >> and operators. > > In a nutshell, the reason for these errors is: > > 1. XSD implements certain basic types by deriving from std::string > (because they have the 'is-a' relationship to xsd:string in XML > Schema). > > 2. MSVC automatically applies the dllimport/dllexport of a derived > class to a base class that doesn't have its own (and which > std::string doesn't). > > 3. In MSVC 2017 (but not 2008) the implementation of std::string > now contains non-inline member functions (or inline ones that > the compiler chooses not to inline in Debug). > > 4. If you have two or more DLLs that include the generated code, > then you may end up with duplicate std::string symbols because > they both export them (see 2 above). > > There are various ways to solve this problem with the easiest > being to place all the generated code into a single DLL. For > other ways see this earlier post: > > http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003011.html > > With this email containing more analysis/information on the problem: > > http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003019.html From rjl at third-monday.com Fri Mar 29 12:48:52 2019 From: rjl at third-monday.com (rjl@third-monday.com) Date: Fri Mar 29 13:06:40 2019 Subject: [xsd-users] Issue with VS2017 In-Reply-To: <31838d9de74fe3d8f46565eb640bbbda@third-monday.com> References: <56da6799477a4cd1f0e3ce3450f9de34@third-monday.com> <31838d9de74fe3d8f46565eb640bbbda@third-monday.com> Message-ID: <4b712beb7f5dd52f8a157be56ac9f889@third-monday.com> This morning's updates: 1) I upgraded to XSD 4.0.0 - no need to upgrade Xerces (from 3.1.1) as far as I can tell. 2) I implemented the updates to the XML code generation as described in the 1st note below (added ' --export-xml-schema --hxx-prologue "\#include \"model/util/Defines.h\"" --export-symbol MODEL_EXPORTAPI) to the xmlSchema.h generation. 3) Pushed on the 'static debug' build for the model n engine libraries to make sure they were, indeed, 'debug'-worthy builds. 4) Figgered out how to make my app programs depend on the libraries using Qt qmake variables so they're relinked should any of the libraries change. Everything seems to work. Point '2)' really isn't needed now as I'm building my libs statically. I think I can enter the weekend with something accomplished! This has been bugging me for days.... Thanks again for the explanation about the VS changes w.r.t. 'std::string'. r On 2019-03-29 09:57, rjl@third-monday.com wrote: > Thanks for the response. > > Yes, I can upgrade to the latest XSD. Will that require an upgrade to > Xerces, as well? > > All of the XSD Tree generated code is linked into the 'model' > DLL/library. The 'engine' DLL/library and all of the apps use those > structures from the model library. > > VS2017 complains mightly about classes in a DLL library with > 'dllexport' that contain 'std::string' attributes (e.g., 'private: > std::string _name;' in the class). > > That's what that 'NamedObject' class looks like as I referenced in my > original msg. > > I like using DLLs for development because they simplify building as > I'm working - no need to re-link apps when some lines in a library > changed. > > However, I think I may have made progress by doing away with DLLs > during development. I think I can build 'static debug' libraries and > apps. That'll increase development time (e.g,. must wait for apps to > re-link) but it'll save the rest of the hair on my head as I move > forward. > > I'll let you know what I find after upgrading and reading the > discussions listed below. > > Thanks again, > r > > On 2019-03-29 09:32, Boris Kolpackov wrote: >> rjl@third-monday.com writes: >> >>> I'm using XSD 3.3.0 and Xerces 3.1 on Windows. >>> >>> [...] >>> >>> I'm migrating to VS2017 [...] >> >> As a first step, I would suggest that you upgrade to at least XSD >> 4.0.0, >> or even to the next version pre-release: >> >> https://codesynthesis.com/~boris/tmp/xsd/4.1.0.a11/ >> >> >>> I'm getting some rather strange LNK2005 errors on std::string methods >>> and operators. >> >> In a nutshell, the reason for these errors is: >> >> 1. XSD implements certain basic types by deriving from std::string >> (because they have the 'is-a' relationship to xsd:string in XML >> Schema). >> >> 2. MSVC automatically applies the dllimport/dllexport of a derived >> class to a base class that doesn't have its own (and which >> std::string doesn't). >> >> 3. In MSVC 2017 (but not 2008) the implementation of std::string >> now contains non-inline member functions (or inline ones that >> the compiler chooses not to inline in Debug). >> >> 4. If you have two or more DLLs that include the generated code, >> then you may end up with duplicate std::string symbols because >> they both export them (see 2 above). >> >> There are various ways to solve this problem with the easiest >> being to place all the generated code into a single DLL. For >> other ways see this earlier post: >> >> http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003011.html >> >> With this email containing more analysis/information on the problem: >> >> http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003019.html From rjl at third-monday.com Fri Mar 29 13:05:35 2019 From: rjl at third-monday.com (rjl@third-monday.com) Date: Fri Mar 29 13:23:14 2019 Subject: [xsd-users] Issue with VS2017 In-Reply-To: <4b712beb7f5dd52f8a157be56ac9f889@third-monday.com> References: <56da6799477a4cd1f0e3ce3450f9de34@third-monday.com> <31838d9de74fe3d8f46565eb640bbbda@third-monday.com> <4b712beb7f5dd52f8a157be56ac9f889@third-monday.com> Message-ID: <6323b29a9b4697de5b21bedd81873581@third-monday.com> Ack! Spoke tooooo soon. Now I'm getting the following: error C2872: 'DOMDocument': ambiguous symbol I see some other messages in this mailing list that says this is fixed in a pre-release binary (4.1.0). For now, I'll move back to XSD 3.3.0. r On 2019-03-29 11:48, rjl@third-monday.com wrote: > This morning's updates: > > 1) I upgraded to XSD 4.0.0 - no need to upgrade Xerces (from 3.1.1) as > far as I can tell. > > 2) I implemented the updates to the XML code generation as described > in the 1st note below (added ' --export-xml-schema --hxx-prologue > "\#include \"model/util/Defines.h\"" --export-symbol MODEL_EXPORTAPI) > to the xmlSchema.h generation. > > 3) Pushed on the 'static debug' build for the model n engine libraries > to make sure they were, indeed, 'debug'-worthy builds. > > 4) Figgered out how to make my app programs depend on the libraries > using Qt qmake variables so they're relinked should any of the > libraries change. > > Everything seems to work. Point '2)' really isn't needed now as I'm > building my libs statically. > > I think I can enter the weekend with something accomplished! This has > been bugging me for days.... > > Thanks again for the explanation about the VS changes w.r.t. > 'std::string'. > > r > > On 2019-03-29 09:57, rjl@third-monday.com wrote: >> Thanks for the response. >> >> Yes, I can upgrade to the latest XSD. Will that require an upgrade to >> Xerces, as well? >> >> All of the XSD Tree generated code is linked into the 'model' >> DLL/library. The 'engine' DLL/library and all of the apps use those >> structures from the model library. >> >> VS2017 complains mightly about classes in a DLL library with >> 'dllexport' that contain 'std::string' attributes (e.g., 'private: >> std::string _name;' in the class). >> >> That's what that 'NamedObject' class looks like as I referenced in my >> original msg. >> >> I like using DLLs for development because they simplify building as >> I'm working - no need to re-link apps when some lines in a library >> changed. >> >> However, I think I may have made progress by doing away with DLLs >> during development. I think I can build 'static debug' libraries and >> apps. That'll increase development time (e.g,. must wait for apps to >> re-link) but it'll save the rest of the hair on my head as I move >> forward. >> >> I'll let you know what I find after upgrading and reading the >> discussions listed below. >> >> Thanks again, >> r >> >> On 2019-03-29 09:32, Boris Kolpackov wrote: >>> rjl@third-monday.com writes: >>> >>>> I'm using XSD 3.3.0 and Xerces 3.1 on Windows. >>>> >>>> [...] >>>> >>>> I'm migrating to VS2017 [...] >>> >>> As a first step, I would suggest that you upgrade to at least XSD >>> 4.0.0, >>> or even to the next version pre-release: >>> >>> https://codesynthesis.com/~boris/tmp/xsd/4.1.0.a11/ >>> >>> >>>> I'm getting some rather strange LNK2005 errors on std::string >>>> methods >>>> and operators. >>> >>> In a nutshell, the reason for these errors is: >>> >>> 1. XSD implements certain basic types by deriving from std::string >>> (because they have the 'is-a' relationship to xsd:string in XML >>> Schema). >>> >>> 2. MSVC automatically applies the dllimport/dllexport of a derived >>> class to a base class that doesn't have its own (and which >>> std::string doesn't). >>> >>> 3. In MSVC 2017 (but not 2008) the implementation of std::string >>> now contains non-inline member functions (or inline ones that >>> the compiler chooses not to inline in Debug). >>> >>> 4. If you have two or more DLLs that include the generated code, >>> then you may end up with duplicate std::string symbols because >>> they both export them (see 2 above). >>> >>> There are various ways to solve this problem with the easiest >>> being to place all the generated code into a single DLL. For >>> other ways see this earlier post: >>> >>> http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003011.html >>> >>> With this email containing more analysis/information on the problem: >>> >>> http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003019.html