From Wesley.Peters at tachyon.com Mon Feb 4 13:25:30 2013 From: Wesley.Peters at tachyon.com (Wesley Peters) Date: Tue Feb 5 07:16:27 2013 Subject: [xsd-users] Modify XSD compiler Message-ID: I'm working on an application where one of our use cases involves a run-time statistics logging server. The design of the server is quite simple, it is fed a stream of objects in queue; these objects are coded in XML by XSD for transport. When the server retrieves the object, it's job is to inject the objects into a MongoDB database. I started looking at deriving classes for each object type from the ones created by the XSD compiler, adding in the calls to store the object into MongoDB, but it struck me it should be possible to add code to the XSD compiler to also generate the MongoDB "serialization" code. Is this an approach other XSD users have employed? Do you have any documentation on the internals of the compiler? Where would I start looking in the source tree to see where your code is walking the parse trees and generating the encoding methods? ________________________________ Confidentiality Notice: The information contained in this electronic e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and is confidential and/or privileged. If you and we have a confidentiality agreement or other non-disclosure obligations between us, this Notice shall be deemed to mark and identify the content of this email and any attachments as confidential and proprietary. If any reader of this communication is not the intended recipient, unauthorized use, disclosure or copying is strictly prohibited, and may be unlawful. If you have received this communication in error, please immediately notify the sender by return e-mail, and delete the original message and all copies from your system. Thank you. IRS Circular 230 Disclosure: To ensure compliance with requirements imposed by the IRS, please be advised that any U.S. federal tax advice contained in this communication (including any attachments) is not intended or written to be used or relied upon, and cannot be used or relied upon, for the purpose of (i) avoiding penalties under the Internal Revenue Code, or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein. E-mail is susceptible to data corruption, interception, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof. From boris at codesynthesis.com Tue Feb 5 09:40:00 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Feb 5 08:43:33 2013 Subject: [xsd-users] Modify XSD compiler In-Reply-To: References: Message-ID: Hi Wesley, Wesley Peters writes: > Is this an approach other XSD users have employed? I think a few thought about it. Here are the relevant emails: http://www.codesynthesis.com/pipermail/xsd-users/2007-April/000931.html http://www.codesynthesis.com/pipermail/xsd-users/2009-July/002397.html > Do you have any documentation on the internals of the compiler? While there is no standalone documentation, the source code is very well commented (or so I believe). > Where would I start looking in the source tree to see where your > code is walking the parse trees and generating the encoding methods? There are fairly simple generatord (e.g., stream insertion/extraction) that should give you a pretty good idea on how to start. Also, the above emails provide an overview. > ... it should be possible to add code to the XSD compiler to also > generate the MongoDB "serialization" code. Another interesting idea would be to run ODB[1] on the XSD-generated classes. While ODB does not support MongoDB right now, we do have plans to support it in the future. Just a thought. [1] http://www.codesynthesis.com/products/odb/ Boris From Wesley.Peters at tachyon.com Tue Feb 5 17:22:10 2013 From: Wesley.Peters at tachyon.com (Wesley Peters) Date: Tue Feb 5 23:05:07 2013 Subject: [xsd-users] Modify XSD compiler In-Reply-To: Message-ID: Yes, very helpful email links. This paragraph in particular: > In a nutshell, you would add the generation of the accept() > declarations in tree-header.cxx. The implementations can be > generated in tree-source.cxx. You can also add an option, > say --generate-visitor , which would trigger > the generation as well as provide the visitor class name. Is a pretty concise summary of what needs to be done. ODB looks interesting, we already looked at it, but we have a short timeline for this. Perhaps worth revisiting in the future. On 2/5/13 6:40 AM, "Boris Kolpackov" wrote: Hi Wesley, Wesley Peters writes: > Is this an approach other XSD users have employed? I think a few thought about it. Here are the relevant emails: http://www.codesynthesis.com/pipermail/xsd-users/2007-April/000931.html http://www.codesynthesis.com/pipermail/xsd-users/2009-July/002397.html > Do you have any documentation on the internals of the compiler? While there is no standalone documentation, the source code is very well commented (or so I believe). > Where would I start looking in the source tree to see where your > code is walking the parse trees and generating the encoding methods? There are fairly simple generatord (e.g., stream insertion/extraction) that should give you a pretty good idea on how to start. Also, the above emails provide an overview. > ... it should be possible to add code to the XSD compiler to also > generate the MongoDB "serialization" code. Another interesting idea would be to run ODB[1] on the XSD-generated classes. While ODB does not support MongoDB right now, we do have plans to support it in the future. Just a thought. [1] http://www.codesynthesis.com/products/odb/ Boris ________________________________ Confidentiality Notice: The information contained in this electronic e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and is confidential and/or privileged. If you and we have a confidentiality agreement or other non-disclosure obligations between us, this Notice shall be deemed to mark and identify the content of this email and any attachments as confidential and proprietary. If any reader of this communication is not the intended recipient, unauthorized use, disclosure or copying is strictly prohibited, and may be unlawful. If you have received this communication in error, please immediately notify the sender by return e-mail, and delete the original message and all copies from your system. Thank you. IRS Circular 230 Disclosure: To ensure compliance with requirements imposed by the IRS, please be advised that any U.S. federal tax advice contained in this communication (including any attachments) is not intended or written to be used or relied upon, and cannot be used or relied upon, for the purpose of (i) avoiding penalties under the Internal Revenue Code, or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein. E-mail is susceptible to data corruption, interception, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof. From luca.paganotti at gmail.com Fri Feb 8 05:19:00 2013 From: luca.paganotti at gmail.com (luca paganotti) Date: Fri Feb 8 05:01:36 2013 Subject: [xsd-users] GML schema compilation Message-ID: Hi all, I'm trying to compile GML schema using cmake and xsdcxx. I'm using a fedora 17 box and the default xsd rpm packages. I'll probably ask a silly and basic question because I'm relatively new to xml data binding and a cmake first time user. I got the 3.1.1 GML schemas from ogc website, stored them in a suitable folder and the run cmake . from the gml base folder. I modified the definition of OGC_BINDINGS_DIR to make GML_DIR point to the folder I copied the schemas in. Now the Compilation goes some steps ahead but fails giving me: buck@buckmobile ~/dev/c++/ogc/gml/3.1.1>cmake . CMake Error at CMakeLists.txt:54 (XSD_SCHEMA): Unknown CMake command "XSD_SCHEMA". I think to have found the problem i.e. the XSD_SCHEMA macro is not defined in CMakeLists.txt so that cmake complains. Am I right? I've found that this macro is defined in another downloadable file ( FindXSD.cmake ) and got it, but as far as I know nothing about cmake modules and how to configure them I'm stuck at this point. How can I tell cmake to use FindXSD.cmake? Maybe this list is not the right place to post such a request, if so, please, drive me to the right place ... P.S. Please forgive my english as it's not mother's language. Thanks for any answer. Luca Paganotti From adam at kde.org Sat Feb 9 17:35:12 2013 From: adam at kde.org (Till Adam) Date: Mon Feb 11 05:03:44 2013 Subject: [xsd-users] Bug: failure to convert string to int In-Reply-To: <6F6001EB-54B3-4A07-A6D4-565918FEE0F5@kde.org> References: <2944999.8nGvGK3roT@till.local> <6F6001EB-54B3-4A07-A6D4-565918FEE0F5@kde.org> Message-ID: <1A54691C-5B69-432D-9813-73AEE9C65D98@kde.org> Folks, just to follow up on this: On Jan 28, 2013, at 3:03 PM, Till Adam wrote: > thanks for checking out the bug. > > On Jan 28, 2013, at 2:34 PM, Boris Kolpackov wrote: > >> Till Adam writes: >> >>> I seem to have stumbled upon a rather low level bug in libxsd on OSX Lion. >> >> I tried to reproduce the bug on 10.8 and 10.3 (I don't have access to >> 10.7) without any success. I tried 32/64, g++/clang++, optimized/non- >> optimized combinations and they all seem to work fine. > > Weird, but encouraging, at least as far as 10.8 is concerned. I'll see if I can test on that. > >> It is also not clear what happens when the test fails. Does it crash? >> Does it print a wrong value? If it crashes, would you be able to get >> the stack trace? > > No, it does not crash, it simply prints 0, for the value of year. It seems to magically find an empty buffer, from what I can tell. > > I'll see if I can get more information. If someone else could confirm that it fails for them on 10.7, I would be grateful. Frank and I did a bit more research and found by trial error that the bug only manifests with a certain version of Xcode (the one for Mountain Lion), only on Lion, only with g++. There was an Xcode update a few days ago, for Lion, which seems to resolve the issue provided one uses clang. So hopefully I just hit a bad time window, with a broken combination of OS/STL/compiler. That said, there are a bunch of bugs in this area on OSX, most can be mitigated by using deployment targets <=10.5, undefining _GLIBCXX_DEBUG or using clang, rather than c++. Just thought I'd post this result here, for documentation's sake, there does not seem to be anything xsd can or should do to help with these issues (other than building out of the box with clang, there's a still a tiny patch needed, but that's a known issue, I believe. Thanks again, Till From boris at codesynthesis.com Mon Feb 11 12:36:09 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Feb 11 12:18:12 2013 Subject: [xsd-users] Bug: failure to convert string to int In-Reply-To: <1A54691C-5B69-432D-9813-73AEE9C65D98@kde.org> References: <2944999.8nGvGK3roT@till.local> <6F6001EB-54B3-4A07-A6D4-565918FEE0F5@kde.org> <1A54691C-5B69-432D-9813-73AEE9C65D98@kde.org> Message-ID: Hi Till, Till Adam writes: > Frank and I did a bit more research and found by trial error that the > bug only manifests with a certain version of Xcode (the one for Mountain > Lion), only on Lion, only with g++. There was an Xcode update a few days > ago, for Lion, which seems to resolve the issue provided one uses clang. > So hopefully I just hit a bad time window, with a broken combination of > OS/STL/compiler. That said, there are a bunch of bugs in this area on > OSX, most can be mitigated by using deployment targets <=10.5, > undefining _GLIBCXX_DEBUG or using clang, rather than c++. > Just thought I'd post this result here, for documentation's sake [...] Yes, thanks, I appreciate you keeping us informed. > [...] other than building out of the box with clang, there's a still a > tiny patch needed, but that's a known issue, I believe. Yes, this is already fixed for the next version, which we are planning to release soon. Boris From Chinenye.Ajagu at flextrade.com Mon Feb 11 13:11:58 2013 From: Chinenye.Ajagu at flextrade.com (Chinenye Ajagu) Date: Mon Feb 11 12:58:01 2013 Subject: [xsd-users] Compilation Error in zc-istream.txx with gcc 4.7.0. Message-ID: <5119346E.8010005@flextrade.com> Hi, I have just attempted to compile the auto-generated source code with my c++ project, and I get a compilation error in libxsd/xsd/cxx/zc-istream.txx. /home/cajagu/workspace/xsd-3.3.0-x86_64-linux-gnu/libxsd/xsd/cxx/zc-istream.txx:35:7: error: 'setg' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] /home/cajagu/workspace/xsd-3.3.0-x86_64-linux-gnu/libxsd/xsd/cxx/zc-istream.txx:35:7: note: declarations in dependent base 'std::basic_streambuf' are not found by unqualified lookup /home/cajagu/workspace/xsd-3.3.0-x86_64-linux-gnu/libxsd/xsd/cxx/zc-istream.txx:35:7: note: use 'this->setg' instead The *zc-istream* class is derived from *std::basic_streambuf* which has the protected function *setg()*. When *zc-istream* attempts to call this method in its *init()* function, the compiler doesn't seem to be able to find that method, unless you make the call by doing *this->setg()* (as suggested with the error message). Making that tiny change fixed the issue for me and I was able to carry on and use everything as normal. I'm using gcc version 4.7. When I switch to using gcc version 4.1.2, this compiles fine with no errors both with an without the change. Maybe it will be worth updating that file to do it in the slightly modified way? Let me know what you think in any case, just in case I should not carry on using it as it is now. Thanks, Chi From boris at codesynthesis.com Mon Feb 11 13:23:50 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Feb 11 13:05:53 2013 Subject: [xsd-users] Compilation Error in zc-istream.txx with gcc 4.7.0. In-Reply-To: <5119346E.8010005@flextrade.com> References: <5119346E.8010005@flextrade.com> Message-ID: Hi Chinenye, Chinenye Ajagu writes: > I have just attempted to compile the auto-generated source code with my > c++ project, and I get a compilation error in > libxsd/xsd/cxx/zc-istream.txx. > > [...] > > I'm using gcc version 4.7. This is a know issue that we have fixed for the next release (coming soon). In the meantime, you can apply this patch to the XSD distribution and everything will compile fine: http://www.codesynthesis.com/~boris/tmp/xsd/xsd-3.3.0-gcc-4.7-clang.patch Boris From rlischner at proteuseng.com Thu Feb 14 07:04:00 2013 From: rlischner at proteuseng.com (Ray Lischner) Date: Thu Feb 14 08:45:28 2013 Subject: [xsd-users] C++11, value semantics, and polymorphism Message-ID: We are jumping head first into C++11 and very much want to take full advantage of value semantics. We want to be able to construct most Code Synthesis objects without resorting to any dynamic memory allocation. Unless there are plans for Code Synthesis 4 to support this functionality, we will undertake the work ourselves. It is a straightforward matter to modify xsd::cxx::tree::one<>, optional<>, and sequence<> to support value semantics as long as the types are not polymorphic. Making corresponding changes to xsd is not difficult. Most of our types are not polymorphic, and they are also the types that we most want to optimize. But we need to support polymorphism, too. That's where we run into a hitch. I want the parsing functions to return values for value classes, but they must return pointers for polymorphic classes. I certainly don't want to force the user of a class to write different code depending on whether it is polymorphic, so the only solution I see is for xsd to generate a pimpl wrapper for polymorphic classes to hide the difference. Thus, xsd would generate non-polymorphic classes normally. For a polymorphic class, it would generate a normal impl class and a pimpl wrapper. The pimpl wrapper would use the real type name; the impl class would append "_impl" to the class name (or something like that). The user should be able to use the wrapper the same way as a non-polymorphic class. Parsing functions would return a value for non-polymorphic types and a wrapper instance by value for polymorphic types. The user sees only values. The only real difference in semantics is that assigning a derived-type to a base-type slices the value if non-polymorphic and preserves the derived-type value if polymorphic. The difference in performance is that value types can be parsed, constructed, moved, and manipulated without touching the heap. One advantage of this approach is that the distinction between ref-to-const and auto_ptr arguments to constructors vanishes, and xsd can generate fewer constructors, passing all required arguments by value. The changes to tree-header.cxx and tree-source.cxx are considerable, so I am asking, "is there any other way to support value semantics and polymorphism?" Ray Lischner, Distinguished Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.9787 rlischner@proteuseng.com f. 443.539.3370 From boris at codesynthesis.com Thu Feb 14 10:32:58 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Feb 14 10:14:38 2013 Subject: [xsd-users] C++11, value semantics, and polymorphism In-Reply-To: References: Message-ID: Hi Ray, Ray Lischner writes: > We are jumping head first into C++11 and very much want to take full > advantage of value semantics. We want to be able to construct most Code > Synthesis objects without resorting to any dynamic memory allocation. > Unless there are plans for Code Synthesis 4 to support this functionality, > we will undertake the work ourselves. Yes, we do plan C++11 support, including move support, for the next release. While it would be great to have someone give this a try before the final release, the bad news is that I will only have something to try probably around end-June. Until then my schedule is very tight. > It is a straightforward matter to modify xsd::cxx::tree::one<>, > optional<>, and sequence<> to support value semantics as long as > the types are not polymorphic. Right, we will have to have two different version of these. In fact, we already do, for fundamental and user-defined types. In C++11 mode we will just use the by-value version for all non-polymorphic types. > But we need to support polymorphism, too. That's where we run into a > hitch. I want the parsing functions to return values for value classes, > but they must return pointers for polymorphic classes. I certainly > don't want to force the user of a class to write different code > depending on whether it is polymorphic, I don't think this would be forcing anybody. Down the line, polymorphic and non-polymorphic roots will be handled in a very different ways. With polymorphic, there will probably be a dynamic_cast to discover the actual type (try supporting this with your pimpl approach). Generally, polymorphism and by-value are in conceptual contradiction. By-value assumes the size of the object is known and fixed (so stack can be allocated for it). Polymorphism assumes we don't even know which object it is, let alone its size. By-value is: here is a chunk of memory, construct object foo there. Polymorphism is: this chunk of memory contains some object that is-a foo. Using dynamic memory to return a polymorphic object is the "C++ way" (think of the virtual destructor mechanism -- it was augmented to call the correct operator delete). Well, that was a bit of philosophical aside ;-). Now, in ODB, for example, we have two versions of the database::load() function. One which returns a (dynamically-allocated) object and one that can load the state into an existing instance: person p; db.load (p, "John Doe"); This works well for when you want to avoid allocations. We could do this with parsing function, but I am not even sure it makes sense: How will you know which object is being parsed? The only way to do this generally is to examine the element name (or xsi:type). But then, seeing that you already have the DOM element, you can just call the parsing constructor directly and place the object anywhere you want. I guess this could still be useful for situations like "the response to this message can only be class foo". > so the only solution I see is for xsd to generate a pimpl wrapper > for polymorphic classes to hide the difference. There are so many issues/complications with this approach that I don't even know where to begin (inheritance, type discovery, code bloat, customizability, etc, etc). I don't think we will go this route. > "is there any other way to support value semantics and polymorphism?" I think we should just keep it simple and return values by-value and use pointers/dynamic memory for polymorphic objects. Parsing functions will return std::unique_ptr for polymorphic roots. Boris From mjklaim at gmail.com Fri Feb 15 06:43:09 2013 From: mjklaim at gmail.com (=?UTF-8?Q?Klaim_=2D_Jo=C3=ABl_Lamotte?=) Date: Fri Feb 15 06:55:44 2013 Subject: [xsd-users] C++11, value semantics, and polymorphism In-Reply-To: References: Message-ID: Boris, recently on boost mailing list they are talking about copy-on-write semantic (someone proposed a copy-on-write smart pointer). Reading it, I re-watched the presentation at boost-con about value semantic, from someone who worked at Adobe ( http://www.youtube.com/watch?v=_BpMYeUFXv8 ) It made me think that the code generated by xsd could benefit from a copy-on-write implementation too. I might be wrong though. I like the simple type they made at adobe, with only read() and write() functions in the interface. Would it be useful in combination with move semantic in the code generated by xsd? Joel Lamotte From boris at codesynthesis.com Mon Feb 18 03:20:10 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Feb 18 03:20:16 2013 Subject: [xsd-users] C++11, value semantics, and polymorphism In-Reply-To: References: Message-ID: Hi Jo?l, Klaim - Jo?l Lamotte writes: > It made me think that the code generated by xsd could benefit from a > copy-on-write implementation too. Copy-on-write is a can of worms. It can be useful in certain controlled situations that is, where you can have a well defined read/write interface rather than "we return a reference but have no idea if the object will be modified or not" (which is what we have in XSD). In general-purpose code, however, it adds too many restrictions and/or complexity (see why COW is effectively no longer an option for std::string in C++11). So, no, I don't think COW is a good idea for XSD. Boris From mjklaim at gmail.com Tue Feb 19 06:14:09 2013 From: mjklaim at gmail.com (=?UTF-8?Q?Klaim_=2D_Jo=C3=ABl_Lamotte?=) Date: Tue Feb 19 06:14:18 2013 Subject: [xsd-users] C++11, value semantics, and polymorphism In-Reply-To: References: Message-ID: On Mon, Feb 18, 2013 at 9:20 AM, Boris Kolpackov wrote: > > Copy-on-write is a can of worms. It can be useful in certain controlled > situations that is, where you can have a well defined read/write interface > rather than "we return a reference but have no idea if the object will > be modified or not" (which is what we have in XSD). In general-purpose > code, however, it adds too many restrictions and/or complexity (see why > COW is effectively no longer an option for std::string in C++11). > > So, no, I don't think COW is a good idea for XSD. > I see, you're the expert on this anyway (I never used COW, was just wondering). Can't wait to see the value semantic version. I don't know exactly what the impact on performance will be but I used it in my big projects, even move-only types, and it does clarify a lot the user code. The only problem is VS2012 being retarded with C++11 features, in particular defaulting and deleting functions which would make declarations far simpler if they were available. Joel Lamotte From H.Klingel at gmx.net Fri Feb 22 03:55:01 2013 From: H.Klingel at gmx.net (H.Klingel@gmx.net) Date: Fri Feb 22 08:24:44 2013 Subject: [xsd-users] toString() method for basic datatypes Message-ID: <20130222085501.247910@gmx.net> Hi, what do you think about a toString() method for the datatype, e.g xsd::cxx::date_time? Regards, Harald From boris at codesynthesis.com Fri Feb 22 08:29:06 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Feb 22 08:29:14 2013 Subject: [xsd-users] toString() method for basic datatypes In-Reply-To: <20130222085501.247910@gmx.net> References: <20130222085501.247910@gmx.net> Message-ID: Hi Harald, H.Klingel@gmx.net writes: > what do you think about a toString() method for the datatype, e.g > xsd::cxx::date_time? You can already do this if you pass the --generate-ostream XSD compiler option and then do: #include xml_schema::date_time dt (...); std::ostringstream ostr; ostr << dt; std::string str (ostr.str ()); If you are using Boost, you could also use lexical_cast: std::string str (boost::lexical_cast (dt)); Boris From rlischner at proteuseng.com Fri Feb 22 08:57:02 2013 From: rlischner at proteuseng.com (Ray Lischner) Date: Fri Feb 22 09:04:46 2013 Subject: [xsd-users] C++11, value semantics, and polymorphism In-Reply-To: References: , Message-ID: > There are so many issues/complications with this approach that I don't > even know where to begin (inheritance, type discovery, code bloat, > customizability, etc, etc). I don't think we will go this route. I put together a prototype that addresses most of these concerns. For polymorphic types, it uses two parallel type hierarchies. One has "type" at its root, which hold a pointer to "impl" (which is the root of the other tree). The impl classes are the real classes with real data members, following the normal Code Synthesis pattern. The type classes are the classes that the developer uses. Each class has all the member functions of the impl class, each one forwarding to the impl class with inline functions. Parsing functions create impls and return type. A downcast<> function template can cast a type to a different type, performing the equivalent dynamic_cast<> on the impl pointers. The upshot is that a developer who does not want to need to know which types are polymorphic can have that fact completely hidden. Our use of Code Synthesis often involves the construction of objects from other objects, so we end up with code sort like this: A a(B("string", C(D("string", value), "string", "string"))); except that some of those types are polymorphic. Right now, we can create all the intermediate objects dynamically and sling auto_ptr<>s around to avoid excess copying. With C++ 11, we want to use value semantics where possible, but that forces the user to know which types are polymorphic and which are not, so we end up with the following: A a(B("string", new C(D("string", value), "string", "string"))); It seems like a minor point until you sit down with a couple dozen schemas and a hundred types, and you need to know which few types are polymorphic roots, and which ones are substitutionGroups. You need to keep the schemas open in another monitor just to write your code. With the type/impl classes hiding the polymorphism, the user no longer needs to know. The user can write everything as values classes, pass objects by value, returns objects by value, and everything just works. Values are implemented as values. Polymorphic classes are implemented with pimpls. The only time it matters to the user of a type is when code needs to downcast. The developer knows the type is polymorphic, so there is no problem exposing polymorphism. The only cost is cognitive, using downcast<> instead of dynamic_cast<>. The other time the type hackery is exposed is when the developer needs to customize a type. In that case, the type and the impl need to be customized; although I imagine that in many cases, one would want to customize only one or the other. Nonetheless, one must assume that the cost of customizing a type is doubled. The code bloat in executable code is minimal because the type classes are purely inline, with no virtual functions. The one<>, optional<>, and sequence<> templates are different, but there are still only two basic specializations: polymorphic and value. The value specialization uses a template I wrote that stores the value in std::aligned_storage<>, using placement new to construct the object when present, or leaving the buffer uninitialized when absent. An additional byte keeps track of whether the object is initialized. Thus, the normal semantics of these templates is preserved when fund=true. In the end, we decided not to follow the route of this prototype. We decided that the complexity costs in Code Synthesis were too high for the modest benefit gained, especially because we would be maintaining these considerable changes to Code Synthesis on our own. But the prototype was interesting to write and I can see the value of hiding polymorphism in certain cases. Ray Lischner, Distinguished Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.9787 rlischner@proteuseng.com f. 443.539.3370 ________________________________________ From: Boris Kolpackov [boris@codesynthesis.com] Sent: Thursday, February 14, 2013 10:32 AM To: Ray Lischner Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] C++11, value semantics, and polymorphism Hi Ray, Ray Lischner writes: > We are jumping head first into C++11 and very much want to take full > advantage of value semantics. We want to be able to construct most Code > Synthesis objects without resorting to any dynamic memory allocation. > Unless there are plans for Code Synthesis 4 to support this functionality, > we will undertake the work ourselves. Yes, we do plan C++11 support, including move support, for the next release. While it would be great to have someone give this a try before the final release, the bad news is that I will only have something to try probably around end-June. Until then my schedule is very tight. > It is a straightforward matter to modify xsd::cxx::tree::one<>, > optional<>, and sequence<> to support value semantics as long as > the types are not polymorphic. Right, we will have to have two different version of these. In fact, we already do, for fundamental and user-defined types. In C++11 mode we will just use the by-value version for all non-polymorphic types. > But we need to support polymorphism, too. That's where we run into a > hitch. I want the parsing functions to return values for value classes, > but they must return pointers for polymorphic classes. I certainly > don't want to force the user of a class to write different code > depending on whether it is polymorphic, I don't think this would be forcing anybody. Down the line, polymorphic and non-polymorphic roots will be handled in a very different ways. With polymorphic, there will probably be a dynamic_cast to discover the actual type (try supporting this with your pimpl approach). Generally, polymorphism and by-value are in conceptual contradiction. By-value assumes the size of the object is known and fixed (so stack can be allocated for it). Polymorphism assumes we don't even know which object it is, let alone its size. By-value is: here is a chunk of memory, construct object foo there. Polymorphism is: this chunk of memory contains some object that is-a foo. Using dynamic memory to return a polymorphic object is the "C++ way" (think of the virtual destructor mechanism -- it was augmented to call the correct operator delete). Well, that was a bit of philosophical aside ;-). Now, in ODB, for example, we have two versions of the database::load() function. One which returns a (dynamically-allocated) object and one that can load the state into an existing instance: person p; db.load (p, "John Doe"); This works well for when you want to avoid allocations. We could do this with parsing function, but I am not even sure it makes sense: How will you know which object is being parsed? The only way to do this generally is to examine the element name (or xsi:type). But then, seeing that you already have the DOM element, you can just call the parsing constructor directly and place the object anywhere you want. I guess this could still be useful for situations like "the response to this message can only be class foo". > so the only solution I see is for xsd to generate a pimpl wrapper > for polymorphic classes to hide the difference. There are so many issues/complications with this approach that I don't even know where to begin (inheritance, type discovery, code bloat, customizability, etc, etc). I don't think we will go this route. > "is there any other way to support value semantics and polymorphism?" I think we should just keep it simple and return values by-value and use pointers/dynamic memory for polymorphic objects. Parsing functions will return std::unique_ptr for polymorphic roots. Boris -- Follow this link to mark it as spam: http://mailfilter.proteus-technologies.com/cgi-bin/learn-msg.cgi?id=EC10B28341.AF0B4 From olitour at gmail.com Sat Feb 23 09:53:42 2013 From: olitour at gmail.com (Olivier Tournaire) Date: Sat Feb 23 09:54:29 2013 Subject: [xsd-users] Struggling with CityGML 2.0.0 parsing Message-ID: Hi all, This is my first post on the mailing list, and have already searched in the past message, but did not found a way to fix the issue I am currently facing while trying to parse a CityGML 2.0.0 file. I have generate the code using the 2.0.0 schemas available at http://www.citygml.org/?id=1540. I used the following options, mainly inspired by code synthesis wiki pages: * xAL / xlink --generate-polymorphic --polymorphic-type-all --generate-inline --generate-forward --disable-warning F001 --disable-warning F002 * smil: --generate-polymorphic --namespace-map http://www.w3.org/1999/02/22-rdf-syntax-ns#=rdif --anonymous-regex %.* transitionFilterPrototype/calcMode%calcMode2% --generate-inline --generate-forward --show-sloc --disable-warning F001 --disable-warning F002 * gml: --root-element-none --generate-polymorphic --polymorphic-type-all --generate-serialization --generate-inline --generate-forward --generate-doxygen --root-element Array --root-element Bag --disable-warning F001 --disable-warning F002 * citygml base: --root-element CityModel --generate-polymorphic --polymorphic-type-all --generate-inline --generate-forward --generate-serialization --generate-doxygen --namespace-map http://www.opengis.net/citygml/profiles/base/2.0=citygml --disable-warning F001 --disable-warning F002 * citygml: --root-element CityModel --generate-polymorphic --polymorphic-type-all --generate-serialization --generate-inline --generate-forward --generate-doxygen --namespace-map http://www.opengis.net/citygml/2.0=citygml --namespace-regex %.* http://www.opengis.net/citygml/([ ^/]*)/2.0%citygml::$1% --disable-warning F001 --disable-warning F002 The code seems to be well generated, and the doxygen doc is really usefull. However, I still have some warnings, but I do not know if they are important or if I can ignore them. I compile the code as a .so with g++ 4.6.3. For smil: /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd: warning T004: generating parsing functions for 4 global elements /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd: info: use --root-element-* options to specify document root(s) /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:35:34: warning T005: assuming type 'animateType' is polymorphic /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:37:96: info: because type 'animateType' is used in a substitution group declared here /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:35:34: info: use --polymorphic-type to indicate this type is polymorphic when compiling schemas that reference it /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:71:40: warning T005: assuming type 'animateMotionType' is polymorphic /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:38:114: info: because type 'animateMotionType' is used in a substitution group declared here /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:71:40: info: use --polymorphic-type to indicate this type is polymorphic when compiling schemas that reference it /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:87:39: warning T005: assuming type 'animateColorType' is polymorphic /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:39:111: info: because type 'animateColorType' is used in a substitution group declared here /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:87:39: info: use --polymorphic-type to indicate this type is polymorphic when compiling schemas that reference it /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:103:30: warning T005: assuming type 'setType' is polymorphic /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:40:84: info: because type 'setType' is used in a substitution group declared here /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:103:30: info: use --polymorphic-type to indicate this type is polymorphic when compiling schemas that reference it For xAL: /home/olivier/work/dev/citygml/CityGML_2.0.0/xAL/xAL.xsd: warning T004: generating parsing functions for 18 global elements /home/olivier/work/dev/citygml/CityGML_2.0.0/xAL/xAL.xsd: info: use --root-element-* options to specify document root(s) I don think that these warnings are a real problem, since I can link a test program against the created libraries without any problem. Now, I have a problem with a simple program trying to parse a CityGML file. Here is my code: #include #include "CityGML.hxx" #include "3.1.1/base/gml.hxx" #include using namespace std; using namespace citygml; using namespace citygml::building; using namespace gml; int main(int argc, char** argv) { string filename("/home/olivier/work/dev/citygml/CityGML_2.0.0/Examples/Building_LOD0-4/Building_and_garage_LOD2-EPSG25832.gml"); auto_ptr iCity( citygml::CityModel(filename) ); cout << "name --> " << iCity->name().front() << endl; if ( iCity->description().present() ) cout << "description --> " << iCity->description() << endl; else cout << "[no description available]" << endl; if ( iCity->id().present() ) cout << "id --> " << iCity->id() << endl; else cout << "[no id available]" << endl; if ( iCity->location().present() ) ; else cout << "[no location available]" << endl; if( iCity->boundedBy().present() ) { if( iCity->boundedBy()->Envelope().present() ) ; if( iCity->boundedBy()->Null().present() ) ; } else cout << "[no boundBy available]" << endl; cout << "Loaded model contains " << iCity->featureMember().size() << " features" << endl; AbstractFeatureCollectionType::featureMember_sequence::const_iterator it = iCity->featureMember().begin(), ite = iCity->featureMember().end(); for(;it!=ite;++it) { cout << "Looping..." << endl; FeaturePropertyType feature = *it; AbstractFeatureType featureType = feature._Feature().get(); if ( dynamic_cast(featureType._clone()) ) cout << "AbstractCityObjectType !" << endl; else if ( dynamic_cast(featureType._clone()) ) cout << "BuildingType !" << endl; } CityModelType cmt; cmt._GenericApplicationPropertyOfCityModel(); return 0; } The problem is in the for loop on the AbstractFeatureCollectionType:: featureMember_sequence . First I need to use the _clone() method, however g++ warns me that the dynamic_cast can never succeeds. The second point is that there are 2 features in my sample file (Building_and_garage_LOD2-EPSG25832.gml in the Examples directory). The first one is a building. Here is the begining of the file: 3D city model LOD2 without Appearance 458868.0 5438343.0 112.0 458892.0 5438362.0 117.0 Example Building LOD2 ..... I can access the name correctly, but none of my dynamic_cast actually succeeds, and I do not understand why. Could you please explain why, and what should I do to get the concrete type of each feature? Hope you could help. Best regards, Olivier From boris at codesynthesis.com Mon Feb 25 06:46:55 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Feb 25 06:47:03 2013 Subject: [xsd-users] C++11, value semantics, and polymorphism In-Reply-To: References: Message-ID: Hi Ray, Ray Lischner writes: > Our use of Code Synthesis often involves the construction of objects > from other objects, so we end up with code sort like this: > > A a(B("string", C(D("string", value), "string", "string"))); > > except that some of those types are polymorphic. Right now, we can create > all the intermediate objects dynamically and sling auto_ptr<>s around to > avoid excess copying. With C++ 11, we want to use value semantics where > possible, but that forces the user to know which types are polymorphic and > which are not, so we end up with the following: > > A a(B("string", new C(D("string", value), "string", "string"))); Hm, I think we can handle this without resorting to the pimpl stuff. Let's take a simple case: A a (D ("string")); Where the argument is of a polymorphic type B: A (const B&); Right now, to avoid copying you are doing essentially this: A a (new D ("string")); What if we overload the A() ctor and do "polymorphic move", similar to the "polymorphic copy" (aka clone), that we are already doing. In other words: A (const B& b) { b_ = b->_clone (); // _clone() uses copy ctor. } A (B&& b) { b_ = b->_move (); // _move() uses move ctor. } Now, when you write: A a (D ("string")); Compared to the dynamic allocation, you are only paying the penalty of constructing D on the stack (which will then be moved into the dynamically allocated instance inside _move()). Presumably this will be cheap. Here is how _move() is implemented: D* D:: _move () // Note: non-const, unlike _clone (). { return new D (std::move (*this)); // Using move ctor. } There is just one problem: the combinatorial explosion of constructors resulting from const-ref/rvalue-ref overloads. You can read more about the issue in this series of blog posts: http://www.codesynthesis.com/~boris/blog/2012/06/19/efficient-argument-passing-cxx11-part1/ Unfortunately, the solution that I propose (pass by value) won't work here (we will end up slicing polymorphic objects). Need to thinks more on this. Any ideas are welcome. Boris From olitour at gmail.com Mon Feb 25 14:42:12 2013 From: olitour at gmail.com (Olivier Tournaire) Date: Tue Feb 26 03:16:08 2013 Subject: [xsd-users] Re: Struggling with CityGML 2.0.0 parsing In-Reply-To: References: Message-ID: Hi all, I finally managed to make my simple test program work. I had tu use references, and all work like a charm: AbstractFeatureCollectionType::featureMember_sequence::const_iterator it = iCity->featureMember().begin(), ite = iCity->featureMember().end(); for(;it!=ite;++it) { // Note the use of references!!! const gml::FeaturePropertyType& f = (*it); const gml::AbstractFeatureType& af = f._Feature().get(); if (BuildingType* b = dynamic_cast (af._clone())) { cout << "Got a Building!" << endl; cout << b->_GenericApplicationPropertyOfBuilding ().size() << endl; } else if (ReliefFeatureType* b = dynamic_cast (af._clone())) { cout << "Got a ReliefFeature!" << endl; } } However, it works, but only on ubuntu ... I tried today to generate the code on windows with VC++ 2010 and struggled with DLL import / export symbols. I finally found that with the options: --export-symbol GML_DLL_DEF --hxx-prologue #include "gml_dll_def.hpp" where gml_dll_def.hpp is classically: #ifndef __GML_DLL_DEF_HPP__ #define __GML_DLL_DEF_HPP__ #if defined(_WIN32) # if defined(gml_EXPORTS) # define GML_DLL_DEF __declspec(dllexport) # else # define GML_DLL_DEF __declspec(dllimport) # endif #else # define GML_DLL_DEF #endif #endif // __GMLL_DLL_DEF_HPP__ Of course, these options are added for each dll. I also add to use the /FORCE:MULTIPLE option in VC++2010 to prevent link errors on "already defined symbol ...". Everything works, except for CityGML. This library (dll) does not link and gives me the following errors: Link: Cr?ation de la biblioth?que F:/work/dev/citygml2-0-0/build_unstable/Release/citygml.lib et de l'objet F:/work/dev/citygml2-0-0/build_unstable/Release/citygml.exp tunnel.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) vegetation.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) waterBody.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) landUse.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) relief.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) texturedSurface.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) transportation.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) cityFurniture.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) cityGMLBase.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) cityObjectGroup.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) generics.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) CityGML.obj : error LNK2019: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) r?f?renc? dans la fonction "unsigned __int64 __cdecl xsd::cxx::tree::bits::find_ns(wchar_t const *,unsigned __int64,unsigned __int64)" (??$find_ns@_W@bits@tree@cxx@xsd@@YA_KPEB_W_K1@Z) appearance.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) bridge.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) building.obj : error LNK2001: symbole externe non r?solu "public: static unsigned __int64 const std::basic_string,class std::allocator >::npos" (?npos@ ?$basic_string@_WU?$char_traits@_W@std@@V?$allocator@_W@2@@std@@2_KB) F:\work\dev\citygml2-0-0\build_unstable\Release\citygml.dll : fatal error LNK1120: 1 externes non r?solus I googled for this error, and found some old threads, but did not manged to make a trick for my case: http://www.codesynthesis.com/pipermail/xsd-users/2009-October/002490.html and http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003010.html Could you please give me some hint on how to solve this issue? It is a real problem, and I do not understand how to properly use --export-xml-schema. Note that my project contains 5 DLLs: * xlink * smil * xAL * gml * citygml Dependencies are as follow: * gml depends on xlink and smil * citygml depends on gml and xAL Any help would really be appreciated. Best regards, Olivier 2013/2/23 Olivier Tournaire > Hi all, > > This is my first post on the mailing list, and have already searched in > the past message, but did not found a way to fix the issue I am currently > facing while trying to parse a CityGML 2.0.0 file. > > I have generate the code using the 2.0.0 schemas available at > http://www.citygml.org/?id=1540. I used the following options, mainly > inspired by code synthesis wiki pages: > > * xAL / xlink > --generate-polymorphic > --polymorphic-type-all > --generate-inline > --generate-forward > --disable-warning F001 > --disable-warning F002 > > * smil: > --generate-polymorphic > --namespace-map http://www.w3.org/1999/02/22-rdf-syntax-ns#=rdif > --anonymous-regex %.* transitionFilterPrototype/calcMode%calcMode2% > --generate-inline > --generate-forward > --show-sloc > --disable-warning F001 > --disable-warning F002 > > * gml: > --root-element-none > --generate-polymorphic > --polymorphic-type-all > --generate-serialization > --generate-inline > --generate-forward > --generate-doxygen > --root-element Array > --root-element Bag > --disable-warning F001 > --disable-warning F002 > > * citygml base: > --root-element CityModel > --generate-polymorphic > --polymorphic-type-all > --generate-inline > --generate-forward > --generate-serialization > --generate-doxygen > --namespace-map http://www.opengis.net/citygml/profiles/base/2.0=citygml > --disable-warning F001 > --disable-warning F002 > > * citygml: > --root-element CityModel > --generate-polymorphic > --polymorphic-type-all > --generate-serialization > --generate-inline > --generate-forward > --generate-doxygen > --namespace-map http://www.opengis.net/citygml/2.0=citygml > --namespace-regex %.* http://www.opengis.net/citygml/([ > ^/]*)/2.0%citygml::$1% > --disable-warning F001 > --disable-warning F002 > > > The code seems to be well generated, and the doxygen doc is really > usefull. However, I still have some warnings, but I do not know if they are > important or if I can ignore them. I compile the code as a .so with g++ > 4.6.3. > > For smil: > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd: > warning T004: generating parsing functions for 4 global elements > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd: info: > use --root-element-* options to specify document root(s) > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:35:34: > warning T005: assuming type 'animateType' is polymorphic > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:37:96: > info: because type 'animateType' is used in a substitution group declared > here > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:35:34: > info: use --polymorphic-type to indicate this type is polymorphic when > compiling schemas that reference it > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:71:40: > warning T005: assuming type 'animateMotionType' is polymorphic > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:38:114: > info: because type 'animateMotionType' is used in a substitution group > declared here > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:71:40: > info: use --polymorphic-type to indicate this type is polymorphic when > compiling schemas that reference it > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:87:39: > warning T005: assuming type 'animateColorType' is polymorphic > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:39:111: > info: because type 'animateColorType' is used in a substitution group > declared here > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:87:39: > info: use --polymorphic-type to indicate this type is polymorphic when > compiling schemas that reference it > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:103:30: > warning T005: assuming type 'setType' is polymorphic > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:40:84: > info: because type 'setType' is used in a substitution group declared here > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:103:30: > info: use --polymorphic-type to indicate this type is polymorphic when > compiling schemas that reference it > > For xAL: > /home/olivier/work/dev/citygml/CityGML_2.0.0/xAL/xAL.xsd: warning T004: > generating parsing functions for 18 global elements > /home/olivier/work/dev/citygml/CityGML_2.0.0/xAL/xAL.xsd: info: use > --root-element-* options to specify document root(s) > > I don think that these warnings are a real problem, since I can link a > test program against the created libraries without any problem. Now, I have > a problem with a simple program trying to parse a CityGML file. > > Here is my code: > > #include > > > #include "CityGML.hxx" > > #include "3.1.1/base/gml.hxx" > > > #include > > > using namespace std; > > using namespace citygml; > > using namespace citygml::building; > > using namespace gml; > > > int main(int argc, char** argv) > > { > > string filename("/home/olivier/work/dev/citygml/CityGML_2.0.0/Examples/Building_LOD0-4/Building_and_garage_LOD2-EPSG25832.gml"); > > auto_ptr iCity( citygml::CityModel(filename) ); > > > cout << "name --> " << iCity->name().front() << endl; > > if ( iCity->description().present() ) > > cout << "description --> " << iCity->description() << endl; > > else > > cout << "[no description available]" << endl; > > if ( iCity->id().present() ) > > cout << "id --> " << iCity->id() << endl; > > else > > cout << "[no id available]" << endl; > > if ( iCity->location().present() ) > > ; > > else > > cout << "[no location available]" << endl; > > if( iCity->boundedBy().present() ) > > { > > if( iCity->boundedBy()->Envelope().present() ) > > ; > > if( iCity->boundedBy()->Null().present() ) > > ; > > } > > else > > cout << "[no boundBy available]" << endl; > > > cout << "Loaded model contains " << iCity->featureMember().size() << " features" << endl; > > > AbstractFeatureCollectionType::featureMember_sequence::const_iterator it = iCity->featureMember().begin(), ite = iCity->featureMember().end(); > > for(;it!=ite;++it) > > { > > cout << "Looping..." << endl; > > FeaturePropertyType feature = *it; > > AbstractFeatureType featureType = feature._Feature().get(); > > > if ( dynamic_cast(featureType._clone()) ) > > cout << "AbstractCityObjectType !" << endl; > > else if ( dynamic_cast(featureType._clone()) ) > > cout << "BuildingType !" << endl; > > } > > > CityModelType cmt; > > cmt._GenericApplicationPropertyOfCityModel(); > > return 0; > > } > > > > The problem is in the for loop on the AbstractFeatureCollectionType:: > featureMember_sequence . First I need to use the _clone() method, however > g++ warns me that the dynamic_cast can never succeeds. The second point is > that there are 2 features in my sample file > (Building_and_garage_LOD2-EPSG25832.gml in the Examples directory). The > first one is a building. Here is the begining of the file: > > > > > > > > xmlns:xAL="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0" xmlns:xlink=" > http://www.w3.org/1999/xlink" > xmlns:gml="http://www.opengis.net/gml" xmlns:dem=" > http://www.opengis.net/citygml/relief/2.0" > xmlns:bldg="http://www.opengis.net/citygml/building/2.0" > xsi:schemaLocation="http://www.opengis.net/citygml/building/2.0../../CityGML/building.xsd > http://www.opengis.net/citygml/relief/2.0 ../../CityGML/relief.xsd"> > 3D city model LOD2 without Appearance > > srsName="urn:ogc:def:crs,crs:EPSG::25832,crs:EPSG::5783"> > 458868.0 5438343.0 112.0 > 458892.0 5438362.0 117.0 > > > > > Example Building LOD2 > ..... > > I can access the name correctly, but none of my dynamic_cast actually > succeeds, and I do not understand why. Could you please explain why, and > what should I do to get the concrete type of each feature? > > Hope you could help. > > Best regards, > > Olivier > > > > From boris at codesynthesis.com Tue Feb 26 04:05:53 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Feb 26 04:05:59 2013 Subject: [xsd-users] toString() method for basic datatypes Message-ID: Hi Harald, In the future please keep your replies CC'ed to the xsd-users mailing list as discussed in the posting guidelines: http://www.codesynthesis.com/support/posting-guidelines.xhtml Harald Klingel writes: > first of all, many thanks for the whole project. From my point of view, it > is the best xsd to c++ converter available. Thanks, I am glad you like it. > Within the generated classes there are enums. As I code it by myself, > the enum values are mapped to the value they are related to. But how > can I get that value if I have the enum value? The XML Schema enumeration mapping in XSD has dual interface, that of a C++ enum and of a string. I assume by "enum value" you mean the C++ enumerator (e.g., color::red) and by "value they are related to" you mean the string value as in the schema/XML (e.g., "red"). If that's the case, the you can do from/to string conversions like this: color r ("red"); const std::string& s (r); Boris From boris at codesynthesis.com Tue Feb 26 04:21:53 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Feb 26 04:22:00 2013 Subject: [xsd-users] Re: Struggling with CityGML 2.0.0 parsing In-Reply-To: References: Message-ID: Hi Olivier, Olivier Tournaire writes: > I finally managed to make my simple test program work. I had tu use > references, and all work like a charm: Glad you figured it out. I just couldn't bring myself to point out the same mistake probably a hundredth time. Note that you are still leaking memory with your (unnecessary) _clone() calls. Just cast the object itself instead of its copy: if (const BuildingType* b = dynamic_cast (&af)) > --export-symbol GML_DLL_DEF > --hxx-prologue #include "gml_dll_def.hpp" > > where gml_dll_def.hpp is classically: > > #ifndef __GML_DLL_DEF_HPP__ > #define __GML_DLL_DEF_HPP__ > > [...] > > #endif // __GMLL_DLL_DEF_HPP__ > > Of course, these options are added for each dll. You ave a separate file like this for each DLL, right? > I also add to use the /FORCE:MULTIPLE option in VC++2010 to prevent > link errors on "already defined symbol ...". That shouldn't be necessary. There must be something wrong with your export setup. > Could you please give me some hint on how to solve this issue? It is a real > problem, and I do not understand how to properly use --export-xml-schema. This thread has step-by-step instructions: http://www.codesynthesis.com/pipermail/xsd-users/2010-September/003011.html > Note that my project contains 5 DLLs: > * xlink > * smil > * xAL > * gml > * citygml > > Dependencies are as follow: > * gml depends on xlink and smil > * citygml depends on gml and xAL Do you really need all these DLLs? GML/CityGML is already complex enough. Trying to package all the components into separate DLLs on Windows is just looking for trouble, IMO. So if I were you, I would just put everything in a single DLL (or even link it directly to the executable). Keep it simple. If you still want all the DLLs, then to work around the std::basic_string issue, you will need to create yet another, "root" DLL on which all other DLLs will depend and which will contain the xml-schema.hxx (see the link above for details). Boris From olitour at gmail.com Tue Feb 26 04:25:11 2013 From: olitour at gmail.com (Olivier Tournaire) Date: Tue Feb 26 04:25:58 2013 Subject: [xsd-users] Re: Struggling with CityGML 2.0.0 parsing In-Reply-To: References: Message-ID: Hi all, It seems that using FORCE:MULTIPLE is not a good idea ... This blog page talk about this issue: http://codesynthesis.com/~boris/blog/2010/01/18/dll-export-cxx-templates/#comments To fix the link errors previously mentionned, I tried this workaround, which make citygml links, but, at execution, the parsing fails: #if defined(_WIN32) // Hell begins here // See http://connect.microsoft.com/VisualStudio/feedback/details/586959/std-string-npos-unresolved-external-symbol-when-optimization-o2-o1-or-ox-enabled # include # if _MSC_VER >= 1600 # if !defined(NPOS_VC2010_WORKAROUND) /* */const std::basic_string::size_type std::basic_string::npos = (std::basic_string::size_type) -1; # define NPOS_VC2010_WORKAROUND # endif // NPOS_VC2010_WORKAROUND // Hell ends here # endif _MSC_VER # if defined(citygml_EXPORTS) # define CITYGML_DLL_DEF __declspec(dllexport) # else # define CITYGML_DLL_DEF __declspec(dllimport) # endif #else # define CITYGML_DLL_DEF #endif The only (working) solution I found is to generate a single DLL for xlink, xAL, smil, gml and citygml. But, IMHO, it is not a viable solution at long term since CityGML offers the ability to developp ADEs. Each ADE will need to have its own DLL, and I might be facing the problem again when I will need to generate the code for an ADE. 2013/2/23 Olivier Tournaire > Hi all, > > This is my first post on the mailing list, and have already searched in > the past message, but did not found a way to fix the issue I am currently > facing while trying to parse a CityGML 2.0.0 file. > > I have generate the code using the 2.0.0 schemas available at > http://www.citygml.org/?id=1540. I used the following options, mainly > inspired by code synthesis wiki pages: > > * xAL / xlink > --generate-polymorphic > --polymorphic-type-all > --generate-inline > --generate-forward > --disable-warning F001 > --disable-warning F002 > > * smil: > --generate-polymorphic > --namespace-map http://www.w3.org/1999/02/22-rdf-syntax-ns#=rdif > --anonymous-regex %.* transitionFilterPrototype/calcMode%calcMode2% > --generate-inline > --generate-forward > --show-sloc > --disable-warning F001 > --disable-warning F002 > > * gml: > --root-element-none > --generate-polymorphic > --polymorphic-type-all > --generate-serialization > --generate-inline > --generate-forward > --generate-doxygen > --root-element Array > --root-element Bag > --disable-warning F001 > --disable-warning F002 > > * citygml base: > --root-element CityModel > --generate-polymorphic > --polymorphic-type-all > --generate-inline > --generate-forward > --generate-serialization > --generate-doxygen > --namespace-map http://www.opengis.net/citygml/profiles/base/2.0=citygml > --disable-warning F001 > --disable-warning F002 > > * citygml: > --root-element CityModel > --generate-polymorphic > --polymorphic-type-all > --generate-serialization > --generate-inline > --generate-forward > --generate-doxygen > --namespace-map http://www.opengis.net/citygml/2.0=citygml > --namespace-regex %.* http://www.opengis.net/citygml/([ > ^/]*)/2.0%citygml::$1% > --disable-warning F001 > --disable-warning F002 > > > The code seems to be well generated, and the doxygen doc is really > usefull. However, I still have some warnings, but I do not know if they are > important or if I can ignore them. I compile the code as a .so with g++ > 4.6.3. > > For smil: > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd: > warning T004: generating parsing functions for 4 global elements > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd: info: > use --root-element-* options to specify document root(s) > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:35:34: > warning T005: assuming type 'animateType' is polymorphic > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:37:96: > info: because type 'animateType' is used in a substitution group declared > here > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:35:34: > info: use --polymorphic-type to indicate this type is polymorphic when > compiling schemas that reference it > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:71:40: > warning T005: assuming type 'animateMotionType' is polymorphic > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:38:114: > info: because type 'animateMotionType' is used in a substitution group > declared here > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:71:40: > info: use --polymorphic-type to indicate this type is polymorphic when > compiling schemas that reference it > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:87:39: > warning T005: assuming type 'animateColorType' is polymorphic > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:39:111: > info: because type 'animateColorType' is used in a substitution group > declared here > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:87:39: > info: use --polymorphic-type to indicate this type is polymorphic when > compiling schemas that reference it > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:103:30: > warning T005: assuming type 'setType' is polymorphic > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20.xsd:40:84: > info: because type 'setType' is used in a substitution group declared here > /home/olivier/work/dev/citygml/CityGML_2.0.0/3.1.1/smil/smil20-language.xsd:103:30: > info: use --polymorphic-type to indicate this type is polymorphic when > compiling schemas that reference it > > For xAL: > /home/olivier/work/dev/citygml/CityGML_2.0.0/xAL/xAL.xsd: warning T004: > generating parsing functions for 18 global elements > /home/olivier/work/dev/citygml/CityGML_2.0.0/xAL/xAL.xsd: info: use > --root-element-* options to specify document root(s) > > I don think that these warnings are a real problem, since I can link a > test program against the created libraries without any problem. Now, I have > a problem with a simple program trying to parse a CityGML file. > > Here is my code: > > #include > > > #include "CityGML.hxx" > > #include "3.1.1/base/gml.hxx" > > > #include > > > using namespace std; > > using namespace citygml; > > using namespace citygml::building; > > using namespace gml; > > > int main(int argc, char** argv) > > { > > string filename("/home/olivier/work/dev/citygml/CityGML_2.0.0/Examples/Building_LOD0-4/Building_and_garage_LOD2-EPSG25832.gml"); > > auto_ptr iCity( citygml::CityModel(filename) ); > > > cout << "name --> " << iCity->name().front() << endl; > > if ( iCity->description().present() ) > > cout << "description --> " << iCity->description() << endl; > > else > > cout << "[no description available]" << endl; > > if ( iCity->id().present() ) > > cout << "id --> " << iCity->id() << endl; > > else > > cout << "[no id available]" << endl; > > if ( iCity->location().present() ) > > ; > > else > > cout << "[no location available]" << endl; > > if( iCity->boundedBy().present() ) > > { > > if( iCity->boundedBy()->Envelope().present() ) > > ; > > if( iCity->boundedBy()->Null().present() ) > > ; > > } > > else > > cout << "[no boundBy available]" << endl; > > > cout << "Loaded model contains " << iCity->featureMember().size() << " features" << endl; > > > AbstractFeatureCollectionType::featureMember_sequence::const_iterator it = iCity->featureMember().begin(), ite = iCity->featureMember().end(); > > for(;it!=ite;++it) > > { > > cout << "Looping..." << endl; > > FeaturePropertyType feature = *it; > > AbstractFeatureType featureType = feature._Feature().get(); > > > if ( dynamic_cast(featureType._clone()) ) > > cout << "AbstractCityObjectType !" << endl; > > else if ( dynamic_cast(featureType._clone()) ) > > cout << "BuildingType !" << endl; > > } > > > CityModelType cmt; > > cmt._GenericApplicationPropertyOfCityModel(); > > return 0; > > } > > > > The problem is in the for loop on the AbstractFeatureCollectionType:: > featureMember_sequence . First I need to use the _clone() method, however > g++ warns me that the dynamic_cast can never succeeds. The second point is > that there are 2 features in my sample file > (Building_and_garage_LOD2-EPSG25832.gml in the Examples directory). The > first one is a building. Here is the begining of the file: > > > > > > > > xmlns:xAL="urn:oasis:names:tc:ciq:xsdschema:xAL:2.0" xmlns:xlink=" > http://www.w3.org/1999/xlink" > xmlns:gml="http://www.opengis.net/gml" xmlns:dem=" > http://www.opengis.net/citygml/relief/2.0" > xmlns:bldg="http://www.opengis.net/citygml/building/2.0" > xsi:schemaLocation="http://www.opengis.net/citygml/building/2.0../../CityGML/building.xsd > http://www.opengis.net/citygml/relief/2.0 ../../CityGML/relief.xsd"> > 3D city model LOD2 without Appearance > > srsName="urn:ogc:def:crs,crs:EPSG::25832,crs:EPSG::5783"> > 458868.0 5438343.0 112.0 > 458892.0 5438362.0 117.0 > > > > > Example Building LOD2 > ..... > > I can access the name correctly, but none of my dynamic_cast actually > succeeds, and I do not understand why. Could you please explain why, and > what should I do to get the concrete type of each feature? > > Hope you could help. > > Best regards, > > Olivier > > > > From boris at codesynthesis.com Tue Feb 26 04:57:10 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Feb 26 04:57:16 2013 Subject: [xsd-users] Re: Struggling with CityGML 2.0.0 parsing In-Reply-To: References: Message-ID: Hi Olivier, Olivier Tournaire writes: > // Hell begins here > // See > > [...] > > // Hell ends here No, this is not a solution. > The only (working) solution I found is to generate a single DLL for xlink, > xAL, smil, gml and citygml. But, IMHO, it is not a viable solution at long > term since CityGML offers the ability to developp ADEs. Each ADE will need > to have its own DLL, and I might be facing the problem again when I will > need to generate the code for an ADE. Try the root DLL and --export-xml-schema approach then. If you set everything up correctly (i.e., std::basic_string is exported from the root DLL and imported by all other DLLs and by the executable), then everything should work. Boris From dawhite32 at gmail.com Wed Feb 27 22:05:32 2013 From: dawhite32 at gmail.com (David White) Date: Wed Feb 27 22:05:39 2013 Subject: [xsd-users] GML 3.2.1 and xsd-parser Message-ID: Hi Boris and xsd-users, May I echo what many others have said - many thanks for your tireless work and support on this excellent product! I was wondering if you could provide a brief update on current status of C++/Parser handling GML. Looking through the xsd-users mail, I understand there has been quite a bit of action in the GML space since Oleg's email below, and I have read your wiki on GML ( http://wiki.codesynthesis.com/Schemas/GML). Recently, I have been trying to generate c++source for a GML 3.2.1 application schema using xsd-parser and am coming up with the same error as Oleg: gml\3.2.1\dictionary.xsd:34:79: error: unable to match restricted particle Is there a solution to this or should I continue to follow your advice to Oleg below. Thanks. David. On Fri, Apr 17, 2009 at 12:40 AM, Boris Kolpackov wrote: > Hi Oleg, > > Oleg Dedkow writes: > > > I'm trying to generate C++ classes for the CityGML 1.0.0 schemas (please, > > see the attached ZIP file) using C++/Parser interface. I'm using xsd > 3.2.0 > > and was able to generate C++ classes using C++/Tree interface. However, > > this does not fit my needs, since I have to handle very large CityGML > > files that could not fit into memory. I get the following error using > > C++/Parser interface: > > > > dictionary.xsd:43:79: error: unable to match restricted particle > > The C++/Parser mapping is not yet capable of handling GML. GML uses > non-trivial complex type restrictions that are not yet support. Also, > the GML schemas are very complex, they use XML Schema polymorphism > extensively, so handling CityGML document with C++/Parser is going > to be a lot of hard work. > > If I were in your situation I would use C++/Tree but parse the documents > chunk by chunk. Here is how I would do it: I would create a top-level SAX > parser which constructs DOM fragments for portions of the document. Once > the chunk is ready, the parser creates the corresponding C++/Tree object > model and frees the DOM fragment. The object model can then be used by the > application. Once the object model is processed, it is freed and the SAX > parser moves on to the next chunk. This way you can handle fairly large > CityGML/GML documents and remain sane at the same time ;-). > > As an example, consider this XML document: > > > >
> Sample records > sample001 >
> > > ... > > > ... > > > ... > > > > > With the above approach the SAX parser will first create a DOM fragment > corresponding to: > >
> Sample records > sample001 >
> > And parse it to the object model. This object model can then be kept > around or processed and discarded. Then the parser will create a DOM > fragment for each record, parse it to the object model, process, and > discard. > > This approach is very similar to the partially in-memory/partially > even-driven XML processing that we do in XSD/e (in embedded systems > even relatively small documents can be too large to fit into RAM). > See Section 6.1, "Customizing Parsers and Serializers" in the > Embedded C++/Hybrid Mapping Getting Started Guide for details on > how this works in XSD/e: > > http://codesynthesis.com/projects/xsde/documentation/cxx/hybrid/guide/#6.1 > > If you are interested, I could create an example that shows how to do > all this with C++/Tree. > > > > The above-mentioned error is caused by the dictionary.xsd schema which > > [...] will it be possible to create some workaround? It would be OK for > > me to edit the local GML schemas. > > You could probably "relax" the GML schema a bit and make it compile with > C++/Parser. What causes the above error is the use of an element > substitution in restriction. Here is an example (here Derived1 is derived- > by-restriction from Base1): > > > > > > > > > > > > > > > > > > > > > You can "relax" the above schema by using base instead of derived in > Derived2. > > Boris > > From boris at codesynthesis.com Thu Feb 28 01:21:24 2013 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Feb 28 01:21:31 2013 Subject: [xsd-users] GML 3.2.1 and xsd-parser In-Reply-To: References: Message-ID: Hi David, David White writes: > May I echo what many others have said - many thanks for your tireless work > and support on this excellent product! Thanks, I am glad you like it. > I was wondering if you could provide a brief update on current status of > C++/Parser handling GML. > > [...] > > Is there a solution to this or should I continue to follow your advice to > Oleg below. Yes, that advice is still valid. In addition, XSD now includes an example, called 'streaming', that shows how to implement progressive parsing and serialization described in that email. Boris