From boris at codesynthesis.com Thu Apr 1 03:46:29 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 1 03:37:39 2010 Subject: [xsd-users] Non intrusive data structures In-Reply-To: <37CA3BD6-C944-4508-84CA-EF1EBE0CB6AD@singularsoftware.com> References: <5099AD0F-C710-49CA-B138-E46C8223AD6B@singularsoftware.com> <37CA3BD6-C944-4508-84CA-EF1EBE0CB6AD@singularsoftware.com> Message-ID: Hi Barrie, Barrie Kovish writes: > Is the existing beta stable with VC++ 2008? Yes, there are no known issues with this compiler and the beta. Boris From toby.ealden at gmail.com Thu Apr 1 07:41:35 2010 From: toby.ealden at gmail.com (Toby Ealden) Date: Thu Apr 1 07:42:03 2010 Subject: [xsd-users] how to achieve XPath support Message-ID: Hi I can't seem to find any clear information about whether/how XPath is supported. The documentation implies it is, and I've seen this example ( http://www.codesynthesis.com/~boris/blog/2009/05/18/running-xpath-on-cxx-tree-object-model/) but it seems to be using something called XQilla to provide the xpath support. What I'm trying to do is locate a node in the tree using xpath and cast up to the C++ class, change a value and then write the data back. I'm clear on how to do all of this except for the first bit - locate a node in the tree using xpath. Is this possible and if so can anybody give me step by step instructions of how to achieve it, for example - 'install xqilla, configure this, link with library xyz.lib etc'. Many thanks in advance! Toby From boris at codesynthesis.com Thu Apr 1 09:43:22 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 1 09:34:25 2010 Subject: [xsd-users] how to achieve XPath support In-Reply-To: References: Message-ID: Hi Toby, Toby Ealden writes: > I can't seem to find any clear information about whether/how XPath is > supported. [...] > > What I'm trying to do is locate a node in the tree using xpath and cast up > to the C++ class, change a value and then write the data back. I'm clear on > how to do all of this except for the first bit - locate a node in the tree > using xpath. > > Is this possible and if so can anybody give me step by step instructions of > how to achieve it, for example - 'install xqilla, configure this, link with > library xyz.lib etc'. Yes, this is possible. The step-by-step instructions would be compiler and platform-specific (both which you failed to mention, BTW). But generally, you need to download, build, and install the XQilla library in addition to Xerces-C++ and then link both to your application. There is also the 'xpath' example in the examples/cxx/tree/ directory of the XSD distribution which shows how to use XPath with C++/Tree. It includes the README file which describes the example in detail as well as the makefiles and project/solution files. So you may want to get this example working first and then base your own code on it. Boris From rlischner at proteuseng.com Mon Apr 12 13:18:16 2010 From: rlischner at proteuseng.com (Ray Lischner) Date: Mon Apr 12 14:59:51 2010 Subject: [xsd-users] Adding error tests to xsd Message-ID: We made some changes to xsd-3.2.0, and have added tests for our changes. Among the changes are new error cases, which we also want to test. The problem is that I cannot figure out how to incorporate the error tests into the build framework. In particular, I want to try running xsd and expect it to exit with a non-zero return code. If xsd exits with a zero return code, that indicates a test failure. Then I want to check the output from xsd and veriyf that it contains the error message that I expect. The build framework is very easy to change for adding conventional files and build rules, but I do not understand it well enough to add new kinds of build rules and actions. I appreciate any help you can offer. Thank you. Ray Lischner, Senior Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.5170 rlischner@proteuseng.com f. 443.539.3370 This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message. From rlischner at proteuseng.com Mon Apr 12 14:55:31 2010 From: rlischner at proteuseng.com (Ray Lischner) Date: Mon Apr 12 14:59:52 2010 Subject: [xsd-users] xsd bypasses customized types Message-ID: We are only now starting the transition to xsd 3.2 (just in time for 3.3!), and we are running into a small problem. We customized a number of types. For example, we have a type CUnsignedInt, which is a restriction of an XML string. The customized type parses the string as a C-style integer and can treat the resulting object as an integer. When serialized to CDR, the customized type stores a binary integer instead of a string. This worked fine in xsd 2.3, but causes problems in xsd 3.2. Among the new features is that any string-based type gets some new constructors, including a default constructor. If a schema derives a type from a string-based type, the derived type get those same constructors. Thus, if a schema derives a type from our CUnsignedInt type, that derived type gets a default constructor that invokes its base class's default constructor. We never defined a default constructor for CUnsignedInt, but we can add one to initialize the numeric value to zero. More serious is that the code generation does not generate a from-base constructor, that is, one cannot construct an instance of the derived type from an instance of CUnsignedInt. This makes it considerably harder to construct an instance of the derived type from an integer. The developer would have to convert the integer to a string, and use the string as the argument to the constructor. This violates the principles of the CUnsignedInt type, which looks and acts as an integer (from the developer's point of view). Only when serializing and parsing XML does CUnsignedInt reveal its string nature. I don't mind it when new features change the way I need to write my customized types. I understand that I must accommodate my customizations. But this change breaks all the code that uses my customization. I cannot shield users from the change by modifying my customization. So what can we do? At this time, we are considering modifying the xsd compiler to add a switch that would disable the new code that inhibits the from-base constructors and inserts the new string-based constructors. But is that the best solution? Thanks for the help. Ray Lischner, Senior Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.5170 rlischner@proteuseng.com f. 443.539.3370 This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message. From boris at codesynthesis.com Tue Apr 13 10:58:52 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Apr 13 10:49:27 2010 Subject: [xsd-users] Adding error tests to xsd In-Reply-To: References: Message-ID: Hi Ray, Ray Lischner writes: > In particular, I want to try running xsd and expect it to exit with a > non-zero return code. If xsd exits with a zero return code, that > indicates a test failure. Then I want to check the output from xsd > and veriyf that it contains the error message that I expect. Here is the makefiles that is based on the tests/cxx/tree/test-template that should do what you want: include $(dir $(lastword $(MAKEFILE_LIST)))../../../../build/bootstrap.make test := $(out_base)/.test # Do-nothing default target # $(out_base)/: $(driver) # Test. # $(test): xsd := $(out_root)/xsd/xsd $(test): $(xsd) $(src_base)/test.xsd $(src_base)/output $(call message,test $$1,$(xsd) cxx-tree $$1 || echo error expected | diff -u $(src_base)/output -,$(src_base)/test.xsd) # Dependencies. # $(call import,$(src_root)/xsd/makefile) If you want to run several test from the same makefile, take a look, for example, at tests/cxx/parser/validation/all/makefile. Boris From rlischner at proteuseng.com Tue Apr 13 11:45:50 2010 From: rlischner at proteuseng.com (Ray Lischner) Date: Tue Apr 13 11:49:12 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: After further investigation, I think I better understand what is happening. Suppose type A is a restriction of unsignedInt, and type B is a restriction of A. When xsd generates constructors for B, it suppresses the B::B(A&) constructor, and instead issues only B::B(unsigned int), that is, because A has no required elements or attributes, B's immediate-base-class constructor is suppressed in favor of B's ultimate-base-class constructor. This works fine until we customize type A. Somehow, we need to tell xsd to favor the immediate-base-class constructor and instead suppress the ultimate-base-class constructor. Thus, the B constructor can take an A object as its from-base constructor, and we can construct an A with all its customizations. Ray Lischner, Senior Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.5170 rlischner@proteuseng.com f. 443.539.3370 This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message. ________________________________________ From: xsd-users-bounces@codesynthesis.com [xsd-users-bounces@codesynthesis.com] On Behalf Of Ray Lischner [rlischner@proteuseng.com] Sent: Monday, April 12, 2010 2:55 PM To: xsd-users@codesynthesis.com Cc: rcresswell@gmail.com Subject: [xsd-users] xsd bypasses customized types We are only now starting the transition to xsd 3.2 (just in time for 3.3!), and we are running into a small problem. We customized a number of types. For example, we have a type CUnsignedInt, which is a restriction of an XML string. The customized type parses the string as a C-style integer and can treat the resulting object as an integer. When serialized to CDR, the customized type stores a binary integer instead of a string. This worked fine in xsd 2.3, but causes problems in xsd 3.2. Among the new features is that any string-based type gets some new constructors, including a default constructor. If a schema derives a type from a string-based type, the derived type get those same constructors. Thus, if a schema derives a type from our CUnsignedInt type, that derived type gets a default constructor that invokes its base class's default constructor. We never defined a default constructor for CUnsignedInt, but we can add one to initialize the numeric value to zero. More serious is that the code generation does not generate a from-base constructor, that is, one cannot construct an instance of the derived type from an instance of CUnsignedInt. This makes it considerably harder to construct an instance of the derived type from an integer. The developer would have to convert the integer to a string, and use the string as the argument to the constructor. This violates the principles of the CUnsignedInt type, which looks and acts as an integer (from the developer's point of view). Only when serializing and parsing XML does CUnsignedInt reveal its string nature. I don't mind it when new features change the way I need to write my customized types. I understand that I must accommodate my customizations. But this change breaks all the code that uses my customization. I cannot shield users from the change by modifying my customization. So what can we do? At this time, we are considering modifying the xsd compiler to add a switch that would disable the new code that inhibits the from-base constructors and inserts the new string-based constructors. But is that the best solution? Thanks for the help. Ray Lischner, Senior Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.5170 rlischner@proteuseng.com f. 443.539.3370 This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message. From boris at codesynthesis.com Tue Apr 13 12:07:18 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Apr 13 11:57:49 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Hi Ray, Ray Lischner writes: > After further investigation, I think I better understand what is > happening. Suppose type A is a restriction of unsignedInt, and > type B is a restriction of A. When xsd generates constructors for > B, it suppresses the B::B(A&) constructor, and instead issues only > B::B(unsigned int), that is, because A has no required elements or > attributes, B's immediate-base-class constructor is suppressed in > favor of B's ultimate-base-class constructor. Yes, that's exactly what happens. And the reason for this is that in such cases B::B(A&) does not add any new functionality since A can be converted without loss of information to its base. > This works fine until we customize type A. Somehow, we need to > tell xsd to favor the immediate-base-class constructor and > instead suppress the ultimate-base-class constructor. We can add some extra intelligence to this logic and avoid suppressing the B(A&) c-tor if the hierarchy includes a customized type. I don't think we can safely suppress the other c-tor thought. Would having both c-tors work for you? From rlischner at proteuseng.com Tue Apr 13 12:01:07 2010 From: rlischner at proteuseng.com (Ray Lischner) Date: Tue Apr 13 14:27:37 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: , Message-ID: Having both constructors is acceptable. Ray Lischner, Senior Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.5170 rlischner@proteuseng.com f. 443.539.3370 This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message. ________________________________________ From: Boris Kolpackov [boris@codesynthesis.com] Sent: Tuesday, April 13, 2010 12:07 PM To: Ray Lischner Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] RE: xsd bypasses customized types Hi Ray, Ray Lischner writes: > After further investigation, I think I better understand what is > happening. Suppose type A is a restriction of unsignedInt, and > type B is a restriction of A. When xsd generates constructors for > B, it suppresses the B::B(A&) constructor, and instead issues only > B::B(unsigned int), that is, because A has no required elements or > attributes, B's immediate-base-class constructor is suppressed in > favor of B's ultimate-base-class constructor. Yes, that's exactly what happens. And the reason for this is that in such cases B::B(A&) does not add any new functionality since A can be converted without loss of information to its base. > This works fine until we customize type A. Somehow, we need to > tell xsd to favor the immediate-base-class constructor and > instead suppress the ultimate-base-class constructor. We can add some extra intelligence to this logic and avoid suppressing the B(A&) c-tor if the hierarchy includes a customized type. I don't think we can safely suppress the other c-tor thought. Would having both c-tors work for you? From rlischner at proteuseng.com Tue Apr 13 13:00:02 2010 From: rlischner at proteuseng.com (Ray Lischner) Date: Tue Apr 13 14:27:37 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: , Message-ID: One difficulty is that we have many schemas. Type A may be customized in schema B, and type C is customized in schema D, so when compiling schema E, which contains types that extend A and C, one must know about all the customizations and pass appropriate command line arguments to xsd. This puts quite a burden on the person writing schema E. Ray Lischner, Senior Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.5170 rlischner@proteuseng.com f. 443.539.3370 This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message. ________________________________________ From: Boris Kolpackov [boris@codesynthesis.com] Sent: Tuesday, April 13, 2010 12:07 PM To: Ray Lischner Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] RE: xsd bypasses customized types Hi Ray, Ray Lischner writes: > After further investigation, I think I better understand what is > happening. Suppose type A is a restriction of unsignedInt, and > type B is a restriction of A. When xsd generates constructors for > B, it suppresses the B::B(A&) constructor, and instead issues only > B::B(unsigned int), that is, because A has no required elements or > attributes, B's immediate-base-class constructor is suppressed in > favor of B's ultimate-base-class constructor. Yes, that's exactly what happens. And the reason for this is that in such cases B::B(A&) does not add any new functionality since A can be converted without loss of information to its base. > This works fine until we customize type A. Somehow, we need to > tell xsd to favor the immediate-base-class constructor and > instead suppress the ultimate-base-class constructor. We can add some extra intelligence to this logic and avoid suppressing the B(A&) c-tor if the hierarchy includes a customized type. I don't think we can safely suppress the other c-tor thought. Would having both c-tors work for you? From boris at codesynthesis.com Wed Apr 14 09:06:37 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Apr 14 08:56:58 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Hi Ray, Ray Lischner writes: > One difficulty is that we have many schemas. Type A may be customized > in schema B, and type C is customized in schema D, so when compiling > schema E, which contains types that extend A and C, one must know > about all the customizations and pass appropriate command line > arguments to xsd. This puts quite a burden on the person writing > schema E. Yes, that's unfortunately the case. The original idea was to have truly separate compilation of schemas but the reality proved we are fighting a loosing battle. For example, in XSD 3.3.0 there is a new option that allows one to specify which types are polymorphic to reduce the generated code size. If the base is in schema B and derived is in schema D, then one has to specify that the base is polymorphic while compiling schema D. There is no other way to figure this out. The recommended way to handle this is to create an options file which contains the --custom-type options for a specific set of schemas (i.e., a set of schemas corresponding to an XML vocabulary). Then when compiling other schemas that use this set of schemas one can add the options file to the XSD command line. In this particular situation I don't see any good alternatives to using the custom type information to decide not to suppress the c-tor. Always generating the c-tor is not an option since in 99.9% of cases it will result in unnecessary code bloat. Having a special option to selectively generate the c-tor for certain types seems to require an even greater amount of work from your side. Let me know if the custom type approach will be acceptable or if you have any other ideas. Boris From rlischner at proteuseng.com Wed Apr 14 09:00:49 2010 From: rlischner at proteuseng.com (Ray Lischner) Date: Wed Apr 14 10:23:16 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: , Message-ID: I think the custom type approach will work. We can create a single, shared options file that lists all of our custom types. Ray Lischner, Senior Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.5170 rlischner@proteuseng.com f. 443.539.3370 This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message. ________________________________________ From: Boris Kolpackov [boris@codesynthesis.com] Sent: Wednesday, April 14, 2010 9:06 AM To: Ray Lischner Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] RE: xsd bypasses customized types Hi Ray, Ray Lischner writes: > One difficulty is that we have many schemas. Type A may be customized > in schema B, and type C is customized in schema D, so when compiling > schema E, which contains types that extend A and C, one must know > about all the customizations and pass appropriate command line > arguments to xsd. This puts quite a burden on the person writing > schema E. Yes, that's unfortunately the case. The original idea was to have truly separate compilation of schemas but the reality proved we are fighting a loosing battle. For example, in XSD 3.3.0 there is a new option that allows one to specify which types are polymorphic to reduce the generated code size. If the base is in schema B and derived is in schema D, then one has to specify that the base is polymorphic while compiling schema D. There is no other way to figure this out. The recommended way to handle this is to create an options file which contains the --custom-type options for a specific set of schemas (i.e., a set of schemas corresponding to an XML vocabulary). Then when compiling other schemas that use this set of schemas one can add the options file to the XSD command line. In this particular situation I don't see any good alternatives to using the custom type information to decide not to suppress the c-tor. Always generating the c-tor is not an option since in 99.9% of cases it will result in unnecessary code bloat. Having a special option to selectively generate the c-tor for certain types seems to require an even greater amount of work from your side. Let me know if the custom type approach will be acceptable or if you have any other ideas. Boris From boris at codesynthesis.com Wed Apr 14 11:27:43 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Apr 14 11:18:02 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Hi Ray, Ray Lischner writes: > I think the custom type approach will work. We can create a single, > shared options file that lists all of our custom types. Here is the patch: http://scm.codesynthesis.com/?p=xsd/xsd.git;a=commitdiff;h=ad02b4ec2d8b0044e03ae5f39e75564118d7ce70 Let me know if there are any problems. Boris From xsd-users at groenndemon.de Wed Apr 14 19:12:36 2010 From: xsd-users at groenndemon.de (Marvin M.) Date: Wed Apr 14 19:12:39 2010 Subject: [xsd-users] Sequence + Choice (or Alternative) Message-ID: <4BC64BE4.60905@groenndemon.de> I would like to have a schema like this: Foo Bar Buzz Bar Foo ist the root node, and an infinite number of rules are allowed. It looks like this works quite well. However, I am unsure how to implement the rule's child nodes: There has to be at least one child node (valid elements are , and ), but there is no restriction on how many there could be. The order of the child nodes has to be preserved. I'm not that experienced with XML schemas yet, and I currently have a sequence of choices that would allow one of the three element types. But as the element types currently don't have a common base type, I think I can't use "--generate-polymorphic", can I? Anyway, I'd like to have something like an Item() method for my Rule class, so that I would be able to iterate through all children and check what element type they are. Thanks, Marvin From rlischner at proteuseng.com Wed Apr 14 17:39:46 2010 From: rlischner at proteuseng.com (Ray Lischner) Date: Thu Apr 15 08:41:02 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: , Message-ID: Thank you. The patch works fine. Ray Lischner, Senior Member of Technical Staff 133 National Business Pkwy, Ste 150 t. 443.539.3448 Annapolis Junction, MD 20701 c. 410.854.5170 rlischner@proteuseng.com f. 443.539.3370 This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message. ________________________________________ From: Boris Kolpackov [boris@codesynthesis.com] Sent: Wednesday, April 14, 2010 11:27 AM To: Ray Lischner Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] RE: xsd bypasses customized types Hi Ray, Ray Lischner writes: > I think the custom type approach will work. We can create a single, > shared options file that lists all of our custom types. Here is the patch: http://scm.codesynthesis.com/?p=xsd/xsd.git;a=commitdiff;h=ad02b4ec2d8b0044e03ae5f39e75564118d7ce70 Let me know if there are any problems. Boris From boris at codesynthesis.com Thu Apr 15 09:27:36 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 15 09:17:45 2010 Subject: [xsd-users] Sequence + Choice (or Alternative) In-Reply-To: <4BC64BE4.60905@groenndemon.de> References: <4BC64BE4.60905@groenndemon.de> Message-ID: Hi Marvin, Marvin M. writes: > > > Foo > Bar > Buzz > Bar > Foo > > > > [...] > > However, I am unsure how to implement the rule's child nodes: > There has to be at least one child node (valid elements are > , and ), but there is no restriction on > how many there could be. > The order of the child nodes has to be preserved. The following post explores this issue as well as the various solutions in great detail: http://www.codesynthesis.com/pipermail/xsd-users/2010-March/002741.html > I'm not that experienced with XML schemas yet, and I currently have a > sequence of choices that would allow one of the three element types. > But as the element types currently don't have a common base type, I > think I can't use "--generate-polymorphic", can I? If you want to use the substitutions group approach, then you will need to create an abstract element and derive , , and from it. Then, instead of using a sequence of choices, you will simply use the sequence of elements (which, in the actual XML document will be substituted with one of the concrete elements). The post mentioned above has an example schema fragment that shows how to do this. > Anyway, I'd like to have something like an Item() method for my Rule > class, so that I would be able to iterate through all children and check > what element type they are. Yes, you will be able to do this with the substitution group approach. Boris From xsd-users at groenndemon.de Thu Apr 15 09:22:54 2010 From: xsd-users at groenndemon.de (Marvin M.) Date: Thu Apr 15 09:22:54 2010 Subject: [xsd-users] Sequence + Choice (or Alternative) In-Reply-To: References: <4BC64BE4.60905@groenndemon.de> Message-ID: <4BC7132E.1090405@groenndemon.de> Hi Boris, Boris Kolpackov schrieb: > The following post explores this issue as well as the various solutions > in great detail: > > http://www.codesynthesis.com/pipermail/xsd-users/2010-March/002741.html I found that post through Google today. > If you want to use the substitutions group approach, then you will need > to create an abstract element and derive , > , and from it. Then, instead of using a sequence of > choices, you will simply use the sequence of elements (which, > in the actual XML document will be substituted with one of the concrete > elements). The post mentioned above has an example schema fragment that > shows how to do this. I tried that, but then I had several global elements (the abstract element and its substitution derivations) which were all legal to use as the document root (xsd.exe complained that it didn't know which one was the root). Also, the schema was much more complex, larger in size and harder to read. Unless you have a solution for those problems, I think I'll stick with the old choice and use the XSD/e approach instead, even though the syntax is less neat. Thanks, Marvin From boris at codesynthesis.com Thu Apr 15 10:19:23 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 15 10:09:32 2010 Subject: [xsd-users] Sequence + Choice (or Alternative) In-Reply-To: <4BC7132E.1090405@groenndemon.de> References: <4BC64BE4.60905@groenndemon.de> <4BC7132E.1090405@groenndemon.de> Message-ID: Hi Marvin, Marvin M. writes: > I tried that, but then I had several global elements (the abstract > element and its substitution derivations) which were all legal to use as > the document root (xsd.exe complained that it didn't know which one was > the root). XSD warned you that it is generating parsing functions for all global elements. That's the default behavior. You can specify which global elements are actual document roots with the --root-element option: --root-element grammar > Also, the schema was much more complex, larger in size and harder > to read. Yes, the schema is a bit more complex, though, IMO, this approach is conceptually cleaner. > Unless you have a solution for those problems, I think I'll stick with > the old choice and use the XSD/e approach instead, even though the > syntax is less neat. Yes, that's not a bad idea either. Boris From xsd-users at groenndemon.de Thu Apr 15 10:12:39 2010 From: xsd-users at groenndemon.de (Marvin M.) Date: Thu Apr 15 10:12:40 2010 Subject: [xsd-users] Sequence + Choice (or Alternative) In-Reply-To: References: <4BC64BE4.60905@groenndemon.de> <4BC7132E.1090405@groenndemon.de> Message-ID: <4BC71ED7.20702@groenndemon.de> Hi Boris, Boris Kolpackov schrieb: > XSD warned you that it is generating parsing functions for all global > elements. That's the default behavior. You can specify which global > elements are actual document roots with the --root-element option: > > --root-element grammar Yes, but the problem is that now XML with other document roots than grammar are valid. I find that a bit ugly. >> Unless you have a solution for those problems, I think I'll stick with >> the old choice and use the XSD/e approach instead, even though the >> syntax is less neat. > > Yes, that's not a bad idea either. Do I have to enable XSDE_REUSE_STYLE in config.nmake? Also, what about XSDE_REUSE_STYLE? Are there any other options I should enable? What files do I need for compilations apart from xsde.lib? I would like to copy those to my own project so I won't have to redistribute the whole xsde package (it's for a university project). Thanks, Marvin From sutambe at dre.vanderbilt.edu Sat Apr 17 18:13:14 2010 From: sutambe at dre.vanderbilt.edu (Sumant Tambe) Date: Sat Apr 17 18:13:18 2010 Subject: [xsd-users] open-source project using codesynthesis xsd Message-ID: <4BCA327A.4080107@dre.vanderbilt.edu> Hi Boris, I was wondering if you could point me to some open-source projects that make heavy use of xsd. My goal is to evaluate my research on XML traversal on a real-world case-study. I need the source code because I need to measure and compare three things: code size, compilation time, and run-time performance. Thanks, Sumant. From boris at codesynthesis.com Mon Apr 19 09:58:57 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Apr 19 09:49:03 2010 Subject: [xsd-users] open-source project using codesynthesis xsd In-Reply-To: <4BCA327A.4080107@dre.vanderbilt.edu> References: <4BCA327A.4080107@dre.vanderbilt.edu> Message-ID: Hi Sumant, Sumant Tambe writes: > I was wondering if you could point me to some open-source projects that > make heavy use of xsd. Now and again I come across open source projects that use XSD but I can't remember any off the top of my head. I remember there is/was a KDE tool for something, though I don't think its use of XSD would qualify as "heavy". Maybe someone on the mailing list can help you... > My goal is to evaluate my research on XML traversal on a real-world > case-study. I need the source code because I need to measure and > compare three things: code size, compilation time, and run-time > performance. You could always take a complex schema and associated XML documents and create a test application. For example, CityGML is fairly complex and has real-world XML documents available that describes real cities and run into the hundreds of megabytes. Boris From rcresswell at gmail.com Mon Apr 19 15:06:58 2010 From: rcresswell at gmail.com (Richard Cresswell) Date: Mon Apr 19 15:07:27 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Boris, I am having trouble using the patch you provided. If I use the --custom-type option for derived objects as you suggested below, xsd throws a FrontendElements::Context::NoEntry exception (originating from a call to FrontendElements::Context::get at xsd/cxx/tree/name-processor.cxx:1018). A simplified example: I run xsd with the flag --custom_type BaseObject and receive the exception. Is it possible to forward declare a custom type that is referenced but not defined in the schema that is being compiled? If I do not use --custom-type the code in your patch has no way to know that BaseObject is custom, causing it to build with the incorrect set of constructors. Thank you, Richard Cresswell On Wed, Apr 14, 2010 at 9:06 AM, Boris Kolpackov wrote: > Hi Ray, > > Ray Lischner writes: > >> One difficulty is that we have many schemas. Type A may be customized >> in schema B, and type C is customized in schema D, so when compiling >> schema E, which contains types that extend A and C, one must know >> about all the customizations and pass appropriate command line >> arguments to xsd. This puts quite a burden on the person writing >> schema E. > > Yes, that's unfortunately the case. The original idea was to have > truly separate compilation of schemas but the reality proved we > are fighting a loosing battle. For example, in XSD 3.3.0 there is a > new option that allows one to specify which types are polymorphic > to reduce the generated code size. If the base is in schema B and > derived is in schema D, then one has to specify that the base is > polymorphic while compiling schema D. There is no other way to > figure this out. > > The recommended way to handle this is to create an options file > which contains the --custom-type options for a specific set of > schemas (i.e., a set of schemas corresponding to an XML vocabulary). > Then when compiling other schemas that use this set of schemas > one can add the options file to the XSD command line. > > In this particular situation I don't see any good alternatives to > using the custom type information to decide not to suppress the > c-tor. Always generating the c-tor is not an option since in > 99.9% of cases it will result in unnecessary code bloat. Having > a special option to selectively generate the c-tor for certain > types seems to require an even greater amount of work from your > side. > > Let me know if the custom type approach will be acceptable or > if you have any other ideas. > > Boris > > From boris at codesynthesis.com Mon Apr 19 15:35:57 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Apr 19 15:26:08 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Hi Richard, Richard Cresswell writes: > Boris, I am having trouble using the patch you provided. If I use the > --custom-type option for derived objects as you suggested below, xsd > throws a FrontendElements::Context::NoEntry exception (originating > from a call to FrontendElements::Context::get at > xsd/cxx/tree/name-processor.cxx:1018). I also discovered (and fixed) this bug while working on the from-base fix. Here is the patch: http://scm.codesynthesis.com/?p=xsd/xsd.git;a=commit;h=c7a135cc66ab0270cebc04664dad7baa2e4c3818 For this bug you only need the changes to the name-processor.cxx. Boris From rcresswell at gmail.com Tue Apr 20 08:02:56 2010 From: rcresswell at gmail.com (Richard Cresswell) Date: Tue Apr 20 08:03:24 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Boris, that fixed the xsd tree generation, but the generated code still does not compile. It creates a default constructor of form: DerivedObject:: DerivedObject () : ::base::BaseObject () { } Shouldn't it detect that DerivedObject is derived from the custom type and create the correct constructors? Applying the patch also broke tests/cxx/tree/name-clash/inheritance. Could this be because of other missed patches? I noticed that name-processor.cxx had changed quite a bit between 3.2 and the patch that you provided. Your cc.set ("cxx-tree-name-processor-stem-set", NameSet ()); cc.set ("cxx-tree-name-processor-member-set", NameSet ()); was still c.context ().set ("cxx-tree-name-processor-stem-set", stem_set); c.context ().set ("cxx-tree-name-processor-member-set", member_set); in my code. I am concerned that my codebase diverges too much from where you are with 3.3, but I am reluctant to upgrade to the 3.3 beta since I will need to patch again when you do the final release. Do you have an ETA for when the official 3.3 will go out? Thanks again, I appreciate your effort and quick responses. Richard On Mon, Apr 19, 2010 at 3:35 PM, Boris Kolpackov wrote: > Hi Richard, > > Richard Cresswell writes: > >> Boris, I am having trouble using the patch you provided. ?If I use the >> --custom-type option for derived objects as you suggested below, xsd >> throws a FrontendElements::Context::NoEntry exception (originating >> from a call to FrontendElements::Context::get at >> xsd/cxx/tree/name-processor.cxx:1018). > > I also discovered (and fixed) this bug while working on the from-base > fix. Here is the patch: > > http://scm.codesynthesis.com/?p=xsd/xsd.git;a=commit;h=c7a135cc66ab0270cebc04664dad7baa2e4c3818 > > For this bug you only need the changes to the name-processor.cxx. > > Boris > From boris at codesynthesis.com Tue Apr 20 10:52:49 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Apr 20 10:42:57 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Hi Richard, Richard Cresswell writes: > Boris, that fixed the xsd tree generation, but the generated code > still does not compile. It creates a default constructor of form: > > DerivedObject:: > DerivedObject () > : ::base::BaseObject () > { > } > > Shouldn't it detect that DerivedObject is derived from the custom type > and create the correct constructors? I assume by "correct constructors" you mean that your custom implementation of BaseObject class does not define a default constructor. Unfortunately, the only way to detect this would be to parse (as in C++) your BaseObject definition and figure out which constructors it defines. The way XSD handles this is that it expects the custom implementation to have the same interface as the generated version would have had. I don't see any other practical way, unfortunately. > Applying the patch also broke tests/cxx/tree/name-clash/inheritance. > Could this be because of other missed patches? I am not sure what's the reason for this. In the current code base all tests pass. > cc.set ("cxx-tree-name-processor-stem-set", NameSet ()); > cc.set ("cxx-tree-name-processor-member-set", NameSet ()); > > was still > > c.context ().set ("cxx-tree-name-processor-stem-set", stem_set); > c.context ().set ("cxx-tree-name-processor-member-set", member_set); The two code fragments are semantically equivalent in this case. > I am concerned that my codebase diverges too much from where you are > with 3.3, but I am reluctant to upgrade to the 3.3 beta since I will > need to patch again when you do the final release. Do you have an ETA > for when the official 3.3 will go out? It should be out later this week or early next week. Boris From rcresswell at gmail.com Tue Apr 20 13:53:56 2010 From: rcresswell at gmail.com (Richard Cresswell) Date: Tue Apr 20 13:54:25 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Boris, Our base customized object is derived from a string, but is implemented as another type in our customized code. The problem is that once we derive an object from the custom base, it creates that derived object with the string constructors. Instead, we want it to see that it is derived from a custom object and create a from-base constructor: DerivedObject::DerivedObject(BaseObject) This way all of the custom BaseObject constructors can be used for the DerivedObject. Does that make more sense? Thank you, Richard On Tue, Apr 20, 2010 at 10:52 AM, Boris Kolpackov wrote: > Hi Richard, > > Richard Cresswell writes: > >> Boris, that fixed the xsd tree generation, but the generated code >> still does not compile. ?It creates a default constructor of form: >> >> DerivedObject:: >> DerivedObject () >> : ::base::BaseObject () >> { >> } >> >> Shouldn't it detect that DerivedObject is derived from the custom type >> and create the correct constructors? > > I assume by "correct constructors" you mean that your custom > implementation of BaseObject class does not define a default > constructor. Unfortunately, the only way to detect this would > be to parse (as in C++) your BaseObject definition and figure > out which constructors it defines. > > The way XSD handles this is that it expects the custom > implementation to have the same interface as the generated > version would have had. I don't see any other practical > way, unfortunately. > > >> Applying the patch also broke tests/cxx/tree/name-clash/inheritance. >> Could this be because of other missed patches? > > I am not sure what's the reason for this. In the current code > base all tests pass. > > >> cc.set ("cxx-tree-name-processor-stem-set", NameSet ()); >> cc.set ("cxx-tree-name-processor-member-set", NameSet ()); >> >> was still >> >> c.context ().set ("cxx-tree-name-processor-stem-set", stem_set); >> c.context ().set ("cxx-tree-name-processor-member-set", member_set); > > The two code fragments are semantically equivalent in this case. > > >> I am concerned that my codebase diverges too much from where you are >> with 3.3, but I am reluctant to upgrade to the 3.3 beta since I will >> need to patch again when you do the final release. ?Do you have an ETA >> for when the official 3.3 will go out? > > It should be out later this week or early next week. > > Boris > From jrs at jmsutton.co.uk Tue Apr 20 13:57:56 2010 From: jrs at jmsutton.co.uk (James Sutton) Date: Tue Apr 20 14:52:39 2010 Subject: [xsd-users] why no expat support for cxx-tree? Message-ID: <15BB809D-C3DB-40E1-B6F4-89BDF5F3219B@jmsutton.co.uk> Hi I am attempting to use xsd to parse MusicXML (http://www.recordare.com/xml.html). Xerces isn't available on the target platform, so I cannot use cxx-tree. Is there any hidden option to get cxx-tree to support expat? This is a big schema and it's going to take a lot of work to reproduce an in-memory image from cxx-parser which would be available immediately from cxx-tree. Thanks James Sutton From boris at codesynthesis.com Tue Apr 20 15:31:56 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Apr 20 15:22:07 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Hi Richard, Richard Cresswell writes: > Our base customized object is derived from a string, but is > implemented as another type in our customized code. The problem is > that once we derive an object from the custom base, it creates that > derived object with the string constructors. Instead, we want it to > see that it is derived from a custom object and create a from-base > constructor: > > DerivedObject::DerivedObject(BaseObject) > > This way all of the custom BaseObject constructors can be used for the > DerivedObject. > > Does that make more sense? I understand what you are trying to achieve. However, there are other possible (and much more common) uses of type customization (for example, adding a data to the generated class without changing the type's interface) that will break should we make the XSD compiler behave the way you describe. One way to get rid of the default c-tor is to derive your BaseObject from a type that doesn't allow default-construction, for example xsd:NMTOKEN, instead of xsd:string. You will still get the string c-tors (from const char* and const std::string&) though. Boris From boris at codesynthesis.com Tue Apr 20 15:43:34 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Apr 20 15:33:46 2010 Subject: [xsd-users] why no expat support for cxx-tree? In-Reply-To: <15BB809D-C3DB-40E1-B6F4-89BDF5F3219B@jmsutton.co.uk> References: <15BB809D-C3DB-40E1-B6F4-89BDF5F3219B@jmsutton.co.uk> Message-ID: Hi James, James Sutton writes: > I am attempting to use xsd to parse MusicXML. Xerces isn't available > on the target platform, so I cannot use cxx-tree. Is there any hidden > option to get cxx-tree to support expat? No, C++/Tree only works with Xerces-C++. There are multiple reasons for that, the primary ones being the use of the XML Schema validation and support for some advanced featured that require the underlying DOM representation. > This is a big schema and it's going to take a lot of work to reproduce > an in-memory image from cxx-parser which would be available immediately > from cxx-tree. There is another problem with MusicXML and C++/Tree that got to do with element ordering. See the thread starting from here for details: http://www.codesynthesis.com/pipermail/xsd-users/2008-May/001696.html There is however another option that addressed both of these problems (i.e., the lack of Xerces-C++ support and element ordering) which is to use the C++/Hybrid mapping in XSD/e (or mobile/embedded systems variant of XSD): http://www.codesynthesis.com/products/xsde/ C++/Hybrid is similar to C++/Tree in that it delivers the data as an in-memory object model. However, it is more lighter-weight and doesn't have any external dependencies. Boris From rcresswell at gmail.com Tue Apr 20 17:14:48 2010 From: rcresswell at gmail.com (Richard Cresswell) Date: Tue Apr 20 17:15:15 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Boris, Don't we still have the same problem that we saw at the beginning of this thread though? The from-base class constructor is being suppressed because it is trying to use the fundamental value constructors. I feel that this is inappropriate when we are deriving from a custom type because it prevents us from extending it using our own constructors. We don't just need to suppress the default constructor, we need to construct this object in the same way that the base custom type is constructed by using the from-base constructor. I think I might be having trouble envisioning the cases that you say this would break. Isn't the suppression of the from-base constructor and its replacement by the fundamental constructors just to counter code bloat? Can you give an example where the functionality I describe would break an object that is derived from a custom type? Thanks Boris, Richard On Tue, Apr 20, 2010 at 3:31 PM, Boris Kolpackov wrote: > Hi Richard, > > Richard Cresswell writes: > >> Our base customized object is derived from a string, but is >> implemented as another type in our customized code. ?The problem is >> that once we derive an object from the custom base, it creates that >> derived object with the string constructors. ?Instead, we want it to >> see that it is derived from a custom object and create a from-base >> constructor: >> >> DerivedObject::DerivedObject(BaseObject) >> >> This way all of the custom BaseObject constructors can be used for the >> DerivedObject. >> >> Does that make more sense? > > I understand what you are trying to achieve. However, there are > other possible (and much more common) uses of type customization > (for example, adding a data to the generated class without changing > the type's interface) that will break should we make the XSD compiler > behave the way you describe. > > One way to get rid of the default c-tor is to derive your BaseObject > from a type that doesn't allow default-construction, for example > xsd:NMTOKEN, instead of xsd:string. You will still get the string > c-tors (from const char* and const std::string&) though. > > Boris > From greg at permedia.ca Wed Apr 21 00:55:49 2010 From: greg at permedia.ca (Greg Carter) Date: Wed Apr 21 00:56:00 2010 Subject: [xsd-users] xml_schema::exception, linux, shared libraries, and visibility=hidden Message-ID: Hi, XSD version - xsd-3.3.0.b2-i686-linux-gnu problem on CentOS 5.4 x86_64, CentOS 5.4 x86 work around - don't compile with visibility=hidden I'm having a problem catching exceptions of type xml_schema::exception thrown from a shared library on linux. My shared libs look like: libxmlbase.so - XSD generated code libmyxml.so - XSD generated code libmylib.so uses above libs, compiled with visibility=hidden myapp Code inside libmyxml.so throws: throw ::xsd::cxx::tree::unexpected_element < char > ( n.name (), n.namespace_ (), "tcfFile", "http://www.geocosm.net/200608/TcfSchema"); Stepping into the above constructor I end up in libxsd/xsd/cxx/tree/exceptions.txx line 106, executing in libxmlbase.so. My catch code in libmylib.so looks like: catch (const xml_schema::exception& e) { cout << "XML exception reading file: " << tcfFile << endl; cout << e << endl; } The problem is that on linux when libmylib is compiled with visibility=hidden I never get into the catch, I do if I compile libmylib with visibility=default. With visibility=hidden I fall through to catch (std::exception& ex) { cout << ex.what() << endl; } which is in myapp. Code is compiled with xerces 3.1.0, (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46) I'm thinking that export defines used for the XSD generated code (XSD option --export-symbol) may need to be extended to the exception types so this works correctly on linux (example #define MYXSDEXPORT __attribute__ ((__visibility__("default"))) ). For additional information see http://gcc.gnu.org/wiki/Visibility , particularly the section on exceptions. For now I can work around the problem and not compile mylib with visibility=hidden, but eventually it would be great if I could. Greg. From boris at codesynthesis.com Wed Apr 21 08:54:57 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Apr 21 08:45:22 2010 Subject: [xsd-users] xml_schema::exception, linux, shared libraries, and visibility=hidden In-Reply-To: References: Message-ID: Hi Greg, Greg Carter writes: > I'm having a problem catching exceptions of type xml_schema::exception > thrown from a shared library on linux. > > My shared libs look like: > libxmlbase.so - XSD generated code > libmyxml.so - XSD generated code > libmylib.so uses above libs, compiled with visibility=hidden > myapp > > Code inside libmyxml.so throws: > > throw ::xsd::cxx::tree::unexpected_element < char > ( > n.name (), > n.namespace_ (), > "tcfFile", > "http://www.geocosm.net/200608/TcfSchema"); > > Stepping into the above constructor I end up in > libxsd/xsd/cxx/tree/exceptions.txx line 106, executing in libxmlbase.so. > > My catch code in libmylib.so looks like: > > catch (const xml_schema::exception& e) > { > cout << "XML exception reading file: " << tcfFile << endl; > cout << e << endl; > } > > The problem is that on linux when libmylib is compiled with > visibility=hidden I never get into the catch, I do if I compile libmylib > with visibility=default. With visibility=hidden I fall through to > catch (std::exception& ex) > { > cout << ex.what() << endl; > } > which is in myapp. > > I'm thinking that export defines used for the XSD generated code (XSD > option --export-symbol) may need to be extended to the exception types > so this works correctly on linux (example #define MYXSDEXPORT > __attribute__ ((__visibility__("default"))) ). This is already done when you generate the XML Schema namespace into a separate file (--generate-xml-schema option) and specify the --export-xml-schema and --export-symbol options, as in: xsd cxx-tree --generate-xml-schema --export-xml-schema --export-symbol \ SCHEMA_EXPORT xml-schema.xsd The xml-schema.hxx file then contains: template class SCHEMA_EXPORT exception< char >; So I believe your setup should work provided that: 1. the XML Schema namespace was compiled as above and 2. all other schemas are compiled with --extern-xml-schema to use xml-schema.hxx and 3. SCHEMA_EXPORT is set to visibility=default when building all the libraries. Can you check that the above conditions are met in your setup? If that's the case, then it would be helpful if you could create a small test case that reproduces this problem. As a side node, we are about to start final testing for XSD 3.3.0 so if there is indeed a problem here and you would like to have it resolved in 3.3.0, we need to move fast with this. Boris From boris at codesynthesis.com Wed Apr 21 09:04:42 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Apr 21 08:54:55 2010 Subject: [xsd-users] RE: xsd bypasses customized types In-Reply-To: References: Message-ID: Hi Richard, Richard Cresswell writes: > Don't we still have the same problem that we saw at the beginning of > this thread though? The from-base class constructor is being > suppressed because it is trying to use the fundamental value > constructors. With the original patch that I sent to Ray, the from-base c-tor is not suppressed if (1) you specify the --generate-from-base-ctor option and (2) there is a custom type in the inheritance hierarchy. If I have a schema fragment like this: Which I then compile with '--custom-type base' and --generate-from-base-ctor, I get the following c-tors in the generated derived class: derived (const char*); derived (const std::string&); derived (const test::base&); > I think I might be having trouble envisioning the cases that you say > this would break. Isn't the suppression of the from-base constructor > and its replacement by the fundamental constructors just to counter > code bloat? Yes, that's correct. > Can you give an example where the functionality I describe would break > an object that is derived from a custom type? When I said that the functionality you proposed would break other code, I was referring to the suppression of the default c-tor, not the suppression of the from-base c-tor. Boris From greg at permedia.ca Wed Apr 21 14:49:15 2010 From: greg at permedia.ca (Greg Carter) Date: Wed Apr 21 14:49:20 2010 Subject: [xsd-users] xml_schema::exception, linux, shared libraries, and visibility=hidden In-Reply-To: References: Message-ID: Boris Kolpackov wrote: > Hi Greg, > This is already done when you generate the XML Schema namespace into > a separate file (--generate-xml-schema option) and specify the > --export-xml-schema and --export-symbol options, as in: > > xsd cxx-tree --generate-xml-schema --export-xml-schema --export-symbol \ > SCHEMA_EXPORT xml-schema.xsd > The xml-schema.hxx file then contains: > > template class SCHEMA_EXPORT exception< char >; > > So I believe your setup should work provided that: > > 1. the XML Schema namespace was compiled as above and > > Hi Boris, I think I have the above correct, here is how I compile my schema namespace to a single header file: $(XSD)\bin\xsd.exe cxx-tree --proprietary-license --generate-xml-schema --export-xml-schema --export-symbol GEOCOSMXMLBASE_SYMBOL_DECL --hxx-prologue "#include " --generate-intellisense --output-dir "$(InputDir)gencode\cpp" --generate-serialization --generate-ostream "$(InputDir)XSDBaseXmlSchema.xsd" My geocosmexportxmlbase.h has #define GEOCOSMXMLBASE_SYMBOL_DECL __attribute__ ((__visibility__("default"))) and the generated code (XSDBaseXmlSchema.hxx) has template class GEOCOSMXMLBASE_SYMBOL_DECL exception< char >; > 2. all other schemas are compiled with --extern-xml-schema to use > xml-schema.hxx and > 3. SCHEMA_EXPORT is set to visibility=default when building all the > libraries. > > This is the xsdcompileoptionsbase.txt options file I use for the base xml: --proprietary-license --generate-intellisense --generate-serialization --generate-ostream --export-maps --generate-polymorphic --export-symbol GEOCOSMXMLBASE_SYMBOL_DECL --hxx-prologue #include --extern-xml-schema XSDBaseXmlSchema.xsd This is a typical command line for the schemas: $(XSD)\bin\xsd.exe cxx-tree --options-file "$(ProjectDir)xsdcompileoptionsbase.txt" --output-dir "$(InputDir)gencode\cpp" "$(InputDir)BurialHistorySchema.xsd" xsdcompileoptions.txt options file for my dependent xml lib --proprietary-license --generate-intellisense --generate-serialization --generate-ostream --generate-polymorphic --import-maps --export-symbol GEOCOSMXML_SYMBOL_DECL --hxx-prologue #include --extern-xml-schema XSDBaseXmlSchema.xsd exportxml.h has: #define GEOCOSMXML_SYMBOL_DECL __attribute__ ((__visibility__("default"))) Here is a typical command line for the dependent schemas $(XSD)\bin\xsd.exe cxx-tree --options-file "$(InputDir)xsdcompileoptions.txt" --output-dir "$(InputDir)gencode\cpp" "$(InputDir)DistributionSchema.xsd" I'm using XSD on Windows to generate the code, compile is done on linux (CentOS 5.4) For the two xml libs, I compile with visibility=default, I did try hidden, and the program ran and found all symbols but I still had the exception issue. I'll try to see if I can replicate with a simple test app... > Can you check that the above conditions are met in your setup? If that's > the case, then it would be helpful if you could create a small test case > that reproduces this problem. > > As a side node, we are about to start final testing for XSD 3.3.0 so if > there is indeed a problem here and you would like to have it resolved in > 3.3.0, we need to move fast with this. > > Boris > > From boris at codesynthesis.com Wed Apr 21 17:55:04 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Apr 21 17:45:24 2010 Subject: [xsd-users] xml_schema::exception, linux, shared libraries, and visibility=hidden In-Reply-To: References: Message-ID: Hi Greg, Greg Carter writes: > For the two xml libs, I compile with visibility=default, I did try > hidden, and the program ran and found all symbols but I still had the > exception issue. I'll try to see if I can replicate with a simple test > app... I believe I have found and fixed the problem. It has to do with a tricky interaction between GCC attributes and specializations. Essentially, GCC will ignore any attempt to set the visibility attribute for an explicit template instantiation if a specialization of a member of this template has been seen. The funny part is that one can't provide the specialization after the explicit instantiation either. In any case, here is a binary with the fix: http://www.codesynthesis.com/~boris/tmp/xsd-3.3.0.b3-i686-windows.zip I have tested and I can see with nm that the symbols for the exception classes are public. Would be great if you could give it a try ASAP and let me know if it works for you. Boris From eric at boostpro.com Thu Apr 22 14:28:40 2010 From: eric at boostpro.com (Eric Niebler) Date: Thu Apr 22 14:28:45 2010 Subject: [xsd-users] XML 1.1 and restricted characters Message-ID: <4BD09558.20908@boostpro.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi XSD gurus, I have the following problem: sometimes characters that are invalid according to the XML 1.0 spec (e.g. 0x1F) end up in my XML files. That causes the resulting XML file to be unreadable. After some research and experimentation, I've found that in XML 1.1, these "restricted characters" are allowed as long as they are escaped (e.g. ). What I need is (A) a way to configure XSD serialization so that it emits the XML declaration as: and (B) to translate restricted characters into their escaped equivalents. Is there an easy way to do this? I can probably accomplish (A) by writing out the declaration myself and using the xml_schema::flags::no_xml_declaration flag. And I could do (B) myself using an ostream filter that knows about restricted characters and translates them (taking care somehow not to translate 0x1f if it appears as a trailing byte of UTF-8-encoded character). Ugh! That seems like a lot of work for something that IMO should be simple. Thoughts? - -- Eric Niebler BoostPro Computing http://www.boostpro.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJL0JVYAAoJEAeJsEDfjLbXTrkH/jz+EaI00sn2p0k9RXfeV4Ub QZjYetjqrvVSg3qmB9z4TlWOjEn42vlxR5lV3TCTgfZ9/cgsTotK1HOPyivMi7ES VT66dc0JXs0e4GDtWZJvY4OelltF68KaH1ggi8wp7pp5fa8iCCq5mBmoerEchKip RfIpRw+lu3inkHw/iKI+XbOj12xtt80qqpkpt3/HX/4SSeBR/XyMOa07QGv30qVq jvyC8VzqAEr1wwB1tB4AhLtzzszd2mvfcLMSCzgCMsM3Jl+Ul2drdr0m7HutKece UMiKwcyG2Z3pPcCmDtVfsNq4SOZDAhJC2Is1ANK7IcfdQG4ZhaaufEBLaEQv/o4= =a6yd -----END PGP SIGNATURE----- From boris at codesynthesis.com Thu Apr 22 15:23:02 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 22 15:13:25 2010 Subject: [xsd-users] XML 1.1 and restricted characters In-Reply-To: <4BD09558.20908@boostpro.com> References: <4BD09558.20908@boostpro.com> Message-ID: Hi Eric, Eric Niebler writes: > What I need is (A) a way to configure XSD serialization so that it emits > the XML declaration as: > > > > and (B) to translate restricted characters into their escaped > equivalents. Is there an easy way to do this? I believe setting the DOM document version should do the trick. In Xerces-C++ 2-series it will be: doc->setVersion (XMLUni::fgVersion1_1); In Xerces-C++ 3-series it will be: doc->setXmlVersion (XMLUni::fgVersion1_1); I think even setting it after populating the DOM document and just before writing it out will work. If not, then you can create an empty DOM document, set its version to 1.1, serialize the object model to this DOM document, and then write it out. Let us know how it goes. Boris From eric at boostpro.com Thu Apr 22 16:45:13 2010 From: eric at boostpro.com (Eric Niebler) Date: Thu Apr 22 16:45:18 2010 Subject: [xsd-users] XML 1.1 and restricted characters In-Reply-To: References: <4BD09558.20908@boostpro.com> Message-ID: <4BD0B559.9090703@boostpro.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 4/22/2010 12:23 PM, Boris Kolpackov wrote: > Hi Eric, > > Eric Niebler writes: > >> What I need is (A) a way to configure XSD serialization so that it emits >> the XML declaration as: >> >> >> >> and (B) to translate restricted characters into their escaped >> equivalents. Is there an easy way to do this? > > I believe setting the DOM document version should do the trick. > In Xerces-C++ 2-series it will be: > > doc->setVersion (XMLUni::fgVersion1_1); > > In Xerces-C++ 3-series it will be: > > doc->setXmlVersion (XMLUni::fgVersion1_1); > > I think even setting it after populating the DOM document and just > before writing it out will work. If not, then you can create an > empty DOM document, set its version to 1.1, serialize the object > model to this DOM document, and then write it out. > > Let us know how it goes. WOW! That was easy. Works like a charm. I can set the version even after populating the DOM. Feature request: pls. expose this xerces functionality as a flag that I can pass to the serialization functions that XSD generates. Thanks, Boris! - -- Eric Niebler BoostPro Computing http://www.boostpro.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJL0LVZAAoJEAeJsEDfjLbXeAUH/2wpV4yrqbBxUb275+J1k9h+ wzS1Gu5iurzJEAxsU35K2TJ9twQ2jdKD9MW5pjjba/mJEtqc88WcOK8MsJuOCH6w 1OhtIEx2fIQqSMylcR5XTfADoxlXFA5VwTPS/gnneBxN2KUDqKvLetSuEv2OM/RK QYZp+S4Xu4aeDFM9JxURXJ3Z/ZIkX+Tn++C3QIkOAS0lSCrqH+mSkgg6jUwIpflt srUbENWhuJmq2KoN9Of4B8plltdlhZXczLEaOffh/CGBC7u46o1adnrWWrhzAPZp 6l72T7/56CvmgZ9bHkd7XRonKod4ZBqWsyuAVbotwWL7DhjacALJaD0CDes8I58= =NO2/ -----END PGP SIGNATURE----- From eric at boostpro.com Mon Apr 26 14:47:03 2010 From: eric at boostpro.com (Eric Niebler) Date: Mon Apr 26 14:47:13 2010 Subject: [xsd-users] xerces exceptions Message-ID: <4BD5DFA7.7050709@boostpro.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've discovered (a bit too late sadly) that xsd lets xerces exceptions propagate. Ideally, I'd like to deal exclusively with std and xsd types and not let any xerces types leak into my code. Is there an option to have xsd catch and translate xerces exceptions? An obvious one would be to translate xercesc::OutOfMemoryException into std::bad_alloc, for instance. - -- Eric Niebler BoostPro Computing http://www.boostpro.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJL1d+nAAoJEAeJsEDfjLbXP60IAJO9ZCGAilc81bf7Du6FYSs7 Kr9ZYne9p8xj6sxXqaLFcuI4TWhFNMuOp7TvQ3pcv3a0g5U+/6uNoRgeLwzMsic9 l9oDq4PZno2OoSsiL8e6E9sAUGCZpKD/uS4kfM6smToEAcgyKuYs3JRoVFHLqd3S ZW99fCpKqj2bGFL+Z5VLigwh9WItFWaptJ/MeHCoEGnkKpSOO0xFWWf+W3BntmS2 TxqfVfguleXpsrnt5T9czEE2uUU9cnIwAoXJMdznYCi1ZK52Cc9x3cNUt/INDPvu 1Oq8wlfvsxBZSw9z1RUWdb0hhwpLqbRsXtlopaNb91uRGug1AeYhs7a8CIJ1RyU= =4qHO -----END PGP SIGNATURE----- From howes at ll.mit.edu Tue Apr 27 09:42:17 2010 From: howes at ll.mit.edu (Brad Howes) Date: Tue Apr 27 09:42:29 2010 Subject: [xsd-users] GML 3.2.1 Message-ID: <324B70EC-BB8A-4A28-84D8-D77C25C817AB@ll.mit.edu> All, After getting GML 3.1.1 up and running, the powers that be decided to move to GML 3.2.1. Right now I'm stuck trying to get anything to generate using cxx-tree. Here is an example of the errors I'm seeing while processing gmlBase.xsd using --file-per-type mode: ../../iso/19139/20070417/gmd/../gmd/../gco/../gco/../../../../gml/3.2.1/gmlBase.xsd:33:77: error: type 'http://www.opengis.net/gml/3.2:metaDataProperty' not found Here are the options I'm using: --namespace-map http://www.opengis.net/gml/3.2=gml --root-element-none --generate-polymorphic The element 'metaDataProperty' is defined in deprecatedTypes.xsd, and I can process that on its own just fine. I am suspicious of the jump into the ISO path files, thinking that perhaps there is something in there that is causing issues with XSD. Any help at all would be most welcome. Hitting the docs once again... Brad -- Brad Howes Group 42 MIT Lincoln Laboratory ? 244 Wood St. ? Lexington, MA 02173 Phone: 781.981.5292 ? Fax: 781.981.3495 ? Secretary: 781.981.7420 From howes at ll.mit.edu Tue Apr 27 13:09:42 2010 From: howes at ll.mit.edu (Brad Howes) Date: Tue Apr 27 13:09:58 2010 Subject: [xsd-users] GML 3.2.1 [Update] Message-ID: <77A6AA04-2884-4CFF-A1E6-52CC222951AE@ll.mit.edu> Bah! Never mind. I just found out that one must only go through the gml.xsd schema per the 3.2.1 spec, and not via any of the component files. Doing it the piece-meai way appeared to work, but may have not been sound... Brad All, After getting GML 3.1.1 up and running, the powers that be decided to move to GML 3.2.1. Right now I'm stuck trying to get anything to generate using cxx-tree. Here is an example of the errors I'm seeing while processing gmlBase.xsd using --file-per-type mode: ../../iso/19139/20070417/gmd/../gmd/../gco/../gco/../../../../gml/3.2.1/gmlBase.xsd:33:77: error: type 'http://www.opengis.net/gml/3.2:metaDataProperty' not found Here are the options I'm using: --namespace-map http://www.opengis.net/gml/3.2=gml --root-element-none --generate-polymorphic The element 'metaDataProperty' is defined in deprecatedTypes.xsd, and I can process that on its own just fine. I am suspicious of the jump into the ISO path files, thinking that perhaps there is something in there that is causing issues with XSD. Any help at all would be most welcome. Hitting the docs once again... Brad -- Brad Howes Group 42 MIT Lincoln Laboratory ? 244 Wood St. ? Lexington, MA 02173 Phone: 781.981.5292 ? Fax: 781.981.3495 ? Secretary: 781.981.7420 From boris at codesynthesis.com Tue Apr 27 16:08:35 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Apr 27 15:59:13 2010 Subject: [xsd-users] Xerces-C++ 3.1.1 released Message-ID: Hi, Xerces-C++ 3.1.1 was released today. This version contains a handful of bug fixes and is binary-compatible with 3.1.0. For the official announcement please see: http://marc.info/?l=xerces-c-dev&m=127238991323077&w=2 One notable new feature in this version is the addition of the Visual Studio 2010 (10.0) project and solution files for the library, examples, and tests. We have also successfully tested the library built with Visual Studio 2010 against various test suites, including the XML Schema Test Suite (XSTS). This release has been tested on all major platforms and comes with pre-compiled libraries (total 18) for various CPU architectures, operating systems, and C++ compilers. You can download them along with the source distributions from the Xerces-C++ download page: http://xerces.apache.org/xerces-c/download.cgi The upcoming XSD 3.3.0 has been extensively tested with this version of Xerces-C++. All pre-compiled binaries will be built using 3.1.1 and the Windows .msi package will contain pre- built Xerces-C++ libraries of this version. Enjoy, Boris From boris at codesynthesis.com Wed Apr 28 06:00:54 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Apr 28 05:51:35 2010 Subject: [xsd-users] XSD 3.3.0 released Message-ID: Hi, We have released XSD 3.3.0. The NEWS file entries for this release are as follows: * New option, --char-encoding, allows you to specify the character encoding that should be used in the generated code. Valid values for the 'char' character type are 'utf8' (default), 'iso8859-1' (new), 'lcp' (Xerces-C++ local code page), and 'custom' (provides support for custom encodings). Note that if you use a non-default character encoding and include some libxsd headers (e.g., xsd/cxx/xml/string.hxx) directly, then you will need to first include the correct xsd/cxx/xml/char-.hxx header, where is iso8859-1, lcp, etc. This mechanism replaces the XSD_USE_LCP macro. For the wchar_t character type the only valid value for this option is 'auto' and the encoding is automatically selected between UTF-16 and UTF-32, depending on the wchar_t type size. * When the XSD compiler is built with Xerces-C++ 3.1.0 or later, the handling of multiple imports for the same namespace is enabled. Before, all subsequent imports for a namespace were ignored which caused errors in some schemas. Note that if your application has XML Schema validation enabled, then you will also need to build it with Xerces-C++ 3.1.0 or later to take advantage of this feature. * Automatic mapping for the urn-style XML namespaces. The last component in the urn name is used to derive the C++ namespace name. * New option, --schema-file-regex, in combination with the existing --type-file-regex, can be used to place the generated files into subdirectories or to resolve file name conflicts in the file-per- type mode (--file-per-type). * Warning id's have changed to start with a letter identifying the component issuing the warning. F - compiler frontend, D - compiler driver, P - C++/Parser mapping, T - C++/Tree mapping. * Strings used to match regular expressions supplied with the --namespace-regex and --anonymous-regex options now include the file component for the schema being compiled. * The XSD_NO_EXPORT macro can be used to omit code generated with the --export/import-maps and, for C++/Tree, --generate-xml-schema options during C++ compilation. This may be useful if you would like to use the same generated code across multiple platforms. C++/Tree * New option, --generate-element-type, triggers the generation of types instead of parsing/serialization functions for root elements. This is primarily useful to distinguish object models with the same root type but with different root elements. For more information, refer to the messaging example and Section 2.9.1, "Element Types" in the C++/Tree Mapping User Manual. To support the customization of the element type naming the --element-type-regex option has been added. See the NAMING CONVENTION section in the compiler command line manual (man pages) for details. * New option, --generate-element-map, triggers the generation of a root element map. The element map allows uniform parsing and serialization of multiple root elements. This option can only be used together with --generate-element-type. For more information, refer to the messaging example and Section 2.9.2, "Element Map" in the C++/Tree Mapping User Manual. * Prior to this version, if the --generate-polymorphic option is specified, the compiler treats all types as potentially polymorphic. Now by default only type hierarchies used in substitution groups and those explicitly declared polymorphic with the new --polymorphic-type option are treated as polymorphic. This results in smaller and faster generated code. If you would like to continue using the old behavior, you will need to specify --polymorphic-type-all. For more information, on this change see Section 2.11, "Mapping for xsi:type and Substitution Groups" in the C++/Tree Mapping User Manual. * New option, --generate-detach, triggers the generation of detach functions for required elements and attributes. For optional and sequence cardinalities the detach functions are now provided by the respective containers even without this option. These functions, for example, allow one to move sub-trees in the object model either within the same tree or between different trees without copying. For more information, refer to Section 2.8 "Mapping for Local Elements and Attributes" in the C++/Tree Mapping User Manual. * New option, --export-xml-schema, causes the compiler to export/import types in the XML Schema namespace using the export symbol provided with the --export-symbol option. * New example, embedded, shows how to embed the binary representation of the schema grammar into an application and then use it to parse and validate XML documents. * New example, compression, shows how to compress an XML document during serialization and decompress it during parsing using the zlib library. * New example, custom/mixed, shows how to use type customization to parse and serialize mixed content. * The streaming example has been extended to show how to perform stream- oriented, partially in-memory XML processing using the C++/Tree mapping. With the partially in-memory parsing and serialization only a part of the object model is in memory at any given time. With this approach one can process parts of the document as they become available as well as handle documents that are too large to fit into memory. * New default/fixed value initialization code. Now the default/fixed values are parsed by the XSD compiler at compile time instead of the standard parsing code at runtime. This will allow the compilation of schemas that use the default/fixed values without support for XML parsing (--suppress-parsing option). * Empty XML Schema enumeration values are now mapped to the 'empty' C++ enumerator name instead of 'cxx'. * XML Schema union types with members that are enumeration types are automatically converted to equivalent enumeration types with a union of all the members' enumerators. This release also includes a large number of bug fixes and performance improvements which should be especially apparent when compiling large and complex schemas. The release also adds support for a range of new operating system and C++ compiler versions, including: * AIX 6.x * Mac OS X 10.6 "Snow Leopard" * Windows 7, * Windows Server 2008 * Visual Studio 2010 (10.0) * GNU g++ 4.5.0 * Intel C++ 11 * Sun Studio 12.1 * IBM XL C++ 11 In particular, Visual Studio 2010 (10.0) project and solution files are provided for all the examples. The XSD source code is now tracked with git and the public, read-only repository (including the web interface) is available at: http://scm.codesynthesis.com In particular, the repository allows you to view a more detailed change history between XSD 3.2.0 and this version. Precompiled binary distributions are available from the product's download page: http://www.codesynthesis.com/products/xsd/download.xhtml Source code for this release is available from the project's web page: http://www.codesynthesis.com/projects/xsd/ SHA1 checksums for the files: ebd8f13f164dafa7c4a107686a1402bbfb61c2c2 xsd-3.3.0.tar.bz2 d773c71380651365d2f309e529b0156af030d56e xsd-3.3.0+dep.tar.bz2 5beddcc360ca7916535f864c8fb8a661f6af5ce8 xsd-3.3.0-powerpc-aix.tar.gz 51b31adc735f0f6c93b2a8bdf7d7802e4884487b xsd_3.3.0-1_i386.deb e818219a1f3acd2735aa1799bdd4873c3141f998 xsd-3.3.0-1.i686.rpm 01f2462d04b3b2978aba2e8bb26305b4ba868e9d xsd-3.3.0-i686-linux-gnu.tar.bz2 b9d87369a9437c350be9cb9200e89b5b4fe1252e xsd_3.3.0-1_amd64.deb 4922c6a0609039033d780222badaa78700137e6e xsd-3.3.0-1.x86_64.rpm f265131eee1f3565041758fca457d9016a634dc6 xsd-3.3.0-x86_64-linux-gnu.tar.bz2 c580d1a0c9efdd9ac00d554569c43c8be979a0aa xsd-3.3.0-ia64-hpux.tar.gz d5d98d219489239e7007e88365ca3d3ecec95aad xsd-3.3.0-i686-macosx.tar.bz2 afa97317653f8438bcea32f73d9ee3f3ce57cf40 xsd-3.3.0-powerpc-macosx.tar.bz2 daac8faadb264d01b1f035a59d0f762c5f31c8c6 xsd-3.3.0-sparc-solaris.tar.gz 9be1e8468dd6bb180a877143044bf07918a74aac xsd-3.3.0-i686-solaris.tar.gz 4d5ed0f88b2ac45fb596b5e56bb1169f3ad19550 xsd-3.3.0-i686-windows.zip 203cab48618296819fbf518c36904ee9b778abf3 xsd-3.3.msi Last but not least, thanks to all the individuals who reported bugs, sent suggestions, and tested the alpha and beta releases. Unfortunately, the list of all who helped would be too long to include in this announcement. Enjoy, Boris From sutambe at dre.vanderbilt.edu Wed Apr 28 11:18:09 2010 From: sutambe at dre.vanderbilt.edu (Sumant Tambe) Date: Wed Apr 28 11:18:14 2010 Subject: [xsd-users] XSD 3.3.0 released In-Reply-To: References: Message-ID: <4BD851B1.9030808@dre.vanderbilt.edu> Hi Boris, Does XSD make use of new C++0x features such as rvalue references, which are now available in VS2010 and GNU g++ 4.5? If not already there, what is the plan going forward? Thanks, Sumant. Boris Kolpackov wrote: > Hi, > > We have released XSD 3.3.0. The NEWS file entries for this release > are as follows: > > * New option, --char-encoding, allows you to specify the character encoding > that should be used in the generated code. Valid values for the 'char' > character type are 'utf8' (default), 'iso8859-1' (new), 'lcp' (Xerces-C++ > local code page), and 'custom' (provides support for custom encodings). > Note that if you use a non-default character encoding and include some > libxsd headers (e.g., xsd/cxx/xml/string.hxx) directly, then you will > need to first include the correct xsd/cxx/xml/char-.hxx header, > where is iso8859-1, lcp, etc. This mechanism replaces the > XSD_USE_LCP macro. > > For the wchar_t character type the only valid value for this option is > 'auto' and the encoding is automatically selected between UTF-16 and > UTF-32, depending on the wchar_t type size. > > * When the XSD compiler is built with Xerces-C++ 3.1.0 or later, the > handling of multiple imports for the same namespace is enabled. Before, > all subsequent imports for a namespace were ignored which caused errors > in some schemas. Note that if your application has XML Schema validation > enabled, then you will also need to build it with Xerces-C++ 3.1.0 or > later to take advantage of this feature. > > * Automatic mapping for the urn-style XML namespaces. The last component > in the urn name is used to derive the C++ namespace name. > > * New option, --schema-file-regex, in combination with the existing > --type-file-regex, can be used to place the generated files into > subdirectories or to resolve file name conflicts in the file-per- > type mode (--file-per-type). > > * Warning id's have changed to start with a letter identifying the > component issuing the warning. F - compiler frontend, D - compiler > driver, P - C++/Parser mapping, T - C++/Tree mapping. > > * Strings used to match regular expressions supplied with the > --namespace-regex and --anonymous-regex options now include the file > component for the schema being compiled. > > * The XSD_NO_EXPORT macro can be used to omit code generated with the > --export/import-maps and, for C++/Tree, --generate-xml-schema options > during C++ compilation. This may be useful if you would like to use > the same generated code across multiple platforms. > > C++/Tree > > * New option, --generate-element-type, triggers the generation of types > instead of parsing/serialization functions for root elements. This > is primarily useful to distinguish object models with the same root > type but with different root elements. For more information, refer > to the messaging example and Section 2.9.1, "Element Types" in the > C++/Tree Mapping User Manual. To support the customization of the > element type naming the --element-type-regex option has been added. > See the NAMING CONVENTION section in the compiler command line manual > (man pages) for details. > > * New option, --generate-element-map, triggers the generation of a root > element map. The element map allows uniform parsing and serialization > of multiple root elements. This option can only be used together with > --generate-element-type. For more information, refer to the messaging > example and Section 2.9.2, "Element Map" in the C++/Tree Mapping > User Manual. > > * Prior to this version, if the --generate-polymorphic option is > specified, the compiler treats all types as potentially polymorphic. > Now by default only type hierarchies used in substitution groups and > those explicitly declared polymorphic with the new --polymorphic-type > option are treated as polymorphic. This results in smaller and faster > generated code. If you would like to continue using the old behavior, > you will need to specify --polymorphic-type-all. For more information, > on this change see Section 2.11, "Mapping for xsi:type and Substitution > Groups" in the C++/Tree Mapping User Manual. > > * New option, --generate-detach, triggers the generation of detach > functions for required elements and attributes. For optional and > sequence cardinalities the detach functions are now provided by the > respective containers even without this option. These functions, for > example, allow one to move sub-trees in the object model either within > the same tree or between different trees without copying. For more > information, refer to Section 2.8 "Mapping for Local Elements and > Attributes" in the C++/Tree Mapping User Manual. > > * New option, --export-xml-schema, causes the compiler to export/import > types in the XML Schema namespace using the export symbol provided > with the --export-symbol option. > > * New example, embedded, shows how to embed the binary representation of > the schema grammar into an application and then use it to parse and > validate XML documents. > > * New example, compression, shows how to compress an XML document during > serialization and decompress it during parsing using the zlib library. > > * New example, custom/mixed, shows how to use type customization to parse > and serialize mixed content. > > * The streaming example has been extended to show how to perform stream- > oriented, partially in-memory XML processing using the C++/Tree mapping. > With the partially in-memory parsing and serialization only a part of > the object model is in memory at any given time. With this approach one > can process parts of the document as they become available as well as > handle documents that are too large to fit into memory. > > * New default/fixed value initialization code. Now the default/fixed values > are parsed by the XSD compiler at compile time instead of the standard > parsing code at runtime. This will allow the compilation of schemas that > use the default/fixed values without support for XML parsing > (--suppress-parsing option). > > * Empty XML Schema enumeration values are now mapped to the 'empty' C++ > enumerator name instead of 'cxx'. > > * XML Schema union types with members that are enumeration types are > automatically converted to equivalent enumeration types with a union > of all the members' enumerators. > > This release also includes a large number of bug fixes and performance > improvements which should be especially apparent when compiling large and > complex schemas. > > The release also adds support for a range of new operating system and C++ > compiler versions, including: > > * AIX 6.x > * Mac OS X 10.6 "Snow Leopard" > * Windows 7, > * Windows Server 2008 > > * Visual Studio 2010 (10.0) > * GNU g++ 4.5.0 > * Intel C++ 11 > * Sun Studio 12.1 > * IBM XL C++ 11 > > In particular, Visual Studio 2010 (10.0) project and solution files are > provided for all the examples. > > The XSD source code is now tracked with git and the public, read-only > repository (including the web interface) is available at: > > http://scm.codesynthesis.com > > In particular, the repository allows you to view a more detailed change > history between XSD 3.2.0 and this version. > > Precompiled binary distributions are available from the product's > download page: > > http://www.codesynthesis.com/products/xsd/download.xhtml > > Source code for this release is available from the project's web page: > > http://www.codesynthesis.com/projects/xsd/ > > SHA1 checksums for the files: > > ebd8f13f164dafa7c4a107686a1402bbfb61c2c2 xsd-3.3.0.tar.bz2 > d773c71380651365d2f309e529b0156af030d56e xsd-3.3.0+dep.tar.bz2 > > 5beddcc360ca7916535f864c8fb8a661f6af5ce8 xsd-3.3.0-powerpc-aix.tar.gz > 51b31adc735f0f6c93b2a8bdf7d7802e4884487b xsd_3.3.0-1_i386.deb > e818219a1f3acd2735aa1799bdd4873c3141f998 xsd-3.3.0-1.i686.rpm > 01f2462d04b3b2978aba2e8bb26305b4ba868e9d xsd-3.3.0-i686-linux-gnu.tar.bz2 > b9d87369a9437c350be9cb9200e89b5b4fe1252e xsd_3.3.0-1_amd64.deb > 4922c6a0609039033d780222badaa78700137e6e xsd-3.3.0-1.x86_64.rpm > f265131eee1f3565041758fca457d9016a634dc6 xsd-3.3.0-x86_64-linux-gnu.tar.bz2 > c580d1a0c9efdd9ac00d554569c43c8be979a0aa xsd-3.3.0-ia64-hpux.tar.gz > d5d98d219489239e7007e88365ca3d3ecec95aad xsd-3.3.0-i686-macosx.tar.bz2 > afa97317653f8438bcea32f73d9ee3f3ce57cf40 xsd-3.3.0-powerpc-macosx.tar.bz2 > daac8faadb264d01b1f035a59d0f762c5f31c8c6 xsd-3.3.0-sparc-solaris.tar.gz > 9be1e8468dd6bb180a877143044bf07918a74aac xsd-3.3.0-i686-solaris.tar.gz > 4d5ed0f88b2ac45fb596b5e56bb1169f3ad19550 xsd-3.3.0-i686-windows.zip > 203cab48618296819fbf518c36904ee9b778abf3 xsd-3.3.msi > > Last but not least, thanks to all the individuals who reported bugs, sent > suggestions, and tested the alpha and beta releases. Unfortunately, the > list of all who helped would be too long to include in this announcement. > > Enjoy, > Boris From boris at codesynthesis.com Wed Apr 28 13:42:22 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Apr 28 13:33:28 2010 Subject: [xsd-users] GML 3.2.1 [Update] In-Reply-To: <77A6AA04-2884-4CFF-A1E6-52CC222951AE@ll.mit.edu> References: <77A6AA04-2884-4CFF-A1E6-52CC222951AE@ll.mit.edu> Message-ID: Hi Brad, Brad Howes writes: > I just found out that one must only go through the gml.xsd schema > per the 3.2.1 spec, and not via any of the component files. Doing > it the piece-meal way appeared to work, but may have not been sound... I just uploaded some sample Makefiles/XSD options files/README files to the Wiki that show how to compile GML 3.2.1 (only tested with XSD 3.3.0): http://wiki.codesynthesis.com/Schemas/GML Let us know how it goes. Boris From howes at ll.mit.edu Wed Apr 28 15:52:43 2010 From: howes at ll.mit.edu (Brad Howes) Date: Wed Apr 28 15:52:52 2010 Subject: [xsd-users] GML 3.2.1 [Update] In-Reply-To: References: <77A6AA04-2884-4CFF-A1E6-52CC222951AE@ll.mit.edu> Message-ID: <52F61EAF-42C8-40C9-95E3-203682641332@ll.mit.edu> On Apr 28, 2010, at 1:42 PM, Boris Kolpackov wrote: > I just uploaded some sample Makefiles/XSD options files/README files > to the Wiki that show how to compile GML 3.2.1 (only tested with XSD > 3.3.0): > > http://wiki.codesynthesis.com/Schemas/GML > > Let us know how it goes. Boris, If you are ever in Cambridge please let me buy you your favorite beverage. And the same holds if I'm ever in South Africa -- assuming that is where you are now. That was one of the nicest tarballs I have seen in some time. Thanks a mil. Regards, Brad -- Brad Howes Group 42 MIT Lincoln Laboratory ? 244 Wood St. ? Lexington, MA 02173 Phone: 781.981.5292 ? Fax: 781.981.3495 ? Secretary: 781.981.7420 From eric at boostpro.com Wed Apr 28 18:07:36 2010 From: eric at boostpro.com (Eric Niebler) Date: Wed Apr 28 18:07:45 2010 Subject: partial in-memory streaming example (was: [xsd-users] XSD 3.3.0 released) In-Reply-To: References: Message-ID: <4BD8B1A8.4030203@boostpro.com> On 4/28/2010 3:00 AM, Boris Kolpackov wrote: > * The streaming example has been extended to show how to perform stream- > oriented, partially in-memory XML processing using the C++/Tree mapping. > With the partially in-memory parsing and serialization only a part of > the object model is in memory at any given time. With this approach one > can process parts of the document as they become available as well as > handle documents that are too large to fit into memory. I got very excited when I read this because my application is running out of memory when serializing large DOM's. I had a look at this example, but sadly it doesn't look like it will solve my problem. IIUC, you're doing incremental DOM serialization by loading a shell for the root, and then serializing children of the root one at a time. That works for large documents that are flat; that is, most of their content are direct children of the root. But what about deeply nested documents? Is there a way to incrementally parse/serialize such a document using the C++/Tree mapping? Aside: it looks like the XSD serialization routines are first serializing XSD DOM to xerces DOM, and then using xerces routines to serialize this to XML. Serializing to xerces DOM is where my app runs out of memory -- sad because it seems like this step could be eliminated entirely if there were some way to go straight from XSD DOM to XML. Thoughts? -- Eric Niebler BoostPro Computing http://www.boostpro.com From boris at codesynthesis.com Thu Apr 29 09:44:23 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 29 09:35:08 2010 Subject: [xsd-users] XSD 3.3.0 released In-Reply-To: <4BD851B1.9030808@dre.vanderbilt.edu> References: <4BD851B1.9030808@dre.vanderbilt.edu> Message-ID: Hi Sumant, Sumant Tambe writes: > Does XSD make use of new C++0x features such as rvalue references, which > are now available in VS2010 and GNU g++ 4.5? No, no support for C++0x yet. > If not already there, what is the plan going forward? Hm, I guess it all depends on whether there is interest from the community. If/when someone needs this for a real application then we will consider it. Boris From boris at codesynthesis.com Thu Apr 29 10:17:25 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 29 10:08:07 2010 Subject: [xsd-users] XSD 3.3.0 blog post Message-ID: Hi, I wrote a blog post that includes additional details/insights/anecdotes on some of the major new features in XSD 3.3.0. I thought some of you might be interested: http://www.codesynthesis.com/~boris/blog/2010/04/29/xsd-3-3-0-released/ Boris From boris at codesynthesis.com Thu Apr 29 10:40:17 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 29 10:31:07 2010 Subject: partial in-memory streaming example (was: [xsd-users] XSD 3.3.0 released) In-Reply-To: <4BD8B1A8.4030203@boostpro.com> References: <4BD8B1A8.4030203@boostpro.com> Message-ID: Hi Eric, Eric Niebler writes: > I got very excited when I read this because my application is running > out of memory when serializing large DOM's. I had a look at this > example, but sadly it doesn't look like it will solve my problem. > > IIUC, you're doing incremental DOM serialization by loading a shell for > the root, and then serializing children of the root one at a time. That > works for large documents that are flat; that is, most of their content > are direct children of the root. But what about deeply nested documents? > Is there a way to incrementally parse/serialize such a document using > the C++/Tree mapping? It is a bit more work but the principle is the same. Most (all) large documents have repetitive elements that account for the size. The idea is to get to that repetitive "level" and then parse/serialize a chunk at a time. If your document looks like this: ... ... ... Then you would construct the parser (the parser here is vocabulary- specific) to handle the , , and elements internally and start returning as document fragments. With serialization, you will have to recreate the and fragments manually and serialize one at a time. This will also work if your document looks like this: ... ... ... ... ... ... ... That is, you have multiple places with repetitive content. > Aside: it looks like the XSD serialization routines are first > serializing XSD DOM to xerces DOM, and then using xerces routines to > serialize this to XML. Serializing to xerces DOM is where my app runs > out of memory -- sad because it seems like this step could be eliminated > entirely if there were some way to go straight from XSD DOM to XML. > Thoughts? Yes, that is theoretically possible though (1) there is some flexibility in being able to "post-process" DOM before writing it out to XML and (2) your case is from a (small) subset of situations where the document is still small enough to fit into memory as aither object model or DOM but not both. The much bigger set of cases is where the document simply cannot fit into memory in either representation. The streaming approach tries to address this more general set of problems. Also, I don't think it will be long before your case will migrate to the bigger set ;-). Boris From boris at codesynthesis.com Thu Apr 29 10:52:42 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 29 10:43:27 2010 Subject: [xsd-users] xerces exceptions In-Reply-To: <4BD5DFA7.7050709@boostpro.com> References: <4BD5DFA7.7050709@boostpro.com> Message-ID: Hi Eric, Eric Niebler writes: > I've discovered (a bit too late sadly) that xsd lets xerces exceptions > propagate. Ideally, I'd like to deal exclusively with std and xsd types > and not let any xerces types leak into my code. Is there an option to > have xsd catch and translate xerces exceptions? No, no such option yet. Though I agree we should do it and I have added an item to the TODO list. What can make it less of a problem is the fact that only "hard" errors are reported by Xerces-C++ as exceptions (e.g., an out of memory condition). Things like XML parsing errors are handled via error handlers and are translated by the XSD-generated code. Boris From eric at boostpro.com Thu Apr 29 10:45:34 2010 From: eric at boostpro.com (Eric Niebler) Date: Thu Apr 29 10:45:48 2010 Subject: [xsd-users] xerces exceptions In-Reply-To: References: <4BD5DFA7.7050709@boostpro.com> Message-ID: <4BD99B8E.6030300@boostpro.com> On 4/29/2010 7:52 AM, Boris Kolpackov wrote: > Hi Eric, > > Eric Niebler writes: > >> I've discovered (a bit too late sadly) that xsd lets xerces exceptions >> propagate. Ideally, I'd like to deal exclusively with std and xsd types >> and not let any xerces types leak into my code. Is there an option to >> have xsd catch and translate xerces exceptions? > > No, no such option yet. Though I agree we should do it and I have > added an item to the TODO list. > > What can make it less of a problem is the fact that only "hard" > errors are reported by Xerces-C++ as exceptions (e.g., an out of > memory condition). Things like XML parsing errors are handled via > error handlers and are translated by the XSD-generated code. Thanks, Boris. Where can I find a complete list of the "hard" errors? I need to know what xerces exceptions to handle in my code. -- Eric Niebler BoostPro Computing http://www.boostpro.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 551 bytes Desc: OpenPGP digital signature Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20100429/63acbce8/signature.pgp From boris at codesynthesis.com Thu Apr 29 13:43:42 2010 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Apr 29 13:34:26 2010 Subject: [xsd-users] xerces exceptions In-Reply-To: <4BD99B8E.6030300@boostpro.com> References: <4BD5DFA7.7050709@boostpro.com> <4BD99B8E.6030300@boostpro.com> Message-ID: Hi Eric, Eric Niebler writes: > Thanks, Boris. Where can I find a complete list of the "hard" errors? I > need to know what xerces exceptions to handle in my code. That would be: xercesc::XMLException (util/XMLException.hpp) Most Xerces-C++ exceptions are derived from this base exception (except for the following three). It has some member functions which should allow you to query a text message. xercesc::OutOfMemoryException (util/OutOfMemoryException.hpp) This one is special in that construction of XMLException may trigger this condition so OutOfMemoryException cannot derive from XMLException. xercesc::DOMException (dom/DOMException.hpp) This is the base for DOM exceptions. If it is thrown from XSD- generated code then that means that either the Xerces-C++ parser/serializer tried to perform an illegal DOM operation (bug in Xerces-C++) or the XSD-generated code did so (bug in XSD). Also has some member functions which should allow you to query a text message. xercesc::SAXException (sax/SAXException.hpp) Similar to DOM exception. While SAX is not used by C++/Tree mapping directly, you may want to handle this exception if you decide to use the streaming approach. Boris