From boris at codesynthesis.com Mon Aug 3 05:52:24 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] XSD accepted into Fedora Message-ID: Hi, I am pleased to announce that thanks to the efforts of Antti Andreimann, XSD has been accepted into the Fedora 10 and 11 package repositories as well as into the next RedHat Linux release. The package name is 'xsd'. Due to a conflict with another package (and similar to the Debian package), the compiler executable has been renamed from 'xsd' to 'xsdcxx'. The XSD runtime include paths remain unchanged and the only thing you need to do if you want to use the package from the Fedora/RedHat repository instead of the one from the Code Synthesis website is to change the XSD compiler name. Enjoy, Boris From shaohua.liu at siemens.com Tue Aug 4 19:06:57 2009 From: shaohua.liu at siemens.com (Liu, Shaohua) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Generate C++ header to be used on unix machine Message-ID: <36AA8A77334555429DAF0A60C6C569DE06CD7502D0@USSLMMBX001.net.plm.eds.com> I used the CodeSynthesis xsd tool to generate a C++ header file from a xsd file with this command: "xsd mySchema.xsd /classes /l:CPP" It created mySchema.h header file. When I opened this header file, I found it references to "System::CodeDom" namespace. I think this is a new library in .NET 3.5 framework. But I will have to use the header file on a Unix machine, so any .NET reference will not work. I found that this xsd tool can be built on Unix machine. So I think it should be able to create C++ header files without .NET references. I just wonder how I can do that on my Windows dev machine. Is there any options that I can use to indicate that the output should be unmanaged C++ or for unix? Thanks Shaohua From boris at codesynthesis.com Wed Aug 5 05:45:13 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Generate C++ header to be used on unix machine In-Reply-To: <36AA8A77334555429DAF0A60C6C569DE06CD7502D0@USSLMMBX001.net.plm.eds.com> References: <36AA8A77334555429DAF0A60C6C569DE06CD7502D0@USSLMMBX001.net.plm.eds.com> Message-ID: Hi Shaohua, Liu, Shaohua writes: > I used the CodeSynthesis xsd tool to generate a C++ header file from > a xsd file with this command: > > "xsd mySchema.xsd /classes /l:CPP" I am afraid you are invoking Microsoft's .Net XSD.exe tool instead of CodeSynthesis XSD. The two products have the same executable name. If you don't need to use the Microsoft's version, you can put the path to the bin\ directory inside the CodeSynthesis XSD installation directory at the beginning of your PATH environment variable. This will make CodeSynthesis xsd.exe the default command invoked when you type xsd. Alternatively, you can use the full path when running the compiler, for example: C:\Program Files\CodeSynthesis XSD 3.2\bin\xsd.exe For more information on how to setup and use CodeSynthesis XSD on Windows, see this document: http://wiki.codesynthesis.com/Using_XSD_with_Microsoft_Visual_Studio In particular, refer to the "Conflict with Microsoft xsd.exe" section at the end. > But I will have to use the header file on a Unix machine, so any > .NET reference will not work. I found that this xsd tool can be > built on Unix machine. So I think it should be able to create > C++ header files without .NET references. Yes, that's correct. CodeSynthesis XSD-generated C++ code does not depend on .Net. > I just wonder how I can do that on my Windows dev machine. Is > there any options that I can use to indicate that the output > should be unmanaged C++ or for unix? If you run the correct executable as mentioned above, by default you will get unmanaged C++ that is portable to UNIX platforms. I suggest that you try something like this: xsd cxx-tree MySchema.xsd Or: C:\Program Files\CodeSynthesis XSD 3.2\bin\xsd.exe cxx-tree MySchema.xsd Boris From Alex.Jakes at acis.uk.com Mon Aug 17 06:23:21 2009 From: Alex.Jakes at acis.uk.com (Alex Jakes) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] How to create XML with mappings for substitution groups with same type using cxx-tree Message-ID: <3B6FFDD1C5CC314C9C191EE32F41CE791EFABB5131@camexsvr01.acis.local> Hello, I have a problem that you have already partly addressed in a posting by Ninh Tran Dang called "[xsd-users] Parsing for the substitutionGroup". http://www.codesynthesis.com/pipermail/xsd-users/2008-March/001543.html I have a large schema that uses the technique of using substitution groups with the same type but with different element names. Essentially using the element name to encode information about the type instead of using the type to do that and I am also using cxx-tree. I agree that this is a bit of a silly way to do things but I am stuck with it and have to cope with it. Your reply to Ninh suggested using DOM directly to parse the document as in the multiroot example. I have done this and it works for parsing such files. The problem comes when I want to generate XML using the XSD generated code. I would normally just build the data structure I want in memory and then serialise it. Do you have any suggestions for how I might do this? Thanks, Alex. This email and the information it contains are confidential and may be privileged, if you have received this email in error please notify us immediately. You should not copy it for any purpose, or disclose its contents to any other person. Internet communications are not secure and therefore ACIS does not accept legal responsibility for the contents of this message as it has been transmitted over a public network. If you suspect the message may have been intercepted or amended, please call the sender. Registered office: ACIS House, 168 Cowley Road, Cambridge, CB4 0DL. Registered in England and Wales, Company Number 3039051 ______________________________________________________________________ This email has been scanned by the MessageLabs Email Security System. For more information please visit http://www.messagelabs.com/email ______________________________________________________________________ From boris at codesynthesis.com Mon Aug 17 09:13:13 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] How to create XML with mappings for substitution groups with same type using cxx-tree In-Reply-To: <3B6FFDD1C5CC314C9C191EE32F41CE791EFABB5131@camexsvr01.acis.local> References: <3B6FFDD1C5CC314C9C191EE32F41CE791EFABB5131@camexsvr01.acis.local> Message-ID: Hi Alex, Alex Jakes writes: > Your reply to Ninh suggested using DOM directly to parse the document > as in the multiroot example. I have done this and it works for parsing > such files. The problem comes when I want to generate XML using the XSD > generated code. I would normally just build the data structure I want > in memory and then serialise it. Do you have any suggestions for how > I might do this? There is the ANSI/NIST-ITL schema which has the same issue. To resolve it I implemented custom xml_schema::type which stores the substitution group information for each type in the object model. It is automatically retrieved during parsing and used to assign proper element names during serialization. The application can also set it if, for example, creating the object model from scratch. This is essentially approach #3 as listed in my reply to Ninh. You can try the same method for your schema. The source code for NIST-ITL test is available from this page: http://wiki.codesynthesis.com/Schemas/NIST-ITL The xml-schema-custom.* files should be usable as-is. You will just need to compile your schemas with new options (see itl.options and xsd.options) and change your application to use the new approach. Let me know if there are any snags. Boris From vherard at alphasqr.com Mon Aug 17 10:27:46 2009 From: vherard at alphasqr.com (vherard@alphasqr.com) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] .if-then else and for-each next constructs in schema Message-ID: <20090817072746.f9339f9fc63982ff33f985f6b62d68cf.22b6ac10a2.wbe@email.secureserver.net> Does codesynthesis xsd schema currently support if-then else, for- next, while constructs? If so can you point me to some examples. not is this something that will be added and does anyone know time-table? Thanks, vh From boris at codesynthesis.com Mon Aug 17 10:45:09 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] .if-then else and for-each next constructs in schema In-Reply-To: <20090817072746.f9339f9fc63982ff33f985f6b62d68cf.22b6ac10a2.wbe@email.secureserver.net> References: <20090817072746.f9339f9fc63982ff33f985f6b62d68cf.22b6ac10a2.wbe@email.secureserver.net> Message-ID: Hi, vherard@alphasqr.com writes: > Does codesynthesis xsd schema currently support if-then else, for-next, > while constructs? If so can you point me to some examples. XML Schema doesn't really support procedural programming so I am not sure what you are referring to. Can you elaborate on what exactly you are trying to achieve? Boris From vherard at alphasqr.com Mon Aug 17 11:13:55 2009 From: vherard at alphasqr.com (vherard@alphasqr.com) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] .if-then else and for-each next constructs in schema Message-ID: <20090817081355.f9339f9fc63982ff33f985f6b62d68cf.6be1fa92fd.wbe@email.secureserver.net> I realize that native xml does not have conditional logic, but I' seen some post where people have added extension to xml. In an case I'm simply looking for a way to do in-memory XML tree modification such as doing some simple branching logic based on some return = 50%; want to branch different part of having to resort to w document. Is this asking too mu -------- Original Message -------- Subject: Re: [xsd-users] .if- in schema From: Boris Kolpacko Date: Mon, August 17, 2009 10:45 am To: vherard@alphasqr.com Cc: xsd-users@codesynthesis.com Hi, vherard@alphasqr.com writes: > D for-next, & XML S not sure wha exactly you are trying Boris From atteeela at gmail.com Tue Aug 18 01:00:22 2009 From: atteeela at gmail.com (Attila) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Help with XML Schema to map to "proper" XSD types. Message-ID: <4aeb04aa0908172200k557bf3c9r8c52c8fdcd015308@mail.gmail.com> Hi, I am creating a new messaging protocol. I would like to be able to have the general structure of my messages as follows: (Please forgive the namespace declarations...for they are wrong --- that is where I am running into trouble) Envelope.xsd: -snip- -snip- Then I have a number of "ServiceObjects" that I would like to associate with a specific XSD and extend the base types of RequestMsg_t, ResponseMsg_t and NotificationMsg_t as follows. ConfigObject.xsd: -snip- -snip- Now there may be many different types of definitions such as AppObject.xsd, ProcessObject.xsd, etc. I just listed ConfigObject.xsd as an example above. I would like to get this following XML instance documents to be validated by the schemas above: SampleCfgAddRequest.xml: -snip- SOMETHING -snip- and also: SampleCfgAddResponse.xml: -snip-
1 1
asdf -snip- I have tried quite a lot using the different namespace declarations, but I cannot figure out how to get it all to work. Other requirements are that an application that uses ConfigObject.xsd (and Envelope.xsd) must not know/care about AppObject.xsd, ProcessObject.xsd, etc... and likewise for each. This is important because not all of our applications will support these different Objects( or Services we can call them). However they must all use a common Envelope in the base code to manage all of this. My focus will be on using Code Synthesis XSD to get it all to work so that I can have the Object manager classes only know about the Envelope and the specific libraries will know how to decode the specific Request/Response/Notification types. Any help would be greatly appreciated. Thank you -- Attila Software Developer atteeela@gmail.com From boris at codesynthesis.com Tue Aug 18 09:25:06 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] .if-then else and for-each next constructs in schema In-Reply-To: <20090817081355.f9339f9fc63982ff33f985f6b62d68cf.6be1fa92fd.wbe@email.secureserver.net> References: <20090817081355.f9339f9fc63982ff33f985f6b62d68cf.6be1fa92fd.wbe@email.secureserver.net> Message-ID: Hi, vherard@alphasqr.com writes: > In an case I'm simply looking for a way to do in-memory XML tree > modification such as doing some simple branching logic based on some > returned value. For example I might have an initial value for rate = 50%; > then if I calculate "rate" and it's more than some value I want to branch > to a different part of the tree else I go to a different part of ree. Since the modifications are done in C++, you can use its branching and looping constructs or use various algorithms provided by the C++ standard library and Boost. > I was hoping to be able to do that without having to resort to writing > transformations of the original document. Is this asking too much? Does > this make sense? I am still not entirely clear what you are trying to achieve. Where are these transformations performed? In C++? Somewhere else? A small but concrete example that shows what you need to achieve would be very helpful. Boris From boris at codesynthesis.com Tue Aug 18 09:42:42 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Help with XML Schema to map to "proper" XSD types. In-Reply-To: <4aeb04aa0908172200k557bf3c9r8c52c8fdcd015308@mail.gmail.com> References: <4aeb04aa0908172200k557bf3c9r8c52c8fdcd015308@mail.gmail.com> Message-ID: Hi Attila, Attila writes: > I would like to get this following XML instance documents to be > validated by the schemas above: > > > SampleCfgAddRequest.xml: > -snip- > > > > > SOMETHING > > > > > -snip- > > and also: > > > SampleCfgAddResponse.xml: > -snip- > > > >
> 1 > 1 >
> > asdf > > > > -snip- There is no way to achieve this exact form for your XML documents. That is, without any namespace-prefix declarations and using type to specify the dynamic type. If you want to keep XML representation as minimal as possible, your best bet is to not use namespaces and to use substitution groups. Then you will be able to do something like this: SOMETHING and
1 1
asdf Note that we use the element names (CfgAddMessageRequest, CfgAddMessageResponse) to carry the type information. In your schemas you will need to make the following changes: 1. Get rid of all targetNamespace/elementFormDefault attributes. 2. Use xs:include instead of xs:import. 3. Add substitution groups: Envelope.xsd: ConfigObject.xsd: You can then pre-load the schemas before parsing your documents as shown in the 'caching' example. Boris From eric at boostpro.com Tue Aug 18 17:25:14 2009 From: eric at boostpro.com (Eric Niebler) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning Message-ID: <4A8B1C3A.2040406@boostpro.com> Hi all. I'm looking for some advice about schema evolution and validation. I have a crazy idea but I don't know enough about XML schema in general and CodeSynth XSD in particular to determine whether it's feasible. First, the problem: My tool allows users to create instance documents that match a particular schema, say schema1.xsd. Now, I release a new version of the tool and a new schema, say schema2.xsd, that adds elements to certain schema types. To keep the old instance documents readable, I make these new elements optional. This results in a schema that is looser than I would like; if I can read instance documents with the missing elements, I can also write them, and I don't want that. I want to express an asymmetry: it's ok for an element to be missing on read, but not on write. Here's my crazy idea: Maybe I could play games with namespace maps and/or xsd:import/xsd:include and xsd:restriction to define a stricter write schema that is defined in terms of -- and refines -- the read schema. Is something like this even possible? Is this the right way to be thinking about this problem, or are there better ways? Thanks in advance, -- Eric Niebler BoostPro Computing http://www.boostpro.com From boris at codesynthesis.com Wed Aug 19 10:01:23 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: <4A8B1C3A.2040406@boostpro.com> References: <4A8B1C3A.2040406@boostpro.com> Message-ID: Hi Eric, Eric Niebler writes: > First, the problem: My tool allows users to create instance documents > that match a particular schema, say schema1.xsd. Now, I release a new > version of the tool and a new schema, say schema2.xsd, that adds > elements to certain schema types. To keep the old instance documents > readable, I make these new elements optional. This results in a schema > that is looser than I would like; if I can read instance documents with > the missing elements, I can also write them, and I don't want that. I > want to express an asymmetry: it's ok for an element to be missing on > read, but not on write. Versioning is quite a tricky matter. How is the document that conforms to the previous version of the schema going to be "fixed up" to conform to the current version before writing it? (Here I assume that you will need to write it though this may not be the case if, for example, documents are either read or created from scratch.) What I am trying to say is that detecting missing elements during serialization is only half of the story. The other half is what should happen once such a situation is detected as well as how will the user (or the software) make sure that such a situation doesn't happen. > Here's my crazy idea: Maybe I could play games with namespace maps > and/or xsd:import/xsd:include and xsd:restriction to define a stricter > write schema that is defined in terms of -- and refines -- the read > schema. Is something like this even possible? If all you need is more strict write validation (there is actually not support for validation during writing in C++/Tree so you will need to re-parse the XML to detect any errors), then the easiest way would be to mark the new elements in your schema with a special attribute (XML Schema allows attributes from other namespaces to be added to declarations), for example: Then you can convert this schema to get a "write version" by adjusting minOccurs for elements with the writeRequired attribute. In fact, you don't even need to have two files: you can process this schema on-the-fly with a simple DOM function, serialize the result to an in-memory buffer and then load it into a grammar cache to be used by Xerces-C++ for validation. > Is this the right way to be thinking about this problem, or are > there better ways? I think the biggest challenge in versioning is how the old document is transformed to a new document. There are several plausible options: default values are assigned to missing parts or it is the responsibility of the user to find and provide the missing parts. While XML Schema does not address versioning, we can definitely do something in the generated code to help with this (perhaps based on some schema extensions similar to the approach shown above). But the fist step is to really understand the use cases and it would be helpful to know how you plan to handle the other problems of versioning. Boris From eric at boostpro.com Wed Aug 19 12:15:29 2009 From: eric at boostpro.com (Eric Niebler) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: References: <4A8B1C3A.2040406@boostpro.com> Message-ID: <4A8C2521.5060102@boostpro.com> Hi Boris, Boris Kolpackov wrote: > > Eric Niebler writes: > >> First, the problem: My tool allows users to create instance documents >> that match a particular schema, say schema1.xsd. Now, I release a new >> version of the tool and a new schema, say schema2.xsd, that adds >> elements to certain schema types. To keep the old instance documents >> readable, I make these new elements optional. This results in a schema >> that is looser than I would like; if I can read instance documents with >> the missing elements, I can also write them, and I don't want that. I >> want to express an asymmetry: it's ok for an element to be missing on >> read, but not on write. > > Versioning is quite a tricky matter. > > How is the document that conforms to the previous version of the schema > going to be "fixed up" to conform to the current version before writing > it? (Here I assume that you will need to write it though this may not be > the case if, for example, documents are either read or created from scratch.) > > What I am trying to say is that detecting missing elements during > serialization is only half of the story. The other half is what should > happen once such a situation is detected as well as how will the user > (or the software) make sure that such a situation doesn't happen. Right. Once we detect a missing element on read, we'll programmatically fill in a default. That will happen in our serialization routines. >> Here's my crazy idea: Maybe I could play games with namespace maps >> and/or xsd:import/xsd:include and xsd:restriction to define a stricter >> write schema that is defined in terms of -- and refines -- the read >> schema. Is something like this even possible? > > If all you need is more strict write validation (there is actually > not support for validation during writing in C++/Tree so you will > need to re-parse the XML to detect any errors) This surprises me. If I write a schema that enforces that a particular sequence have 3 or more elements, and I only insert 2 elements into the sequence, you're saying that this schema violation won't be detected on write, but only when reading the instance document back in? > , then the easiest > way would be to mark the new elements in your schema with a special > attribute (XML Schema allows attributes from other namespaces to > be added to declarations), for example: > > > > > > > > > Then you can convert this schema to get a "write version" by adjusting > minOccurs for elements with the writeRequired attribute. In fact, you > don't even need to have two files: you can process this schema on-the-fly > with a simple DOM function, serialize the result to an in-memory buffer > and then load it into a grammar cache to be used by Xerces-C++ for > validation. Interesting suggestion! What do you mean by a "simple DOM function"? Is this something simpler than a full-blown XSLT transform? >> Is this the right way to be thinking about this problem, or are >> there better ways? > > I think the biggest challenge in versioning is how the old document > is transformed to a new document. There are several plausible options: > default values are assigned to missing parts I looked into default values for schema elements but support in XML Schema is very weak. The element type must be primitive or a simpleType, IIRC. And the behavior of CodeSynth XSD wasn't appropriate. Empty elements are handled differently than missing elements, which differs from how defaulted attributes are handled. I don't understand the reasoning here, but it seems to make default values for elements useless for versioning. Please correct me if I'm wrong. > or it is the responsibility > of the user to find and provide the missing parts. While XML Schema does > not address versioning, we can definitely do something in the generated > code to help with this (perhaps based on some schema extensions similar > to the approach shown above). In-tool support for versioning would rock. Consider this a +1 for an xse:writeRequired attribute that CodeSynth XSD recognizes and does something sensible with. > But the fist step is to really understand > the use cases and it would be helpful to know how you plan to handle > the other problems of versioning. Yes, it's a complicated problem. In our tool, we've identified several versioning scenarios. Far and away the most common versioning scenario is the one I described above: adding a new element to an existing schema type. That's the narrow problem I'm currently trying to address. To handle more complicated versioning scenarios (e.g., an element moves from a child to a parent, and other more complex variants), we imagine we can handle this with XSLT transforms that can convert instance documents from version N to N+1 on the fly on read. We're even looking into tools for automatically generating the XSLT from diffs between schema(N) and schema(N+1). Frankly, I have low expectations from such tools and expect that we'll have to resort to writing the XSLT by hand. Thanks, -- Eric Niebler BoostPro Computing http://www.boostpro.com From atteeela at gmail.com Wed Aug 19 22:41:49 2009 From: atteeela at gmail.com (Attila) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Re: Help with XML Schema to map to "proper" XSD types. In-Reply-To: References: <4aeb04aa0908172200k557bf3c9r8c52c8fdcd015308@mail.gmail.com> Message-ID: <4aeb04aa0908191941k3e1416f9y878c273933491c24@mail.gmail.com> Thanks Boris, this is exactly what I am looking for. On 8/18/09, Boris Kolpackov wrote: > Hi Attila, > > Attila writes: > >> I would like to get this following XML instance documents to be >> validated by the schemas above: >> >> >> SampleCfgAddRequest.xml: >> -snip- >> >> >> >> >> SOMETHING >> >> >> >> >> -snip- >> >> and also: >> >> >> SampleCfgAddResponse.xml: >> -snip- >> >> >> >>
>> 1 >> 1 >>
>> >> asdf >> >> >> >> -snip- > > There is no way to achieve this exact form for your XML documents. > That is, without any namespace-prefix declarations and using type > to specify the dynamic type. If you want to keep XML representation > as minimal as possible, your best bet is to not use namespaces and > to use substitution groups. Then you will be able to do something > like this: > > > > > SOMETHING > > > > > and > > > >
> 1 > 1 >
> > asdf > > > > > Note that we use the element names (CfgAddMessageRequest, > CfgAddMessageResponse) to carry the type information. > > In your schemas you will need to make the following changes: > > 1. Get rid of all targetNamespace/elementFormDefault attributes. > > 2. Use xs:include instead of xs:import. > > 3. Add substitution groups: > > Envelope.xsd: > > > > > > > > > > > > > > > > > > > > > ConfigObject.xsd: > > type="CfgAddMessageRequest_t" > substitutionGroup="RequestMsg"/> > > type="CfgAddMessageResponse_t" > substitutionGroup="RequestMsg"/> > > You can then pre-load the schemas before parsing your documents as > shown in the 'caching' example. > > Boris > -- Attila Software Developer atteeela@gmail.com From boris at codesynthesis.com Thu Aug 20 12:34:59 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: <4A8C2521.5060102@boostpro.com> References: <4A8B1C3A.2040406@boostpro.com> <4A8C2521.5060102@boostpro.com> Message-ID: Hi Eric, Eric Niebler writes: > Right. Once we detect a missing element on read, we'll programmatically > fill in a default. That will happen in our serialization routines. Is it going to be done as a reaction to a validation during serialization error or proactively before the validation? > > If all you need is more strict write validation (there is actually > > not support for validation during writing in C++/Tree so you will > > need to re-parse the XML to detect any errors) > > This surprises me. If I write a schema that enforces that a particular > sequence have 3 or more elements, and I only insert 2 elements into the > sequence, you're saying that this schema violation won't be detected on > write, but only when reading the instance document back in? Yes, that's what will happen by default. If you need validation on serialization, the only way to get it with C++/Tree is to re-parse the resulting document after serialization. For example, serialize it into a memory buffer if it is not very big, re-parse it (perhaps using SAX2 for speed), and, if everything is ok, write the memory buffer to a file, etc. On the surface validation during serialization often seem like a good idea. However, once you start thinking about what to do in case of an error, its usefulness becomes questionable, except, maybe, for debugging, in which case the re-parsing approach works just fine. For more information see the following post. It is about in-memory validation but a lot of questions raised there also apply to validation during serialization: http://www.codesynthesis.com/pipermail/xsd-users/2008-January/001443.html > > Then you can convert this schema to get a "write version" by adjusting > > minOccurs for elements with the writeRequired attribute. In fact, you > > don't even need to have two files: you can process this schema on-the-fly > > with a simple DOM function, serialize the result to an in-memory buffer > > and then load it into a grammar cache to be used by Xerces-C++ for > > validation. > > Interesting suggestion! What do you mean by a "simple DOM function"? Is > this something simpler than a full-blown XSLT transform? By simple DOM function I mean a function in your program that will load the schema as a DOM tree, find all the elements with the writeRequired attribute, change their minOccurs to 1, and serialize the modified DOM tree into a memory buffer. This memory buffer can then be passed directly to loadGrammar(). Full-blown XSLT will also work and is probably simpler and quicker to implement (especially if you have multiple schema files connected via include/import). But the DOM approach is tidier since you don't need to carry two sets of schemas with your application. > I looked into default values for schema elements but support in XML > Schema is very weak. The element type must be primitive or a simpleType, > IIRC. And the behavior of CodeSynth XSD wasn't appropriate. Empty > elements are handled differently than missing elements, which differs > from how defaulted attributes are handled. I don't understand the > reasoning here, but it seems to make default values for elements useless > for versioning. Please correct me if I'm wrong. Default elements in XML Schema are a misnomer. The spec requires the empty element to be present in the XML instance in order for it to have the default value. This makes default elements practically unusable. > In-tool support for versioning would rock. Consider this a +1 for an > xse:writeRequired attribute that CodeSynth XSD recognizes and does > something sensible with. Yes, I agree. We just need to figure out what it is that we can do that is sensible ;-). > Yes, it's a complicated problem. In our tool, we've identified several > versioning scenarios. Far and away the most common versioning scenario > is the one I described above: adding a new element to an existing schema > type. That's the narrow problem I'm currently trying to address. I think that's the only scenario that could be practically addressed or helped by the tool. For example, for elements that are optional but marked as required during serialization we could generate an interface that is something between optional and required. That is, the user can still query whether the element is present or not but it cannot set it to the "not present" state. For such elements we could also require initialization during construction. In other words, it could be just like a required element except it may not be set during parsing and there is a way to detect this situation. We could also implement a check during serialization to make sure these elements are actually specified. But I feel that it is only a part of the solution. The user of the mapping still has to detect the missing elements and provide some default values manually. I am wondering if there is a way to maybe automate it somehow. Are the default values that you assign to missing elements known during schema compilation or are they computed based on the other parts of the document at runtime? Boris From eric at boostpro.com Fri Aug 21 11:52:58 2009 From: eric at boostpro.com (Eric Niebler) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: References: <4A8B1C3A.2040406@boostpro.com> <4A8C2521.5060102@boostpro.com> Message-ID: <4A8EC2DA.9060401@boostpro.com> Boris Kolpackov wrote: > Hi Eric, > > Eric Niebler writes: > >> Right. Once we detect a missing element on read, we'll programmatically >> fill in a default. That will happen in our serialization routines. > > Is it going to be done as a reaction to a validation during serialization > error or proactively before the validation? Well, the idea is that the element will be optional, so it's absence on read will not be a schema violation. But I've already told you that, so I feel like I must not be understanding your question. Can you clarify? In our tool, after we (de-)serialize from XML to DOM, we walk the DOM and find the missing elements and fill in defaults. Does that clear it up? Ideally, we would mark up the xsd and have CodeSynth fill in the defaults for us, but that's something else entirely. >>> If all you need is more strict write validation (there is actually >>> not support for validation during writing in C++/Tree so you will >>> need to re-parse the XML to detect any errors) >> >> This surprises me. If I write a schema that enforces that a particular >> sequence have 3 or more elements, and I only insert 2 elements into the >> sequence, you're saying that this schema violation won't be detected on >> write, but only when reading the instance document back in? > > Yes, that's what will happen by default. If you need validation on > serialization, the only way to get it with C++/Tree is to re-parse > the resulting document after serialization. For example, serialize > it into a memory buffer if it is not very big, re-parse it (perhaps > using SAX2 for speed), and, if everything is ok, write the memory > buffer to a file, etc. Ouch. These documents are large (10-100 Mb uncompressed XML) and writing them is slow. It seems to me that an extra write/read/validate is not going to be a satisfactory solution, except as a debugging tool. > On the surface validation during serialization often seem like a good > idea. However, once you start thinking about what to do in case of > an error, its usefulness becomes questionable, except, maybe, for > debugging I disagree, and this is where I see a situation where CodeSynth can provide a value-add over plain mapping and validation. Imagine that we have optional elements with special CodeSynth markup for what the value the element should take when it's missing. If CodeSynth did a schema validation pass on serialization, this is where the value for missing elements could be filled in automatically. That is, CodeSynth can proactively correct simple schema violations with a little guidance from the xsd author. > , in which case the re-parsing approach works just fine. > For more information see the following post. It is about in-memory > validation but a lot of questions raised there also apply to > validation during serialization: > > http://www.codesynthesis.com/pipermail/xsd-users/2008-January/001443.html If the issue is merely one of error detection and reporting, then yes I agree that validation on serialization doesn't make much sense except for debugging. But if you're willing to consider error correction, then it does make sense. >>> Then you can convert this schema to get a "write version" by adjusting >>> minOccurs for elements with the writeRequired attribute. In fact, you >>> don't even need to have two files: you can process this schema on-the-fly >>> with a simple DOM function, serialize the result to an in-memory buffer >>> and then load it into a grammar cache to be used by Xerces-C++ for >>> validation. >> >> Interesting suggestion! What do you mean by a "simple DOM function"? Is >> this something simpler than a full-blown XSLT transform? > > By simple DOM function I mean a function in your program that will load > the schema as a DOM tree, find all the elements with the writeRequired > attribute, change their minOccurs to 1, and serialize the modified DOM > tree into a memory buffer. This memory buffer can then be passed > directly to loadGrammar(). > > Full-blown XSLT will also work and is probably simpler and quicker > to implement (especially if you have multiple schema files connected > via include/import). But the DOM approach is tidier since you don't > need to carry two sets of schemas with your application. Confused. If I use XSLT to process the schema, then I can still ship only 1 set of schema, right? >> I looked into default values for schema elements but support in XML >> Schema is very weak. The element type must be primitive or a simpleType, >> IIRC. And the behavior of CodeSynth XSD wasn't appropriate. Empty >> elements are handled differently than missing elements, which differs >> from how defaulted attributes are handled. I don't understand the >> reasoning here, but it seems to make default values for elements useless >> for versioning. Please correct me if I'm wrong. > > Default elements in XML Schema are a misnomer. The spec requires the > empty element to be present in the XML instance in order for it to > have the default value. This makes default elements practically > unusable. That's the conclusion I reached. I see this as an opportunity for CodeSynth. >> In-tool support for versioning would rock. Consider this a +1 for an >> xse:writeRequired attribute that CodeSynth XSD recognizes and does >> something sensible with. > > Yes, I agree. We just need to figure out what it is that we can do > that is sensible ;-). So maybe xse:writeRequired isn't quite what I want. I'd like a way to provide default values for optional elements in a way that they are filled in automatically on read; and on write, either (a) fill them in automatically, or (b) flag their absence as an error. This would address the large majority of our versioning scenarios. >> Yes, it's a complicated problem. In our tool, we've identified several >> versioning scenarios. Far and away the most common versioning scenario >> is the one I described above: adding a new element to an existing schema >> type. That's the narrow problem I'm currently trying to address. > > I think that's the only scenario that could be practically addressed > or helped by the tool. For example, for elements that are optional > but marked as required during serialization we could generate an > interface that is something between optional and required. That is, > the user can still query whether the element is present or not but > it cannot set it to the "not present" state. Right. > For such elements we > could also require initialization during construction. Ah, but we are using the --generate-default-ctor option, so that wouldn't help us. > In other > words, it could be just like a required element except it may not > be set during parsing and there is a way to detect this situation. > > We could also implement a check during serialization to make sure > these elements are actually specified. Yes. Or to fill in a default. > But I feel that it is only a part of the solution. The user of the > mapping still has to detect the missing elements and provide some > default values manually. We're already doing that in our tool, so that's ok. > I am wondering if there is a way to maybe > automate it somehow. Me too! > Are the default values that you assign to missing elements known > during schema compilation or are they computed based on the other > parts of the document at runtime? For the most part, they would be known during schema compilation. In the cases where they must be computed from other parts of the document, we can fall back to doing that part ourselves. No big deal. -- Eric Niebler BoostPro Computing http://www.boostpro.com From bpringle at sympatico.ca Sat Aug 22 12:43:59 2009 From: bpringle at sympatico.ca (Bill Pringlemeir) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: <4A8B1C3A.2040406@boostpro.com> (Eric Niebler's message of "Tue, 18 Aug 2009 17:25:14 -0400") References: <4A8B1C3A.2040406@boostpro.com> Message-ID: <87eir3u7cg.fsf@sympatico.ca> On 18 Aug 2009, eric@boostpro.com wrote: > First, the problem: My tool allows users to create instance documents > that match a particular schema, say schema1.xsd. Now, I release a new > version of the tool and a new schema, say schema2.xsd, that adds > elements to certain schema types. To keep the old instance documents > readable, I make these new elements optional. This results in a schema > that is looser than I would like; if I can read instance documents > with the missing elements, I can also write them, and I don't want > that. I want to express an asymmetry: it's ok for an element to be > missing on read, but not on write. I was considering a similar problem/issue. You are delivered an 'Enterprise' or 'Industry' schema but your application only needs a subset of the data to operate on. A straight transformation produces a lot of code that will not be needed. I was thinking of something like XSLT (metaphorically, not exactly) that would transform a schema from one version to another. The PCCTS has a program called sorcerer which has a language for transforming trees. I think that a similar type of activity could automate this transformation. Ie, given an input schema different types would be qualified as not applicable, read-only, write-only and read-write; XSLT referenced above is an analog to this transformation syntax. Elements could be specified to have defaults (not present in the original) that would need to ful-fill the input schema's document model. The code could then output a schema that would comply with the input schema, but have a reduced document model. When this was feed to a data binding tool like XSD, the generated source would be much smaller. ... but that is all fantasy. I vaguely remember someone posting something that sounded like this on the xsd-user list. It sounds like you are trying to do something like this by hand. Fwiw, Bill Pringlemeir. From bpringle at sympatico.ca Sat Aug 22 13:15:16 2009 From: bpringle at sympatico.ca (Bill Pringlemeir) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: (Boris Kolpackov's message of "Wed, 19 Aug 2009 16:01:23 +0200") References: <4A8B1C3A.2040406@boostpro.com> Message-ID: <87ab1ru5wb.fsf@sympatico.ca> On 19 Aug 2009, boris@codesynthesis.com wrote: > But the fist step is to really understand the use cases and it would > be helpful to know how you plan to handle the other problems of > versioning. I can give several use cases not necessarily related to versioning, 1. You are a North American wishing to write an open source TV capture application. In order to do this you need schedule information. There is a SOAP service provided by Tribune Media Services, http://xmlbeans.googlepages.com/ - entry TMS XTVD - Tribune Media Services eXtensible TV Data It is highly unlikely that you will be generating the program schedules in your program. At the SOAP level, you wish to generate/write the REQUEST messages and read the RESPONSE. You might wish to use the generate the RESPONSE schedule by reading from a persisted file. However, this would generally be a subset of the returned data and you might choose another data storage mechanism such as a database. This would be a benefit if you wanted to query for any recent 'movies' that will be airing and automate the capture of these programs via a TV tuner card. At first if might seem like XML persistence is a good thing, but it would probably only be useful for debugging. 2. You are part of a financial institution that is using the FpML schemas as a standard. Your application only deals with Mortgages. Derivative, currency exchange, and many other data modeled in the FpML are not in the scope of your application. You want to use the FpML schemas (and any future updates) and restrict the amount of generated code needed by your application's domain. These are both generally code size issues. Maybe these are similar problems or a sub-set of the versioning issue? Fwiw, Bill Pringlemeir. From aamano1 at msn.com Wed Aug 26 11:52:41 2009 From: aamano1 at msn.com (Alan Amano) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Excluding Schema Elements For the XSD Compiler (C++/Tree) To Ignore Message-ID: With C++/Tree, are there any ways to exclude certain simple/complex type elements so that the XSD compiler will ignore generating code for those types? I have a very large schema file and it?s generating C++ source and header files that are huge, resulting in lengthy C++ compile/link times downstream. The schema file is a ?baseline? and is shared among multiple users and I am only using a fraction of the type definitions in the schema file. Obviously, I can extract only those types I?m using into a separate schema file, but I?d rather not do this and risk getting out of sync from the baseline. Unless I'm missing something, there weren?t any obvious XSD compiler options that I can see to accomplish this, so if you can recommend any methods or options to exclude types from the XSD compiler, I would appreciate it. From ssak22 at gmail.com Wed Aug 26 16:32:06 2009 From: ssak22 at gmail.com (ss ak) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] xsd to xml parser Message-ID: <467867bb0908261332t11e14ceah312a7d670bd1d262@mail.gmail.com> I have generated the C++ binding using the below... xsd.exe cxx-tree --root-element-first --generate-serialization --generate-from-base-ctor --generate-default-ctor v4.xsd using the classes,i filled in the values... Now how to i make these classes into XML??? or print as an XML. There is method in the set of generated methods... parse (::xsd::cxx::xml::dom::parser< char >& p,::xml_schema::flags f) what does this do???what can i give the input to this...i didnt understand... i tried this... xsd::cxx::xml::dom::auto_ptr parser (impl->createLSParser (DOMImplementationLS::MODE_SYNCHRONOUS, 0)); myclass.parse(parser,????); what should be these values... I just want to fill in teh values and make an XML which can be validated with the schema... Please point me the right direction or just give me what i have to do..please it is the need of the hour... thanks, From boris at codesynthesis.com Wed Aug 26 16:43:12 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] xsd to xml parser In-Reply-To: <467867bb0908261332t11e14ceah312a7d670bd1d262@mail.gmail.com> References: <467867bb0908261332t11e14ceah312a7d670bd1d262@mail.gmail.com> Message-ID: Hi, ss ak writes: > I have generated the C++ binding using the below... > > xsd.exe cxx-tree --root-element-first --generate-serialization > --generate-from-base-ctor --generate-default-ctor v4.xsd > > using the classes,i filled in the values... > > Now how to i make these classes into XML??? or print as an XML. You would use serialization functions for that which are generated when you specify the --generate-serialization option. Take a look at Section 2.5, "Adding Serialization" in the C++/Tree Mapping Getting Started Guide for a quick overview: http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#2.5 For more a detailed information, see Chapter 6, "Serialization" in the same document: http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#6 There are also a number of examples that demonstrate how to save the object model to XML. The 'library' example in the examples/cxx/tree/ directory would be a good start. Boris From ssak22 at gmail.com Wed Aug 26 17:13:28 2009 From: ssak22 at gmail.com (ss ak) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] xsd to xml parser In-Reply-To: References: <467867bb0908261332t11e14ceah312a7d670bd1d262@mail.gmail.com> Message-ID: <467867bb0908261413r7ddc2c65s6bf960a5a0df85ba@mail.gmail.com> I donot have any method like in the example generated... // Serialize to a string. std::ostringstream oss; mp(oss, p, map); std::string xml (oss.str ()); example has: xml_schema::namespace_infomap map; map["lib"].name = "http://www.codesynthesis.com/library"; map["lib"].schema = "library.xsd"; // Write it out. catalog_ (std::cout, *c, map); --->this type of method is not generated..how do i get this... } catch (const xml_schema::exception& e) { cerr << e << endl; return 1; } On Wed, Aug 26, 2009 at 1:43 PM, Boris Kolpackov wrote: > Hi, > > ss ak writes: > > > I have generated the C++ binding using the below... > > > > xsd.exe cxx-tree --root-element-first --generate-serialization > > --generate-from-base-ctor --generate-default-ctor v4.xsd > > > > using the classes,i filled in the values... > > > > Now how to i make these classes into XML??? or print as an XML. > > You would use serialization functions for that which are generated > when you specify the --generate-serialization option. Take a look > at Section 2.5, "Adding Serialization" in the C++/Tree Mapping > Getting Started Guide for a quick overview: > > http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#2.5 > > For more a detailed information, see Chapter 6, "Serialization" in > the same document: > > http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#6 > > There are also a number of examples that demonstrate how to save the > object model to XML. The 'library' example in the examples/cxx/tree/ > directory would be a good start. > > Boris > > From ssak22 at gmail.com Wed Aug 26 17:16:04 2009 From: ssak22 at gmail.com (ss ak) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] xsd to xml parser In-Reply-To: <467867bb0908261413r7ddc2c65s6bf960a5a0df85ba@mail.gmail.com> References: <467867bb0908261332t11e14ceah312a7d670bd1d262@mail.gmail.com> <467867bb0908261413r7ddc2c65s6bf960a5a0df85ba@mail.gmail.com> Message-ID: <467867bb0908261416r47e47fe5x48f7aa68f3ff9b90@mail.gmail.com> On Wed, Aug 26, 2009 at 2:13 PM, ss ak wrote: > I donot have any method like in the example generated... > > Same Question... > what does PARSE method which is created does... what are the parameters to be given...rather how to create those parameters parse (::xsd::cxx::xml::dom::parser< char >& p,::xml_schema::flags f) > On Wed, Aug 26, 2009 at 1:43 PM, Boris Kolpackov > wrote: > >> Hi, >> >> ss ak writes: >> >> > I have generated the C++ binding using the below... >> > >> > xsd.exe cxx-tree --root-element-first --generate-serialization >> > --generate-from-base-ctor --generate-default-ctor v4.xsd >> > >> > using the classes,i filled in the values... >> > >> > Now how to i make these classes into XML??? or print as an XML. >> >> You would use serialization functions for that which are generated >> when you specify the --generate-serialization option. Take a look >> at Section 2.5, "Adding Serialization" in the C++/Tree Mapping >> Getting Started Guide for a quick overview: >> >> >> http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#2.5 >> >> For more a detailed information, see Chapter 6, "Serialization" in >> the same document: >> >> http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#6 >> >> There are also a number of examples that demonstrate how to save the >> object model to XML. The 'library' example in the examples/cxx/tree/ >> directory would be a good start. >> >> Boris >> >> > From ssak22 at gmail.com Wed Aug 26 17:48:56 2009 From: ssak22 at gmail.com (ss ak) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] xsd cxx-tree --generate-serialization Message-ID: <467867bb0908261448w337175cax7226d0c2c567ccc4@mail.gmail.com> xsd cxx-tree --generate-serialization test.xsd This type of method is not generated when i use void hello (std::ostream&, const hello_t&, const xml_schema::namespace_infomap& = xml_schema::namespace_infomap ()); As in http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#2.5 While parsing and accessing the XML data may be everything you need, there are applications that require creating new or modifying existing XML documents. By default XSD does not produce serialization code. We will need to request it with the --generate-serialization options: $ xsd cxx-tree --generate-serialization hello.xsd If we now examine the generated hello.hxx file, we will find a set of overloaded serialization functions, including the following version: void hello (std::ostream&, const hello_t&, const xml_schema::namespace_infomap& = xml_schema::namespace_infomap ()); From ssak22 at gmail.com Wed Aug 26 20:16:33 2009 From: ssak22 at gmail.com (ss ak) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Never failing in Parsing Message-ID: <467867bb0908261716v6c0ebf0p9aebe3b13ad94397@mail.gmail.com> I have enum values {in,out} but what ever values given,parser never gives an exception... please look at the code and tell me what is wrong...I am here only after scanning all the examples... It need of the hour of my work...please xml_schema::namespace_infomap map; map[""].schema = "test.xsd"; std::ostringstream oss; nt_(oss, var_nt, map); //here nt is my class generated std::string xml (oss.str ()); printf("\n XML %s",xml.c_str()); //here it prints the xml valid or not try{ xml_schema::properties props; props.no_namespace_schema_location("test.xsd"); nt_(std::cout,var_nt,map,"UTF-8",xml_schema::flags::dont_initialize); } catch (const xml_schema::exception& e) { printf("\n test.xml: write error %s",e.what()); } From boris at codesynthesis.com Thu Aug 27 09:32:08 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Re: Never failing in Parsing In-Reply-To: <467867bb0908261716v6c0ebf0p9aebe3b13ad94397@mail.gmail.com> References: <467867bb0908261716v6c0ebf0p9aebe3b13ad94397@mail.gmail.com> Message-ID: Hi, I see you have figured out how to serialize the object model back to XML. In the future, please don't send multiple emails asking essentially the same question. One is enough and someone will reply when/if they have time. If you need more predictable assistance, I suggest you consider purchasing a Priority Support contract: http://www.codesynthesis.com/support/ ss ak writes: > but what ever values given,parser never gives an exception... You mean serializer, not parser? Your code below doesn't perform any parsing, only serialization. > xml_schema::namespace_infomap map; > > map[""].schema = "test.xsd"; > > std::ostringstream oss; > > nt_(oss, var_nt, map); //here nt is my class generated > > std::string xml (oss.str ()); > > printf("\n XML %s",xml.c_str()); //here it prints the xml valid or not The C++/Tree mapping doesn't support automatic XML Schema validation during serialization. What you can do, however, is re-parse the XML to make sure it is valid. So given the code above, you can add: try { std::istringstream iss (xml); nt_(iss, ""); } catch (const xml_schema::exception& e) { std::cerr << e << std::endl; } Boris From boris at codesynthesis.com Thu Aug 27 10:42:37 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Excluding Schema Elements For the XSD Compiler (C++/Tree) To Ignore In-Reply-To: References: Message-ID: Hi Alan, Alan Amano writes: > With C++/Tree, are there any ways to exclude certain simple/complex > type elements so that the XSD compiler will ignore generating code > for those types? I have a very large schema file and it?s generating > C++ source and header files that are huge, resulting in lengthy C++ > compile/link times downstream. The schema file is a "baseline" and > is shared among multiple users and I am only using a fraction of the > type definitions in the schema file. Obviously, I can extract only > those types I?m using into a separate schema file, but I?d rather > not do this and risk getting out of sync from the baseline. Unless > I'm missing something, there weren?t any obvious XSD compiler > options that I can see to accomplish this, so if you can recommend > any methods or options to exclude types from the XSD compiler, I > would appreciate it. There is no automatic way to do this at the moment. The approach that can (almost) work is to compile your schema in the file-per- type mode. This will result in a separate set of C++ files generated for each type defined in the schema. You can then pick the ones that you need and discard the rest. The reason I said it can almost work is because the parsing and serialization functions for the root elements will be generated in the file corresponding to the schema file which in turn includes all the generated header files. Here is an example: Let's say we have test.xsd which defines two types: 'foo' and 'bar'. It also defines global element 'root' of type 'foo'. If we compile this schema in the file-per-type mode, we will end up with the following C++ files: foo.hxx, foo.cxx - class foo bar.hxx, bar.cxx - class bar test.hxx, test.cxx - root() parsing/serialization functions The test.hxx file will also have #include directives for foo.hxx and bar.hxx. So if we remove bar.?xx because it is not used in our application, we will get a compile error. There are two ways to work around this: you can rename the test.?xx files and change the test.hxx file to only include headers that are needed (you can already generate parsing/serialization functions only for certain root elements with the --root-element* options). This should work since, presumably, root element names and their types don't change very often. Or you can drop the parsing and serialization functions altogther and call the parsing constructors and serialization operators directly. For this you will need to perform your own XML-to-DOM and DOM-to-XML conversions (see the 'performace' example for some sample code that shows how to do this). As for a fully-automatic way to do this, it would be quite easy to implement for a single-file schemas. For example, you would specify the root element(s) and the compiler will generate only what's needed, directly and indirectly, by this root element(s) (this will not work that easily for schemas that use polymorphism). This, however, won't work for multi-file schemas in the file-per- schema mode. When we compile included/imported schemas, we don't know which types from it will be used by the including/importing schemas. This can be worked-around if we require that when the "minimal generated code" feature is on, all schemas are compiled together. We already do something like this in XSD/e to support polymorphism. Boris From boris at codesynthesis.com Thu Aug 27 12:15:30 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: <4A8EC2DA.9060401@boostpro.com> References: <4A8B1C3A.2040406@boostpro.com> <4A8C2521.5060102@boostpro.com> <4A8EC2DA.9060401@boostpro.com> Message-ID: Hi Eric, Eric Niebler writes: > > Is it going to be done as a reaction to a validation during serialization > > error or proactively before the validation? > > Well, the idea is that the element will be optional, so it's absence on > read will not be a schema violation. But I've already told you that, so > I feel like I must not be understanding your question. Can you clarify? > > In our tool, after we (de-)serialize from XML to DOM, we walk the DOM > and find the missing elements and fill in defaults. Does that clear it > up? Yes, I was thinking about these two options: you can either do it proactively like you do now or it can theoretically be done as a reaction to validation during serialization errors. > Ideally, we would mark up the xsd and have CodeSynth fill in the > defaults for us, but that's something else entirely. Yes, it seems there are two general ways in which the user may want this to be handled: (1) Fill in the defaults for missing elements. (2) Notify the user somehow about missing elements, presumably, during parsing. > > On the surface validation during serialization often seem like a good > > idea. However, once you start thinking about what to do in case of > > an error, its usefulness becomes questionable, except, maybe, for > > debugging > > I disagree, and this is where I see a situation where CodeSynth can > provide a value-add over plain mapping and validation. Imagine that we > have optional elements with special CodeSynth markup for what the value > the element should take when it's missing. If CodeSynth did a schema > validation pass on serialization, this is where the value for missing > elements could be filled in automatically. Wouldn't this be more naturally done during parsing so that the application developer doesn't have to worry about missing elements? > That is, CodeSynth can proactively correct simple schema violations > with a little guidance from the xsd author. Such things can be implemented quite easily either during parsing or during serialization. And I agree, they can be useful. It is the full XML Schema validation that, IMO, is not very useful since it is not clear what to do when there is an error. > If the issue is merely one of error detection and reporting, then yes I > agree that validation on serialization doesn't make much sense except > for debugging. But if you're willing to consider error correction, then > it does make sense. I think this will only work for automatic error correction, as above. In case the user intervention is required, it is not clear how to supply the error information (e.g., position, description, etc.) that can be usable for anything other than an error message. > > Full-blown XSLT will also work and is probably simpler and quicker > > to implement (especially if you have multiple schema files connected > > via include/import). But the DOM approach is tidier since you don't > > need to carry two sets of schemas with your application. > > Confused. If I use XSLT to process the schema, then I can still ship > only 1 set of schema, right? If you have a full-blown XSLT processor inside your application, then yes, you can ;-). > So maybe xse:writeRequired isn't quite what I want. I'd like a way to > provide default values for optional elements in a way that they are > filled in automatically on read; and on write, either (a) fill them in > automatically, or (b) flag their absence as an error. If they are filled in on read, why do you also need this on write? Perhaps the application can create the object model and not provide new elements (because you generate default c-tors)? Is that the use case you had in mind? > > But I feel that it is only a part of the solution. The user of the > > mapping still has to detect the missing elements and provide some > > default values manually. > > We're already doing that in our tool, so that's ok. That sounds like quite a tedious task. I wonder what API can we use to notify the application about missing elements and request the default values..? We could use DOM to return the default values but returning an object model instance would be more convenient. We could also pass the reference to the containing object model node in case the handler needs it. One wild idea is to specify the handler function in the schema file, for example: The signature of the person_age_value() function would be: int person_age_value (person& p); Then we can do something like: std::map ages = ... int person_age_value (person& p) { return ages[p.name ()]; } We would then have the companion attribute, xse:defaultValue, which is used to specify the default value in some form. Maybe an XPath to an element in the "default values file", for example: The remaining question is how the values from this file are accessed to construct object model nodes. The cleanest approach would be to somehow embed these values directly into the generated code. Ideally, we would have a static object model instance that represents the default value. But it is not clear how to initialize it (Xerces-C++ is not usable during static initialization). Boris From Laura_E_Fowler at raytheon.com Thu Aug 27 13:46:16 2009 From: Laura_E_Fowler at raytheon.com (Laura E Fowler) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] hexBinary basic question Message-ID: I'm sure that I'm doing something stupid, but I'm having trouble using hexBinary. I have an uint64_t CRC value that is to be represented in hexBinay, so the xsd has the field defined as: I'm converting my uint64_t value to hexBinary as follows: xml_schema::hex_binary((void *)(&dataResidue),sizeof(dataResidue)) I'm running on a little-endian system, so I have tried converting my uint64_t to big-endian. Neither little-endian or big-endian gave me what I expected. For example: If my CRC contains the value of: 0x2d8ab0bc260ef5a I get a hexBinary of: 0106000030706040 Can you please point me in the correct direction? I've searched the archives, but didn't find anything that helped much. Thanks, Laura From Laura_E_Fowler at raytheon.com Thu Aug 27 15:52:04 2009 From: Laura_E_Fowler at raytheon.com (Laura E Fowler) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Re: hexBinary basic question In-Reply-To: References: Message-ID: Yes, I did do something stupid. I used the value before it was set, so it was just a simple (stupid) coding error. I did have to put it into big-endian, but that's to be expected. Now it's working like I expected it to work. Sorry for bothering everyone. Laura From: Laura E Fowler/US/Raytheon To: xsd-users@codesynthesis.com Date: 08/27/2009 12:46 PM Subject: hexBinary basic question I'm sure that I'm doing something stupid, but I'm having trouble using hexBinary. I have an uint64_t CRC value that is to be represented in hexBinay, so the xsd has the field defined as: I'm converting my uint64_t value to hexBinary as follows: xml_schema::hex_binary((void *)(&dataResidue),sizeof(dataResidue)) I'm running on a little-endian system, so I have tried converting my uint64_t to big-endian. Neither little-endian or big-endian gave me what I expected. For example: If my CRC contains the value of: 0x2d8ab0bc260ef5a I get a hexBinary of: 0106000030706040 Can you please point me in the correct direction? I've searched the archives, but didn't find anything that helped much. Thanks, Laura From ssak22 at gmail.com Thu Aug 27 12:33:07 2009 From: ssak22 at gmail.com (ss ak) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Re: Never failing in Parsing In-Reply-To: References: <467867bb0908261716v6c0ebf0p9aebe3b13ad94397@mail.gmail.com> Message-ID: <467867bb0908270933p2c7cc271l6defdd4325e175d1@mail.gmail.com> Thanks Boris... Last Q hope fully...when do we use this type of parsing...i mean using the xerces calls directly.. when i m using xsd.exe to generate my classes giving xsd as input... using namespace xercesc; const XMLCh ls_id[] = {xercesc::chLatin_L, xercesc::chLatin_S, xercesc::chNull}; DOMImplementation* impl ( DOMImplementationRegistry::getDOMImplementation (ls_id)); // Use the error handler implementation provided by the XSD runtime xsd::cxx::tree::error_handler eh; xsd::cxx::xml::dom::bits::error_handler_proxy ehp (eh); xml_schema::dom::auto_ptr parser ( impl->createLSParser (DOMImplementationLS::MODE_SYNCHRONOUS, 0)); DOMConfiguration* conf (parser->getDomConfig ()); conf->setParameter (XMLUni::fgDOMComments, false); conf->setParameter (XMLUni::fgDOMDatatypeNormalization, true); conf->setParameter (XMLUni::fgDOMEntities, false); conf->setParameter (XMLUni::fgDOMNamespaces, true); conf->setParameter (XMLUni::fgDOMElementContentWhitespace, false); // Set error handler. // conf->setParameter (XMLUni::fgDOMErrorHandler, &ehp); if (validate) { conf->setParameter (XMLUni::fgDOMValidate, true); conf->setParameter (XMLUni::fgXercesSchema, true); conf->setParameter (XMLUni::fgXercesSchemaFullChecking, false); // If we are validating, pre-load and cache the schema. // parser->loadGrammar ("test.xsd", Grammar::SchemaGrammarType, true); conf->setParameter (XMLUni::fgXercesUseCachedGrammarInParse, true); } else { conf->setParameter (XMLUni::fgDOMValidate, false); conf->setParameter (XMLUni::fgXercesSchema, false); conf->setParameter (XMLUni::fgXercesSchemaFullChecking, false); } conf->setParameter (XMLUni::fgXercesUserAdoptsDOMDocument, true); // Create memory buffer input source. MemBufInputSource is (reinterpret_cast (buf), size, file, false); is.setCopyBufToStream (false); Wrapper4InputSource wis (&is, false); // Parsing loop. os::time start; for (unsigned long i (0); i < iter; ++i) {// First parse XML to DOM reusing the parser we created above. xml_schema::dom::auto_ptr doc (parser->parse (&wis)); eh.throw_if_failed (); // Then parse DOM to the object model. auto_ptr r (test::root_ (*doc)); } Please ...let me know when we use this type of thing...when using xsd to generate all my classes ... On Thu, Aug 27, 2009 at 6:32 AM, Boris Kolpackov wrote: > Hi, > > I see you have figured out how to serialize the object model back > to XML. In the future, please don't send multiple emails asking > essentially the same question. One is enough and someone will reply > when/if they have time. If you need more predictable assistance, I > suggest you consider purchasing a Priority Support contract: > > http://www.codesynthesis.com/support/ > > > ss ak writes: > > > but what ever values given,parser never gives an exception... > > You mean serializer, not parser? Your code below doesn't perform > any parsing, only serialization. > > > > xml_schema::namespace_infomap map; > > > > map[""].schema = "test.xsd"; > > > > std::ostringstream oss; > > > > nt_(oss, var_nt, map); //here nt is my class generated > > > > std::string xml (oss.str ()); > > > > printf("\n XML %s",xml.c_str()); //here it prints the xml valid or not > > The C++/Tree mapping doesn't support automatic XML Schema validation > during serialization. What you can do, however, is re-parse the XML > to make sure it is valid. So given the code above, you can add: > > try > { > std::istringstream iss (xml); > nt_(iss, ""); > } > catch (const xml_schema::exception& e) > { > std::cerr << e << std::endl; > } > > Boris > From ssak22 at gmail.com Thu Aug 27 19:57:43 2009 From: ssak22 at gmail.com (ss ak) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Datetime Message-ID: <467867bb0908271657k1664b4ffg16428a572b31cf3e@mail.gmail.com> I have a datetime as string..Thu Aug 27 15:42:20 2009 would like assign to one of the variable generated as XSD:DataTime. Is there any way to this...or we have to parse the string and allocate it DateTime dt(11,11,11,11,11,11); Please reply.. From boris at codesynthesis.com Fri Aug 28 13:02:01 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Re: Never failing in Parsing In-Reply-To: <467867bb0908270933p2c7cc271l6defdd4325e175d1@mail.gmail.com> References: <467867bb0908261716v6c0ebf0p9aebe3b13ad94397@mail.gmail.com> <467867bb0908270933p2c7cc271l6defdd4325e175d1@mail.gmail.com> Message-ID: Hi, ss ak writes: > Last Q hope fully...when do we use this type of parsing...i mean using the > xerces calls directly.. > when i m using xsd.exe to generate my classes giving xsd as input... The generated parsing and serialization functions perform all the necessary low-level Xerces-C++ operations for most common cases. However, there are situations where you may want to configure the Xerces-C++ parser/serializer in a special way or reuse the same parser/serializer to parse/serialize several documents. In such situations you will need to write code similar to the one you have shown (taken from the performance example). In this particular case, the direct interaction with Xerces-C++ is used to achieve two things: (1) reuse the same parser to parse multiple XML documents and (2) pre-load and cache the schema that will be used to validate these documents. Boris From boris at codesynthesis.com Fri Aug 28 13:05:56 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] hexBinary basic question In-Reply-To: References: Message-ID: Hi Laura, Laura E Fowler writes: > I have an uint64_t CRC value that is to be represented in hexBinay, so the > xsd has the field defined as: > The xsd:unsignedLong type in XML Schema is 64-bit unsigned integer. You could use that instead of hexBinary. This way you don't need to be concerned with endian-ness issues. Just a thought. Boris From boris at codesynthesis.com Fri Aug 28 13:21:02 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] Re: Datetime In-Reply-To: <467867bb0908271657k1664b4ffg16428a572b31cf3e@mail.gmail.com> References: <467867bb0908271657k1664b4ffg16428a572b31cf3e@mail.gmail.com> Message-ID: Hi, ss ak writes: > I have a datetime as string..Thu Aug 27 15:42:20 2009 > would like assign to one of the variable generated as XSD:DataTime. > > Is there any way to this...or we have to parse the string and allocate it > DateTime dt(11,11,11,11,11,11); With the default mapping of the dateTime type you will have to parse the string and initialize the instance with individual parts. If all you need is to support this particular format in a few places in your application, then implementing a function that converts it to xml_schema::date_time is probably the simplest approach. Alternatively, you can provide a custom mapping for this type and base it, for example, on the date_time class from Boost which appears to support all kinds of text representations of date/time. There is the 'calendar' example in the examples/cxx/tree/custom/ directory which shows how to do this for the date type. Boris From juanpablo.perez at gmail.com Fri Aug 28 14:11:45 2009 From: juanpablo.perez at gmail.com (Juan Pablo Perez) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] cxx-tree Hello example compilation problem on AIX 5.3 using XL C++ v7 Message-ID: <9ef90e310908281111g104e8027m3f1711bfa6af4b49@mail.gmail.com> Hi all, I've installed XSD 3.2.0 on AIX 5.3 w/ XL C++ v7.0 Xerces version is 3.0.1 I've tried to compile the tree Hello example with some issues. XSD binary runs just fine over hello.xsd and generates hello.cxx and hello.hpp without errors or warnings. Then, when I try to compile hello.cxx into hello.o using xlC, I get a LOT of errors, starting with this: "/usr/include/xsd/cxx/tree/exceptions.hxx", line 48.13: 1540-0121 (S) A template cannot have "C" linkage. I've googled and read a lot... and seems to be a headers issue with some "C" linkage zone not properly defined, but I can't find the root of the problem. Since this is not code I wrote and it's just one of the examples (a really simple one), I'm a little lost... can you help me? Thanks in advance. Juan From boris at codesynthesis.com Fri Aug 28 14:49:23 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] cxx-tree Hello example compilation problem on AIX 5.3 using XL C++ v7 In-Reply-To: <9ef90e310908281111g104e8027m3f1711bfa6af4b49@mail.gmail.com> References: <9ef90e310908281111g104e8027m3f1711bfa6af4b49@mail.gmail.com> Message-ID: Hi Juan, Juan Pablo Perez writes: > Then, when I try to compile hello.cxx into hello.o using xlC, I get a > LOT of errors, starting with this: I just tried to reproduce this but everything is working for me. My setup is: AIX 5.3 IBM XL C/C++ Enterprise Edition V7.0 Version: 07.00.0000.0013 I did the following: 1. Download xerces-c-3.0.1-powerpc-aix-xlc-7.0.tar.gz from Xerces-C++ website 2. Download xsd-3.2.0-powerpc-aix.tar.gz from Code Synthesis website 3. Unpack the two archives 4. cd xsd-3.2.0-powerpc-aix/examples/cxx/tree/hello 5. gmake CPPFLAGS=-I.../xerces-c-3.0.1-powerpc-aix-xlc-7.0/include LDFLAGS=-L.../xerces-c-3.0.1-powerpc-aix-xlc-7.0/lib Which version of XL C++ are you using? What command line did you use to build the hello example? Do you get the same error if you try the above? > "/usr/include/xsd/cxx/tree/exceptions.hxx", line 48.13: 1540-0121 (S) > A template cannot have "C" linkage. This looks like system/compiler header issue. Perhaps missing '}' after 'extern "C" {'. Boris From juanpablo.perez at gmail.com Fri Aug 28 14:52:22 2009 From: juanpablo.perez at gmail.com (Juan Pablo Perez) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] cxx-tree Hello example compilation problem on AIX 5.3 using XL C++ v7 In-Reply-To: References: <9ef90e310908281111g104e8027m3f1711bfa6af4b49@mail.gmail.com> Message-ID: <9ef90e310908281152g6b887c3es4ba22fac648d1a1f@mail.gmail.com> Boris, Thanks for the REALLY quick reply. I've done steps 1 to 4. I have xerces headers and libs and XSD headers copied over to /usr/include and /usr/lib (installed both) and the enviroment variables for includes and libs are ok. XL C++ is v7.0 I'm trying with: xlC -c hello.cxx Unfortunatly I can't try gmake since it's not available on this server. Regards. Juan On Fri, Aug 28, 2009 at 3:49 PM, Boris Kolpackov wrote: > Hi Juan, > > Juan Pablo Perez writes: > >> Then, when I try to compile hello.cxx into hello.o using xlC, I get a >> LOT of errors, starting with this: > > I just tried to reproduce this but everything is working for me. My > setup is: > > AIX 5.3 > IBM XL C/C++ Enterprise Edition V7.0 > Version: 07.00.0000.0013 > > I did the following: > > 1. Download xerces-c-3.0.1-powerpc-aix-xlc-7.0.tar.gz from Xerces-C++ website > 2. Download xsd-3.2.0-powerpc-aix.tar.gz from Code Synthesis website > 3. Unpack the two archives > 4. cd xsd-3.2.0-powerpc-aix/examples/cxx/tree/hello > 5. gmake CPPFLAGS=-I.../xerces-c-3.0.1-powerpc-aix-xlc-7.0/include LDFLAGS=-L.../xerces-c-3.0.1-powerpc-aix-xlc-7.0/lib > > Which version of XL C++ are you using? What command line did you use > to build the hello example? Do you get the same error if you try the > above? > > >> "/usr/include/xsd/cxx/tree/exceptions.hxx", line 48.13: 1540-0121 (S) >> A template cannot have "C" linkage. > > This looks like system/compiler header issue. Perhaps missing '}' > after 'extern "C" {'. > > Boris > From boris at codesynthesis.com Fri Aug 28 15:10:42 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] cxx-tree Hello example compilation problem on AIX 5.3 using XL C++ v7 In-Reply-To: <9ef90e310908281152g6b887c3es4ba22fac648d1a1f@mail.gmail.com> References: <9ef90e310908281111g104e8027m3f1711bfa6af4b49@mail.gmail.com> <9ef90e310908281152g6b887c3es4ba22fac648d1a1f@mail.gmail.com> Message-ID: Hi Juan, Juan Pablo Perez writes: > I'm trying with: > > xlC -c hello.cxx Can you try this (not that it should make any difference): xlC_r -qrtti -c hello.cxx If that still doesn't work, can you send me (off list) the result of preprocessing hello.cxx. That is, run: xlC_r -qrtti -E hello.cxx >hello.i And send me hello.i.gz. Also can you run xlC_r -qversion and send the output along? Thanks, Boris From bpringle at sympatico.ca Sun Aug 30 01:13:53 2009 From: bpringle at sympatico.ca (Bill Pringlemeir) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: (Boris Kolpackov's message of "Thu, 27 Aug 2009 18:24:25 +0200") References: <4A8B1C3A.2040406@boostpro.com> <87ab1ru5wb.fsf@sympatico.ca> Message-ID: <87eiqt51f2.fsf@sympatico.ca> On 27 Aug 2009, boris@codesynthesis.com wrote: > Bill Pringlemeir writes: >> 1. You are a North American wishing to write an open source TV capture >> application. > I am not sure I follow. Are you saying you don't want to generate > the serialization code (already possible)? Or that you would like the > serialization to a (relational) database instead of XML? Can you > elaborate a bit? I was thinking I would move the data to a DB by myself. Meaning that the serialization code for many classes is not that useful. But perhaps that is distracting from my main point. The schema has parts of a SOAP infra-structure with 'request' and 'response' messages. I meant that the service provider will never be the application. So I will only ever want my application to have 'getters' and parsing code for the 'response' and only 'setters' and serialization code for the request. There are other objects with-in a request/response that you would like to be generic. For this application that is not that extreme. However other SOAP services might have many entry points having full-blown objects for these ephemeral messages is not useful. It is useful for the re-useable data that might be inside of those messages. We tried to avoid different message types (classes) by using optionals and enumeration to denote the request type to reduce the code size of the messaging parts of schemas. Using attributes as opposed to elements is also *very* helpful with XSD if you have the latitude to code your own schema. However, when you use an off the shelf SOAP schema, this is not possible. I guess that some of the constructors are needed for the parser to construct the object. However, the application will never construct a response. XSD tends to generate multiple constructors which can be very expensive with elements. Especially if polymorphism, base-ctor, and auto_ptr variants are all in use. >> 2. You are part of a financial institution that is using the FpML >> schemas as a standard. Your application only deals with Mortgages. > Hm, looks like you and Alan Amano are after the same feature. I just > replied to his email with some ideas: I think parts of the versioning discussion are still relevant. For example, the 'default' on write does not have to be so complex as a function/method. You may always work in USD in the above example. When specifying a currency type in any message, it might be convenient to fill it in with USD. If you were to custom code this (from a DOM), you might throw in defaults for the currency type, etc and not have this in the object model. There seem to be many situations where industry/committee schemas are used, but only a subset of the information is needed for a particular application. A main point in all of these discussions being that we wish to produce and parse XML as defined by a primary schema, but wish to deal in an object model that is a subset. Perhaps the versioning issue is the inverse? Xpath/xslt/XQuery type rules to transform an industry schema to a working schema for XSD to process. I don't know what kind of rules would allow transformation so that the input/output documents were handled. It is fairly easy to ignore input xml (). Perhaps for the output, would be the appropriate translation. This isn't much more help than just editing the schema by hand. However, a well thought out set of transformations that could be applied programatically would lower the barrier for a novice and also allow for updates of the primary schema. The down side to that is a new language that could just as well confuse things. In many ways XSD has some transformation capabilities with the 'reg-ex' type options, but they are currently limited to the generated code as opposed to the schemas (with the exceptions of name spaces). Having used XSLT with docbook, I wouldn't wish it on anyone! Fwiw, Bill Pringlemeir. -- Anyone who trades liberty for security deserves neither liberty nor security - Benjamin Franklin From juanpablo.perez at gmail.com Mon Aug 31 08:53:15 2009 From: juanpablo.perez at gmail.com (Juan Pablo Perez) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] cxx-tree Hello example compilation problem on AIX 5.3 using XL C++ v7 In-Reply-To: References: <9ef90e310908281111g104e8027m3f1711bfa6af4b49@mail.gmail.com> <9ef90e310908281152g6b887c3es4ba22fac648d1a1f@mail.gmail.com> <9ef90e310908281212y311de7e0w96e7cba5f331dee1@mail.gmail.com> <9ef90e310908281323x34cf4ff3x7d75295425bfdf77@mail.gmail.com> Message-ID: <9ef90e310908310553l2221c224y2eac6db47328ad49@mail.gmail.com> Ok, Following up on this... Boris, you were 100% right! That was the problem :D Added those options and the example compiled as is. Thanks a lot Best Regards. Juan On Fri, Aug 28, 2009 at 6:41 PM, Boris Kolpackov wrote: > Hi Juan, > > Ok, I think I found something. XL C++ has the -qcinc option which > instructs the compiler to wrap #include into extern "C" {} blocks > for certain directories. I think this is exactly what's happening. > It seem that the /usr/include directory is treated this way by > default (this also happens on my box). For some reason it also > happens for files in /usr/vacpp/include in your case (this doesn't > happen on my box). Can you try to add the following command line > options and see if this helps: > > -qnocinc=/usr/include/xsd -qnocinc=/usr/include/xercesc -qnocinc=/usr/vacpp/include > > Boris > From boris at codesynthesis.com Mon Aug 31 09:35:38 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] cxx-tree Hello example compilation problem on AIX 5.3 using XL C++ v7 In-Reply-To: <9ef90e310908310553l2221c224y2eac6db47328ad49@mail.gmail.com> References: <9ef90e310908281111g104e8027m3f1711bfa6af4b49@mail.gmail.com> <9ef90e310908281152g6b887c3es4ba22fac648d1a1f@mail.gmail.com> <9ef90e310908281212y311de7e0w96e7cba5f331dee1@mail.gmail.com> <9ef90e310908281323x34cf4ff3x7d75295425bfdf77@mail.gmail.com> <9ef90e310908310553l2221c224y2eac6db47328ad49@mail.gmail.com> Message-ID: Hi Juan, Juan Pablo Perez writes: > Boris, you were 100% right! That was the problem :D > Added those options and the example compiled as is. Great, thanks for letting us know. I have added a wiki page with a note on this issue: http://wiki.codesynthesis.com/XSD/AIX_XLC Can you verify one last thing for me? Can you try to compile without the last -qnocinc option (-qnocinc=/usr/vacpp/include) and see if it still compiles cleanly? Boris From boris at codesynthesis.com Thu Aug 27 12:24:25 2009 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: <87ab1ru5wb.fsf@sympatico.ca> References: <4A8B1C3A.2040406@boostpro.com> <87ab1ru5wb.fsf@sympatico.ca> Message-ID: Hi Bill, Bill Pringlemeir writes: > 1. You are a North American wishing to write an open source TV capture > application. > > In order to do this you need schedule information. There is a SOAP > service provided by Tribune Media Services, > > http://xmlbeans.googlepages.com/ > - entry TMS XTVD - Tribune Media Services eXtensible TV Data > > It is highly unlikely that you will be generating the program > schedules in your program. At the SOAP level, you wish to > generate/write the REQUEST messages and read the RESPONSE. You might > wish to use the generate the RESPONSE schedule by reading from a > persisted file. However, this would generally be a subset of the > returned data and you might choose another data storage mechanism such > as a database. This would be a benefit if you wanted to query for any > recent 'movies' that will be airing and automate the capture of these > programs via a TV tuner card. At first if might seem like XML > persistence is a good thing, but it would probably only be useful for > debugging. I am not sure I follow. Are you saying you don't want to generate the serialization code (already possible)? Or that you would like the serialization to a (relational) database instead of XML? Can you elaborate a bit? > 2. You are part of a financial institution that is using the FpML > schemas as a standard. Your application only deals with Mortgages. > Derivative, currency exchange, and many other data modeled in the FpML > are not in the scope of your application. You want to use the FpML > schemas (and any future updates) and restrict the amount of generated > code needed by your application's domain. Hm, looks like you and Alan Amano are after the same feature. I just replied to his email with some ideas: http://www.codesynthesis.com/pipermail/xsd-users/2009-August/002434.html If you have any thoughts on this, please let us know. Boris From juanpablo.perez at gmail.com Mon Aug 31 09:34:29 2009 From: juanpablo.perez at gmail.com (Juan Pablo Perez) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] cxx-tree Hello example compilation problem on AIX 5.3 using XL C++ v7 In-Reply-To: References: <9ef90e310908281111g104e8027m3f1711bfa6af4b49@mail.gmail.com> <9ef90e310908281152g6b887c3es4ba22fac648d1a1f@mail.gmail.com> <9ef90e310908281212y311de7e0w96e7cba5f331dee1@mail.gmail.com> <9ef90e310908281323x34cf4ff3x7d75295425bfdf77@mail.gmail.com> <9ef90e310908310553l2221c224y2eac6db47328ad49@mail.gmail.com> Message-ID: <9ef90e310908310634j5b8bd7fl6b73495f62fe985f@mail.gmail.com> Tested! Works as well. On Mon, Aug 31, 2009 at 10:35 AM, Boris Kolpackov wrote: > Hi Juan, > > Juan Pablo Perez writes: > >> Boris, you were 100% right! That was the problem :D >> Added those options and the example compiled as is. > > Great, thanks for letting us know. I have added a wiki page with > a note on this issue: > > http://wiki.codesynthesis.com/XSD/AIX_XLC > > Can you verify one last thing for me? Can you try to compile without > the last -qnocinc option (-qnocinc=/usr/vacpp/include) and see if it > still compiles cleanly? > > Boris > From eric at boostpro.com Mon Aug 31 12:45:07 2009 From: eric at boostpro.com (Eric Niebler) Date: Sun Oct 11 15:34:10 2009 Subject: [xsd-users] import, include, namespaces, restriction and schema versioning In-Reply-To: References: <4A8B1C3A.2040406@boostpro.com> <4A8C2521.5060102@boostpro.com> <4A8EC2DA.9060401@boostpro.com> Message-ID: <4A9BFE13.1050201@boostpro.com> Boris Kolpackov wrote: > Eric Niebler writes: >> Ideally, we would mark up the xsd and have CodeSynth fill in the >> defaults for us, but that's something else entirely. > > Yes, it seems there are two general ways in which the user may want > this to be handled: > > (1) Fill in the defaults for missing elements. Yes. > (2) Notify the user somehow about missing elements, presumably, > during parsing. I wasn't thinking of that. By "user" you mean me, the user of CodeSynth, right? And by "notify" you mean invoke a registered callback. Huh, could be useful, but is beyond my basic needs. >>> On the surface validation during serialization often seem like a good >>> idea. However, once you start thinking about what to do in case of >>> an error, its usefulness becomes questionable, except, maybe, for >>> debugging >> I disagree, and this is where I see a situation where CodeSynth can >> provide a value-add over plain mapping and validation. Imagine that we >> have optional elements with special CodeSynth markup for what the value >> the element should take when it's missing. If CodeSynth did a schema >> validation pass on serialization, this is where the value for missing >> elements could be filled in automatically. > > Wouldn't this be more naturally done during parsing so that the > application developer doesn't have to worry about missing elements? More naturally? Why? My intention is that the resulting XML produced by our tools is always in the most up-to-date form and doesn't need to be read back in in order to be patched up. But the fix-up-on-write thought is beyond my basic needs. For my scenario, what I'd like is: 1) On write, assert when a writeRequired element is missing. 2) On read, when a writeRequired element is missing, fill in a default value. >> That is, CodeSynth can proactively correct simple schema violations >> with a little guidance from the xsd author. > > Such things can be implemented quite easily either during parsing > or during serialization. And I agree, they can be useful. It is the > full XML Schema validation that, IMO, is not very useful since it > is not clear what to do when there is an error. OK, understood. >> If the issue is merely one of error detection and reporting, then yes I >> agree that validation on serialization doesn't make much sense except >> for debugging. But if you're willing to consider error correction, then >> it does make sense. > > I think this will only work for automatic error correction, as above. > In case the user intervention is required, it is not clear how to > supply the error information (e.g., position, description, etc.) that > can be usable for anything other than an error message. Right. It's ok, I probably don't need this. >>> Full-blown XSLT will also work and is probably simpler and quicker >>> to implement (especially if you have multiple schema files connected >>> via include/import). But the DOM approach is tidier since you don't >>> need to carry two sets of schemas with your application. >> Confused. If I use XSLT to process the schema, then I can still ship >> only 1 set of schema, right? > > If you have a full-blown XSLT processor inside your application, then > yes, you can ;-). We may need it anyway. >> So maybe xse:writeRequired isn't quite what I want. I'd like a way to >> provide default values for optional elements in a way that they are >> filled in automatically on read; and on write, either (a) fill them in >> automatically, or (b) flag their absence as an error. > > If they are filled in on read, why do you also need this on write? Because I wouldn't want my tool to produce XML that is missing writeRequired elements. I may decide to publish the current schema (minus CodeSynth extensions like xse:writeRequired or xse:refType) to allow 3rd parties utilities to consume the XML my tool produces. But asserting on the absence of a writeRequired elements achieves this end just as well. > Perhaps the application can create the object model and not provide > new elements (because you generate default c-tors)? Is that the use > case you had in mind? No, see above. >>> But I feel that it is only a part of the solution. The user of the >>> mapping still has to detect the missing elements and provide some >>> default values manually. >> We're already doing that in our tool, so that's ok. > > That sounds like quite a tedious task. I wonder what API can we use > to notify the application about missing elements and request the > default values..? We could use DOM to return the default values > but returning an object model instance would be more convenient. > We could also pass the reference to the containing object model > node in case the handler needs it. I was hoping it could be as simple as something like this: ... ... name="SomeOtherType"> > One wild idea is to specify the handler function in the schema > file, for example: > > > > > xse:defaultHandler="person_age_value"/> > > > > The signature of the person_age_value() function would be: > > int person_age_value (person& p); > > Then we can do something like: > > std::map ages = ... > > int person_age_value (person& p) > { > return ages[p.name ()]; > } > > We would then have the companion attribute, xse:defaultValue, which > is used to specify the default value in some form. Maybe an XPath to > an element in the "default values file", for example: > > > > > xse:defaultValue="/values/person/age"/> > > > > The remaining question is how the values from this file are > accessed to construct object model nodes. The cleanest approach > would be to somehow embed these values directly into the generated > code. Ideally, we would have a static object model instance that > represents the default value. But it is not clear how to initialize > it (Xerces-C++ is not usable during static initialization). Whoa, you're going WAY beyond anything I had in mind. :-) Pretty cool, but feels over-engineered for my purposes. Could it really be accomplished? -- Eric Niebler BoostPro Computing http://www.boostpro.com