From boris at codesynthesis.com Sun Feb 2 22:32:43 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Feb 2 22:30:38 2014 Subject: [xsd-users] inline namespaces In-Reply-To: <68F9DD51BF61E943BFA45D744048A7A7319D719980@nbp-ex01.proteus-technologies.com> References: <68F9DD51BF61E943BFA45D744048A7A7319D719980@nbp-ex01.proteus-technologies.com> Message-ID: Hi Ray, Ray Lischner writes: > Perhaps the --namespace-regex option can use a magic character in > the C++ namespace to mean "make this inline" I don't see why we can't support this. One tricky question is how do we refer to names in inline namespaces from other generated files? For example, let's say we mapped XML namespace foo to C++ namespace foo::v1 (v1 is inline). I think for all the code that is generated for XML namespace foo we should use the long form (foo::v1::object). However, if we have another schema with XML namespace bar that imports foo, we could use foo::v1::object or foo::object. And I think both variants could be desirable in different situations. In fact, one could come up with a scenario where both approaches are used at the same time depending on the importing namespace. We could achieve the short version (foo::object) by simply using a different namespace mapping (e.g., foo=foo instead of foo=foo::v1) when compiling other schemas. While a bit inconvenient (have to specify two mappings for the same namespace), it will work for the separate compilation (file-per-schema) mode. It won't work for the file-per-type mode where all the schemas are compiled at once. Though in this case I don't think it makes much sense to use the short variant, since this mode is normally used in the "intertwined blob" kind of schemas where "schema A uses schema B" statements don't apply. So perhaps we should keep it simple and let the user use a different mapping for different schemas, if desired. This is already quite an "advanced" area so a bit of extra inconvenience won't be too bad. Plus it will keep the XSD compiler really simple (we won't have to distinguish between different namespaces where the symbol is referenced and eliminating inline namespace in certain cases). What do you think? Boris From ebrabant at ramlabs.com Wed Feb 5 21:36:49 2014 From: ebrabant at ramlabs.com (Edward Brabant) Date: Wed Feb 5 21:36:58 2014 Subject: [xsd-users] Attribute named "type" generates post_xxx() skeleton code with invalid return data type Message-ID: <264F8608DB6967428964D02E0D2D6948C7CC6B@ORD2MBX04B.mex05.mlsrvr.com> The XSD definition of the element iwhich contains the attribute named "type": The corresponding entry in the XSD map file is: type std::string; This causes XSD version 3.3.0 (on CentOS 6.3) to generate skeleton code with the signature: virtual void post_type (); The return type should be std::string, per the entry in the XSD map file above, but is void instead. Note that if I change the attribute name to something other than "type", e.g. "types", the skeleton code is generated correctly, i.e. the return type is std::string. Regards, Ed Brabant Contractor JTRS Test Application (JTAP) team, JTRS Test and Evaluation Lab (JTEL), Joint Tactical Networking Center (JTNC) Technical Directorate (TD), Space and Naval Warfare (SPAWAR) Systems Command (SSC) Pacific From boris at codesynthesis.com Wed Feb 5 23:50:52 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Feb 5 23:48:47 2014 Subject: [xsd-users] Attribute named "type" generates post_xxx() skeleton code with invalid return data type In-Reply-To: <264F8608DB6967428964D02E0D2D6948C7CC6B@ORD2MBX04B.mex05.mlsrvr.com> References: <264F8608DB6967428964D02E0D2D6948C7CC6B@ORD2MBX04B.mex05.mlsrvr.com> Message-ID: Hi Edward, Edward Brabant writes: > > > > > [...] > > > [...] > > The corresponding entry in the XSD map file is: > > type std::string; > > This causes XSD version 3.3.0 (on CentOS 6.3) to generate skeleton > code with the signature: > > virtual void post_type (); > > Note that if I change the attribute name to something other than > "type", e.g. "types", the skeleton code is generated correctly, > i.e. the return type is std::string. The attribute's type is anonymous which XSD automatically morphs to a named type. What most likely happens is you have another anonymous type before it that got assigned the 'type' name. And to avoid name clashes, the next 'type' types will be named 'type1', 'type2', etc. The tricky part is that there is no easy way to see what names XSD assignes to anonymous types. Plus, even if you could, that would not be very stable, since the names can change when you change the schema. So the best option here is to specify a custom regex for anonymous types that includes both the element/attribute name and the outer type name (so you will end up with 'domainfinder_type'). This name should be stable and should not conflict with any other names. For more information on specifying such a regex, see the --anonymous-regex option. With this method you can also see the assigned names with the --anonymous-regex-trace option. Boris From ebrabant at ramlabs.com Thu Feb 6 11:36:27 2014 From: ebrabant at ramlabs.com (Edward Brabant) Date: Thu Feb 6 11:36:35 2014 Subject: [xsd-users] Attribute named "type" generates post_xxx() skeleton code with invalid return data type In-Reply-To: References: <264F8608DB6967428964D02E0D2D6948C7CC6B@ORD2MBX04B.mex05.mlsrvr.com>, Message-ID: <264F8608DB6967428964D02E0D2D6948C7CEAA@ORD2MBX04B.mex05.mlsrvr.com> Boris, Thanks for the tip. Regards, Ed ________________________________________ From: Boris Kolpackov [boris@codesynthesis.com] Sent: Wednesday, February 05, 2014 8:50 PM To: Edward Brabant Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] Attribute named "type" generates post_xxx() skeleton code with invalid return data type Hi Edward, Edward Brabant writes: > > > > > [...] > > > [...] > > The corresponding entry in the XSD map file is: > > type std::string; > > This causes XSD version 3.3.0 (on CentOS 6.3) to generate skeleton > code with the signature: > > virtual void post_type (); > > Note that if I change the attribute name to something other than > "type", e.g. "types", the skeleton code is generated correctly, > i.e. the return type is std::string. The attribute's type is anonymous which XSD automatically morphs to a named type. What most likely happens is you have another anonymous type before it that got assigned the 'type' name. And to avoid name clashes, the next 'type' types will be named 'type1', 'type2', etc. The tricky part is that there is no easy way to see what names XSD assignes to anonymous types. Plus, even if you could, that would not be very stable, since the names can change when you change the schema. So the best option here is to specify a custom regex for anonymous types that includes both the element/attribute name and the outer type name (so you will end up with 'domainfinder_type'). This name should be stable and should not conflict with any other names. For more information on specifying such a regex, see the --anonymous-regex option. With this method you can also see the assigned names with the --anonymous-regex-trace option. Boris From ebrabant at ramlabs.com Thu Feb 6 15:13:15 2014 From: ebrabant at ramlabs.com (Edward Brabant) Date: Thu Feb 6 15:13:57 2014 Subject: [xsd-users] Attribute named "type" generates post_xxx() skeleton code with invalid return data type In-Reply-To: <264F8608DB6967428964D02E0D2D6948C7CEAA@ORD2MBX04B.mex05.mlsrvr.com> References: <264F8608DB6967428964D02E0D2D6948C7CC6B@ORD2MBX04B.mex05.mlsrvr.com>, , <264F8608DB6967428964D02E0D2D6948C7CEAA@ORD2MBX04B.mex05.mlsrvr.com> Message-ID: <264F8608DB6967428964D02E0D2D6948C7E1E3@ORD2MBX04B.mex05.mlsrvr.com> After several unsuccessful tries using the --anonymous-regex option, I avoided the problem altogether by changing element definition in the XSD file so that the 'type' attribute is no longer an anonymous type. This required changing the 'type' attribute entry in the map file. Original 'domainfinder' element and 'type' attribute definition in the XSD file: Revised 'domainfinder' element and 'type' attribute definition in the XSD file: Original 'type' attribute entry in the map file: type "std::string"; Revised 'type' attribute entry in the map file: type domainfinder_elem_type_attr_datatype "std::string"; Regards, Ed Brabant Contractor JTRS Test Application (JTAP) team, JTRS Test and Evaluation Lab (JTEL), Joint Tactical Networking Center (JTNC) Technical Directorate (TD), Space and Naval Warfare (SPAWAR) Systems Command (SSC) Pacific ________________________________________ From: Boris Kolpackov [boris@codesynthesis.com] Sent: Wednesday, February 05, 2014 8:50 PM To: Edward Brabant Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] Attribute named "type" generates post_xxx() skeleton code with invalid return data type Hi Edward, Edward Brabant writes: > > > > > [...] > > > [...] > > The corresponding entry in the XSD map file is: > > type std::string; > > This causes XSD version 3.3.0 (on CentOS 6.3) to generate skeleton > code with the signature: > > virtual void post_type (); > > Note that if I change the attribute name to something other than > "type", e.g. "types", the skeleton code is generated correctly, > i.e. the return type is std::string. The attribute's type is anonymous which XSD automatically morphs to a named type. What most likely happens is you have another anonymous type before it that got assigned the 'type' name. And to avoid name clashes, the next 'type' types will be named 'type1', 'type2', etc. The tricky part is that there is no easy way to see what names XSD assignes to anonymous types. Plus, even if you could, that would not be very stable, since the names can change when you change the schema. So the best option here is to specify a custom regex for anonymous types that includes both the element/attribute name and the outer type name (so you will end up with 'domainfinder_type'). This name should be stable and should not conflict with any other names. For more information on specifying such a regex, see the --anonymous-regex option. With this method you can also see the assigned names with the --anonymous-regex-trace option. Boris From pavel.fiala at cern.ch Fri Feb 7 12:05:04 2014 From: pavel.fiala at cern.ch (Pavel Fiala) Date: Fri Feb 7 12:49:39 2014 Subject: [xsd-users] Tree streaming serialization Message-ID: <52F51240.6030509@cern.ch> Hello, I'm trying to use XSD/Tree streaming serialization of a large document but I'm facing a problem - memory consumption. It's possible to reproduce it with the bundled example "xsd/examples/cxx/tree/streaming". I've changed the number of generated position elements to 50000 and it takes ~9 MB of memory, see attached screenshot. I'm using xsd 3.3.0 and XercesC 3.1.1 on 64bit linux distribution. It seems that the problem is how DOMElements are created as they are probably not released. The same behaviour is when I try to call DOMElement::release() in serializer::next() just after serializing each element. Any idea how to fix the memory usage and release each element after serializing it? I have a problem with my own schema and XML serialization as it takes up to 2 GB of memory. Parsing using the streaming example works fine with only ~5 MB of memory. Cheers, Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: position-50000.png Type: image/png Size: 38566 bytes Desc: not available Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20140207/0be3ee6c/position-50000-0001.png From raviraja at qti.qualcomm.com Fri Feb 7 16:55:51 2014 From: raviraja at qti.qualcomm.com (Rangarajan, Ravi_Sowmian) Date: Fri Feb 7 23:31:35 2014 Subject: [xsd-users] Q: schemaLocation in schema file Message-ID: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> Hi Code synthesis representative. Pls. let me know if query below requires a support license#, in which case I will try to get that detail from our team. Currently, my schema is looking for a namespace component and a location component in schemaLocation attribute. If location field is missing, code synthesis throws an exception. Since location and/or namespace fields in schemaLocation are optional, then I am wondering what would be the best way to specify schemaLocation in the schema template such that code synthesis parser would handle the three scenarios below properly? 1. schemaLocation field itself is missing. 2. schemaLocation is present and it has namespace field, location is missing. 3. schemaLocation is present and so are namespace & location fields. Just to highlight with example. If the part in red is missing from MPD files, then code synthesis throws exception. xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 MPD.xsd" thanks Ravi From boris at codesynthesis.com Sun Feb 9 05:12:59 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Feb 9 05:10:53 2014 Subject: [xsd-users] Q: schemaLocation in schema file In-Reply-To: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> Message-ID: Hi Ravi, Rangarajan, Ravi_Sowmian writes: > Currently, my schema is looking for a namespace component and a location > component in schemaLocation attribute. If location field is missing, code > synthesis throws an exception. Since location and/or namespace fields in > schemaLocation are optional, then I am wondering what would be the best way > to specify schemaLocation in the schema template such that code synthesis > parser would handle the three scenarios below properly? Generally, it is not a good idea to rely on the schemaLocation attribute in production applications. It is better to ignore it and specify the location/schema from within the application. For more information on various ways to achieve this, refer to Section 5.1, "XML Schema Validation and Searching" in the C++/Tree Mapping Getting Started Guide: http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#5.1 Boris From boris at codesynthesis.com Sun Feb 9 05:28:29 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Feb 9 05:26:23 2014 Subject: [xsd-users] Tree streaming serialization In-Reply-To: <52F51240.6030509@cern.ch> References: <52F51240.6030509@cern.ch> Message-ID: Hi Pavel, Pavel Fiala writes: > I'm trying to use XSD/Tree streaming serialization of a large > document but I'm facing a problem - memory consumption. It's > possible to reproduce it with the bundled example > "xsd/examples/cxx/tree/streaming". I've changed the number of > generated position elements to 50000 and it takes ~9 MB of memory, > see attached screenshot. What happens if you try to serialize, say, 1M of position elements (maybe send them to /dev/null)? The 9MB is not really large enough to indicate constantly increaing memory consumption (it could just be cached; see below). > It seems that the problem is how DOMElements are created as they are > probably not released. What happens with elements is they are returned to the document's cache and then (potentially) reused. BTW, for the next release of XSD we have improved and generalized streaming parser and serializer significantly. In particular, you no longer need to modify them for your XML vocabulary. They are completely generic and can do streaming parsing/serialization any number of levels deep. Let me know if you would like the updated 'streaming' example. Boris From pavel.fiala at cern.ch Mon Feb 10 08:02:56 2014 From: pavel.fiala at cern.ch (Pavel Fiala) Date: Mon Feb 10 08:15:01 2014 Subject: [xsd-users] Tree streaming serialization In-Reply-To: References: <52F51240.6030509@cern.ch> Message-ID: <52F8CE00.2070400@cern.ch> Hi Boris, I executed the streaming example with 1M and 10M position elements. The output file size is ~38 MB for 1M position elements and ~387 MB for 10M position elements, the memory consumption for 10M is basically 10 times 1M memory, plots are attached. If you have anything new about serialization, I would be interested in it as the streaming serialization is the main issue with my own schema. Thank you, Pavel On 9.2.2014 11:28, Boris Kolpackov wrote: > Hi Pavel, > > Pavel Fiala writes: > >> I'm trying to use XSD/Tree streaming serialization of a large >> document but I'm facing a problem - memory consumption. It's >> possible to reproduce it with the bundled example >> "xsd/examples/cxx/tree/streaming". I've changed the number of >> generated position elements to 50000 and it takes ~9 MB of memory, >> see attached screenshot. > What happens if you try to serialize, say, 1M of position elements > (maybe send them to /dev/null)? The 9MB is not really large enough > to indicate constantly increaing memory consumption (it could just > be cached; see below). > > >> It seems that the problem is how DOMElements are created as they are >> probably not released. > What happens with elements is they are returned to the document's > cache and then (potentially) reused. > > BTW, for the next release of XSD we have improved and generalized > streaming parser and serializer significantly. In particular, you > no longer need to modify them for your XML vocabulary. They are > completely generic and can do streaming parsing/serialization > any number of levels deep. Let me know if you would like the > updated 'streaming' example. > > Boris -------------- next part -------------- A non-text attachment was scrubbed... Name: position-1M.png Type: image/png Size: 35118 bytes Desc: not available Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20140210/3ad96685/position-1M-0001.png -------------- next part -------------- A non-text attachment was scrubbed... Name: position-10M.png Type: image/png Size: 34194 bytes Desc: not available Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20140210/3ad96685/position-10M-0001.png From raviraja at qti.qualcomm.com Mon Feb 10 14:33:15 2014 From: raviraja at qti.qualcomm.com (Rangarajan, Ravi_Sowmian) Date: Mon Feb 10 23:01:18 2014 Subject: [xsd-users] Q: schemaLocation in schema file In-Reply-To: References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> Message-ID: <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> Hi Boris, Sorry for not being clear the first time. We do specify/pass name/location of schema to code synthesis parser via the code snippet below. What I was asking is that code synthesis throws an exception (and fails parsing) if an XML instance file doesn't contain namespace as well as location components in schemaLocation field (as shown in green below). I tried to remove schemaLocation field from my schema file or just specify namespace (missing out the location part) as in red below - I cannot even compile the XSD files without the entry as in green below. My question was how do I tell code synthesis to ignore if schemaLocation field is completely missing from XML instance file or it could be present, but location part could be missing? (I mean any way other than telling code synthesis not to validate schema). Thank you Ravi Way I pass schema_location to code synthesis: ======================================= xml_schema::properties schemaProperties; schemaProperties.schema_location(...) what is in my schema (first few lines): ================================= xlink.xsd and xml.xsd which it in turn imports are present in the same directory. What keeps code synthesis happy: ============================= > writes: > Currently, my schema is looking for a namespace component and a > location component in schemaLocation attribute. If location field is > missing, code synthesis throws an exception. Since location and/or > namespace fields in schemaLocation are optional, then I am wondering > what would be the best way to specify schemaLocation in the schema > template such that code synthesis parser would handle the three scenarios below properly? Generally, it is not a good idea to rely on the schemaLocation attribute in production applications. It is better to ignore it and specify the location/schema from within the application. For more information on various ways to achieve this, refer to Section 5.1, "XML Schema Validation and Searching" in the C++/Tree Mapping Getting Started Guide: http://www.codesynthesis.com/projects/xsd/documentation/cxx/tree/guide/#5.1 Boris From boris at codesynthesis.com Mon Feb 10 23:17:38 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Feb 10 23:15:34 2014 Subject: [xsd-users] Q: schemaLocation in schema file In-Reply-To: <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> Message-ID: Hi Ravi, Rangarajan, Ravi_Sowmian writes: > My question was how do I tell code synthesis to ignore if schemaLocation > field is completely missing from XML instance file or it could be present, > but location part could be missing? (I mean any way other than telling > code synthesis not to validate schema). > > xml_schema::properties schemaProperties; > > schemaProperties.schema_location(...) If you specified the schema_location() correctly (full path starting with file://, as described in the manual), then the schemaLocation attribute in the XML document should be completely ignored. Is that not the behavior you are getting? Boris From raviraja at qti.qualcomm.com Tue Feb 11 01:30:38 2014 From: raviraja at qti.qualcomm.com (Rangarajan, Ravi_Sowmian) Date: Tue Feb 11 01:32:11 2014 Subject: [xsd-users] Q: schemaLocation in schema file In-Reply-To: References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> Message-ID: <698646F0D0104244B95E527D289651D1122AEAA2@NASANEXD02B.na.qualcomm.com> Yes Boris, we had specified file://filepath/schema_name.xsd and namespace string in schema_location() API call - where filepath is in the local directory of the server where parsing of MPD happens. Path starts like /opt/.... which means file:///opt/.../schema_name.xsd It doesn't ignore schemaLocation with this specification. Should the third / be removed or what else can be wrong that makes code synthesis not ignore schemaLocation? Thanks again Ravi P.S: I tired dont_validate option and parser didn't abort as was happening earlier - only drawback I see with this option is that validation is totally disabled instead of relaxing specifications for schemaLocation alone. -----Original Message----- From: Boris Kolpackov [mailto:boris@codesynthesis.com] Sent: Monday, February 10, 2014 8:18 PM To: Rangarajan, Ravi_Sowmian Cc: xsde-users@codesynthesis.com; xsd-users@codesynthesis.com Subject: Re: [xsd-users] Q: schemaLocation in schema file Hi Ravi, Rangarajan, Ravi_Sowmian writes: > My question was how do I tell code synthesis to ignore if > schemaLocation field is completely missing from XML instance file or > it could be present, but location part could be missing? (I mean any > way other than telling code synthesis not to validate schema). > > xml_schema::properties schemaProperties; > > schemaProperties.schema_location(...) If you specified the schema_location() correctly (full path starting with file://, as described in the manual), then the schemaLocation attribute in the XML document should be completely ignored. Is that not the behavior you are getting? Boris From raviraja at qti.qualcomm.com Tue Feb 11 01:48:28 2014 From: raviraja at qti.qualcomm.com (Rangarajan, Ravi_Sowmian) Date: Tue Feb 11 01:54:55 2014 Subject: [xsd-users] Q: schemaLocation in schema file References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> Message-ID: <698646F0D0104244B95E527D289651D1122AEAC1@NASANEXD02B.na.qualcomm.com> Just wanted to add that I am not setting no_namespace_schema_location field of schemaProperties, I am only setting schema_location of schemaProperties. Thanks Ravi -----Original Message----- From: Rangarajan, Ravi_Sowmian Sent: Monday, February 10, 2014 10:31 PM To: xsde-users@codesynthesis.com Cc: xsd-users@codesynthesis.com Subject: RE: [xsd-users] Q: schemaLocation in schema file Yes Boris, we had specified file://filepath/schema_name.xsd and namespace string in schema_location() API call - where filepath is in the local directory of the server where parsing of MPD happens. Path starts like /opt/.... which means file:///opt/.../schema_name.xsd It doesn't ignore schemaLocation with this specification. Should the third / be removed or what else can be wrong that makes code synthesis not ignore schemaLocation? Thanks again Ravi P.S: I tired dont_validate option and parser didn't abort as was happening earlier - only drawback I see with this option is that validation is totally disabled instead of relaxing specifications for schemaLocation alone. -----Original Message----- From: Boris Kolpackov [mailto:boris@codesynthesis.com] Sent: Monday, February 10, 2014 8:18 PM To: Rangarajan, Ravi_Sowmian Cc: xsde-users@codesynthesis.com; xsd-users@codesynthesis.com Subject: Re: [xsd-users] Q: schemaLocation in schema file Hi Ravi, Rangarajan, Ravi_Sowmian writes: > My question was how do I tell code synthesis to ignore if > schemaLocation field is completely missing from XML instance file or > it could be present, but location part could be missing? (I mean any > way other than telling code synthesis not to validate schema). > > xml_schema::properties schemaProperties; > > schemaProperties.schema_location(...) If you specified the schema_location() correctly (full path starting with file://, as described in the manual), then the schemaLocation attribute in the XML document should be completely ignored. Is that not the behavior you are getting? Boris From boris at codesynthesis.com Tue Feb 11 02:02:44 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Feb 11 02:00:40 2014 Subject: [xsd-users] Tree streaming serialization In-Reply-To: <52F8CE00.2070400@cern.ch> References: <52F51240.6030509@cern.ch> <52F8CE00.2070400@cern.ch> Message-ID: Hi Pavel, Pavel Fiala writes: > I executed the streaming example with 1M and 10M position elements. > The output file size is ~38 MB for 1M position elements and ~387 MB > for 10M position elements, the memory consumption for 10M is > basically 10 times 1M memory, plots are attached. Thanks for testing this. I went through the Xerces-C++ DOM recycling code and it should re-use released elements. I don't see anything broken with that code off the top of my head. Can you try the following: in serializer.cxx, in the create() functions (two of them), right at the beginning add this line: doc_.reset (dom_impl_.createDocument ()); This will reset the document for every node created which should free all the memory allocated by the document. Can you then check if this makes any difference for the memory consumption? Boris From pavel.fiala at cern.ch Tue Feb 11 04:02:17 2014 From: pavel.fiala at cern.ch (Pavel Fiala) Date: Tue Feb 11 10:01:28 2014 Subject: [xsd-users] Tree streaming serialization In-Reply-To: References: <52F51240.6030509@cern.ch> <52F8CE00.2070400@cern.ch> Message-ID: <52F9E719.9040508@cern.ch> Hi Boris, On 11.2.2014 08:02, Boris Kolpackov wrote: > Can you try the following: in serializer.cxx, in the create() > functions (two of them), right at the beginning add this line: > > doc_.reset (dom_impl_.createDocument ()); this works, memory goes down under 1MB. I will try a simple example using Xerces only to create and release 1M elements and probably ask Xerces team directly if the behaviour stays the same. Thanks, Pavel -------------- next part -------------- A non-text attachment was scrubbed... Name: position-1M-reset.png Type: image/png Size: 36183 bytes Desc: not available Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20140211/846881c4/position-1M-reset-0001.png From boris at codesynthesis.com Tue Feb 11 23:46:44 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Tue Feb 11 23:44:39 2014 Subject: [xsd-users] Q: schemaLocation in schema file In-Reply-To: <698646F0D0104244B95E527D289651D1122AEAA2@NASANEXD02B.na.qualcomm.com> References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AEAA2@NASANEXD02B.na.qualcomm.com> Message-ID: Hi Ravi, Here is what I did: 1. I took the 'library' example from the XSD distribution and modified the library.xml file so that the schemaLocation attribute contains library1.xsd instead of library.xsd (file that doesn't exist). I then run the example and verified that I get the errors. 2. I then added: #include // getcwd To the top of driver.cxx and the following code fragment at the beginning of main() (just after using namespace library;): char cwd[1024]; getcwd (cwd, 1024); xml_schema::properties props; props.schema_location ( "http://www.codesynthesis.com/library", "file:///" + std::string (cwd) + "/library.xsd"); cerr << "schema location: " << props.schema_location () << endl; I also changed the parsing function call that reads like this: std::auto_ptr c (catalog_ (argv[1])); To instead read like this: std::auto_ptr c (catalog_ (argv[1], 0, props)); That is, I am passing the properties with the schema location to the parsing function (perhaps you forgot to this step)? 3. I then re-built the example and ran it on the modified library.xml. The example now runs without any problems and the schema location printed, in my case, as: schema location: http://www.codesynthesis.com/library file:////home/boris/work/xsd/xsd-default/examples/cxx/tree/test/library.xsd 4. I then modified the schemaLocation attribue to only include the namespace, without the schema path. This results in an error that says: library.xml:13:65 error: schemaLocation does not contain namespace- location pairs So this is invalid per the XML Schema specification. Can you try these steps and see if you get the same results? If you do get the same results, then check what you are doing different in your code. Boris From Reiner.Rohlfs at unige.ch Wed Feb 12 09:10:12 2014 From: Reiner.Rohlfs at unige.ch (Reiner Rohlfs) Date: Wed Feb 12 09:13:22 2014 Subject: [xsd-users] XSD generated code does not compile with g++ 4.8 Message-ID: Hello, I generate C++ code of the attached xml schema like this: xsd cxx-tree fits_data_model_schema.xsd and try to compile it with g++ 4.8.2: g++ -c fits_data_model_schema.cxx and get following error from the compiler: In file included from /usr/local/include/xsd/cxx/zc-istream.hxx:215:0, from /usr/local/include/xsd/cxx/tree/parsing/date-time.txx:7, from /usr/local/include/xsd/cxx/tree/parsing.hxx:10, from fits_data_model_schema.hxx:68, from fits_data_model_schema.cxx:41: /usr/local/include/xsd/cxx/zc-istream.txx: In instantiation of ?void xsd::cxx::zc_streambuf::init() [with C = char]?: /usr/local/include/xsd/cxx/zc-istream.txx:17:13: required from ?xsd::cxx::zc_streambuf::zc_streambuf(const xsd::cxx::ro_string&) [with C = char]? /usr/local/include/xsd/cxx/zc-istream.txx:64:20: required from ?xsd::cxx::zc_istream_base::zc_istream_base(const xsd::cxx::ro_string&) [with C = char]? /usr/local/include/xsd/cxx/zc-istream.txx:82:45: required from ?xsd::cxx::zc_istream::zc_istream(const xsd::cxx::ro_string&) [with C = char]? /usr/local/include/xsd/cxx/tree/parsing/unsigned-long.hxx:69:30: required from ?static long long unsigned int xsd::cxx::tree::traits::create(const std::basic_string<_CharT>&, const xercesc_3_1::DOMElement*, xsd::cxx::tree::flags, xsd::cxx::tree::container*) [with C = char; xsd::cxx::tree::container = xsd::cxx::tree::_type]? /usr/local/include/xsd/cxx/tree/parsing/unsigned-long.hxx:51:66: required from ?static long long unsigned int xsd::cxx::tree::traits::create(const xercesc_3_1::DOMAttr&, xsd::cxx::tree::flags, xsd::cxx::tree::container*) [with C = char; xsd::cxx::tree::container = xsd::cxx::tree::_type]? fits_data_model_schema.cxx:210:65: required from here /usr/local/include/xsd/cxx/zc-istream.txx:35:20: error: ?setg? was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] setg (b, b, e); ^ /usr/local/include/xsd/cxx/zc-istream.txx:35:20: note: declarations in dependent base ?std::basic_streambuf? are not found by unqualified lookup /usr/local/include/xsd/cxx/zc-istream.txx:35:20: note: use ?this->setg? instead To compile the same file with g++ version 4.2.1 is working fine. I don't know if this a compiler error or if an update of XSD is necessary. Any help would be very much appreciated. Bye the way, changing the line to works fine with both versions of the compiler. Here some information about my system: MAC OS version 10.8.3 XSD version 3.3.0 , downloaded 09. 12. 2013 > g++ --version g++ (GCC) 4.8.2 Copyright (C) 2013 Free Software Foundation, Inc. Thank you for your help. Best Regards Reiner Rohlfs -------------- next part -------------- A non-text attachment was scrubbed... Name: fits_data_model_schema.xsd Type: application/octet-stream Size: 470 bytes Desc: fits_data_model_schema.xsd Url : http://codesynthesis.com/pipermail/xsd-users/attachments/20140212/0a2fef11/fits_data_model_schema.obj From boris at codesynthesis.com Wed Feb 12 09:35:12 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Wed Feb 12 09:33:07 2014 Subject: [xsd-users] XSD generated code does not compile with g++ 4.8 In-Reply-To: References: Message-ID: Hi Reiner, Reiner Rohlfs writes: > and try to compile it with g++ 4.8.2: > > g++ -c fits_data_model_schema.cxx > > and get following error from the compiler: Yes, this is a know problem which is fixed for the next release of XSD. In the meantime, you can apply the following patch to your XSD 3.3.0 installation to fix this issue: http://codesynthesis.com/~boris/tmp/xsd/xsd-3.3.0-gcc-4.7-clang.patch Boris From Reiner.Rohlfs at unige.ch Wed Feb 12 10:33:29 2014 From: Reiner.Rohlfs at unige.ch (Reiner Rohlfs) Date: Wed Feb 12 10:45:13 2014 Subject: [xsd-users] XSD generated code does not compile with g++ 4.8 In-Reply-To: References: Message-ID: Hi Boris, thanks a lot for the quick answer. And it is working fine. Reiner. On Feb 12, 2014, at 3:35 PM, Boris Kolpackov wrote: > Hi Reiner, > > Reiner Rohlfs writes: > >> and try to compile it with g++ 4.8.2: >> >> g++ -c fits_data_model_schema.cxx >> >> and get following error from the compiler: > > Yes, this is a know problem which is fixed for the next release > of XSD. In the meantime, you can apply the following patch to > your XSD 3.3.0 installation to fix this issue: > > http://codesynthesis.com/~boris/tmp/xsd/xsd-3.3.0-gcc-4.7-clang.patch > > Boris > > From raviraja at qti.qualcomm.com Wed Feb 12 15:09:18 2014 From: raviraja at qti.qualcomm.com (Rangarajan, Ravi_Sowmian) Date: Wed Feb 12 22:26:34 2014 Subject: [xsd-users] Q: schemaLocation in schema file In-Reply-To: References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AEAA2@NASANEXD02B.na.qualcomm.com> Message-ID: <698646F0D0104244B95E527D289651D12240B720@NASANEXD02B.na.qualcomm.com> Hi Boris, Firstly, thanks so much for digging deep in order to help resolve my issue. I do pass options/arguments as per snippet below. ContentFile (arg[1] in your example?) is the XML instance that we want to parse against the schema MPD.xsd. Earler, with just don't_initialize directive, we got the same error as you (library.xml:13:65 error: schemaLocation does not contain namespace location pairs) even though we were overriding schemaLocation as shown below. After adding don't_validate in addition to don't_initialize, now parser doesn't throw exception and is able to proceed. I do think doing validation is a good thing, but for an optional param like schemaLocation per 3GPP spec, skipping validation shouldn't be too bad? Thanks Ravi const std::string MPD_NAMESPACE_STRING = "urn:mpeg:dash:schema:mpd:2011"; const std::string MPD_SCHEMA_FILE = "/opt/lte/etc/xsd/MPD.xsd"; const std::string MPD_SCHEMA_URI = "file://" + MPD_SCHEMA_FILE; xml_schema::properties schemaProperties; schemaProperties.schema_location(MPD_NAMESPACE_STRING, MPD_SCHEMA_URI); std::auto_ptr myMPD; xml_schema::namespace_infomap map; try { std::auto_ptr myMPD ( mpd_2011::MPD(contentFile, xml_schema::flags::dont_initialize | xml_schema::flags::dont_validate, schemaProperties )); } catch { } -----Original Message----- From: Boris Kolpackov [mailto:boris@codesynthesis.com] Sent: Tuesday, February 11, 2014 8:47 PM To: Rangarajan, Ravi_Sowmian Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] Q: schemaLocation in schema file Hi Ravi, Here is what I did: 1. I took the 'library' example from the XSD distribution and modified the library.xml file so that the schemaLocation attribute contains library1.xsd instead of library.xsd (file that doesn't exist). I then run the example and verified that I get the errors. 2. I then added: #include // getcwd To the top of driver.cxx and the following code fragment at the beginning of main() (just after using namespace library;): char cwd[1024]; getcwd (cwd, 1024); xml_schema::properties props; props.schema_location ( "http://www.codesynthesis.com/library", "file:///" + std::string (cwd) + "/library.xsd"); cerr << "schema location: " << props.schema_location () << endl; I also changed the parsing function call that reads like this: std::auto_ptr c (catalog_ (argv[1])); To instead read like this: std::auto_ptr c (catalog_ (argv[1], 0, props)); That is, I am passing the properties with the schema location to the parsing function (perhaps you forgot to this step)? 3. I then re-built the example and ran it on the modified library.xml. The example now runs without any problems and the schema location printed, in my case, as: schema location: http://www.codesynthesis.com/library file:////home/boris/work/xsd/xsd-default/examples/cxx/tree/test/library.xsd 4. I then modified the schemaLocation attribue to only include the namespace, without the schema path. This results in an error that says: library.xml:13:65 error: schemaLocation does not contain namespace- location pairs So this is invalid per the XML Schema specification. Can you try these steps and see if you get the same results? If you do get the same results, then check what you are doing different in your code. Boris From boris at codesynthesis.com Thu Feb 13 22:10:08 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Feb 13 22:08:04 2014 Subject: [xsd-users] Q: schemaLocation in schema file In-Reply-To: <698646F0D0104244B95E527D289651D12240B720@NASANEXD02B.na.qualcomm.com> References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AEAA2@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D12240B720@NASANEXD02B.na.qualcomm.com> Message-ID: Hi Ravi, Rangarajan, Ravi_Sowmian writes: > Earler, with just don't_initialize directive, we got the same error > as you (library.xml:13:65 error: schemaLocation does not contain > namespace location pairs) even though we were overriding schemaLocation > as shown below. Overriding schemaLocation doesn't help in this case since the parser still validates (and then ignores) this attribute's content which should be a sequence of namespace/path pairs. To put it another way, according to the XML Schema spec, the document that contains mal- formed schemaLocation attribute is invalid. > After adding don't_validate in addition to don't_initialize, now > parser doesn't throw exception and is able to proceed. I do think > doing validation is a good thing, but for an optional param like > schemaLocation per 3GPP spec, skipping validation shouldn't be > too bad? If you are sure that the documents you will be parsing are valid (except for the schemaLocation attribute as mentioned above), then disabling validation should be ok. The only other approach that I can think of is to pre-process the XML document and remove the schemaLocation attribute altogether. Boris From raviraja at qti.qualcomm.com Fri Feb 14 01:23:54 2014 From: raviraja at qti.qualcomm.com (Rangarajan, Ravi_Sowmian) Date: Fri Feb 14 10:22:23 2014 Subject: [xsd-users] Q: schemaLocation in schema file In-Reply-To: References: <698646F0D0104244B95E527D289651D1122ADC71@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AE732@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D1122AEAA2@NASANEXD02B.na.qualcomm.com> <698646F0D0104244B95E527D289651D12240B720@NASANEXD02B.na.qualcomm.com> Message-ID: <698646F0D0104244B95E527D289651D122410966@NASANEXD02B.na.qualcomm.com> Hi Boris, Excellent analysis and behavior of the parser was very well explained. Now options are quite clear to me. Thanks so much Ravi -----Original Message----- From: Boris Kolpackov [mailto:boris@codesynthesis.com] Sent: Thursday, February 13, 2014 7:10 PM To: Rangarajan, Ravi_Sowmian Cc: xsd-users@codesynthesis.com Subject: Re: [xsd-users] Q: schemaLocation in schema file Hi Ravi, Rangarajan, Ravi_Sowmian writes: > Earler, with just don't_initialize directive, we got the same error as > you (library.xml:13:65 error: schemaLocation does not contain > namespace location pairs) even though we were overriding > schemaLocation as shown below. Overriding schemaLocation doesn't help in this case since the parser still validates (and then ignores) this attribute's content which should be a sequence of namespace/path pairs. To put it another way, according to the XML Schema spec, the document that contains mal- formed schemaLocation attribute is invalid. > After adding don't_validate in addition to don't_initialize, now > parser doesn't throw exception and is able to proceed. I do think > doing validation is a good thing, but for an optional param like > schemaLocation per 3GPP spec, skipping validation shouldn't be too > bad? If you are sure that the documents you will be parsing are valid (except for the schemaLocation attribute as mentioned above), then disabling validation should be ok. The only other approach that I can think of is to pre-process the XML document and remove the schemaLocation attribute altogether. Boris From fred at essentel.com Sat Feb 15 21:11:28 2014 From: fred at essentel.com (Fred OLeary) Date: Sat Feb 15 22:02:22 2014 Subject: [xsd-users] Parsing mixed content Message-ID: Hello, Although I've searched the archives I've been unable to resolve the following. Our xml has 'optional' text content of the form: There may be stuff here or Our XSD for this is of the form: I would like the generated classes to contain methods to retreive the content if is present and hopefully a method to indicate if content is available For example there is not text content for Any assistance much appreciated From boris at codesynthesis.com Sun Feb 16 11:07:08 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Sun Feb 16 11:05:00 2014 Subject: [xsd-users] Parsing mixed content In-Reply-To: References: Message-ID: Hi Fred, Fred OLeary writes: > Our XSD for this is of the form: > > > The simplest solution is to change (I would even say "fix") your schema: > I would like the generated classes to contain methods to retreive the > content if is present and hopefully a method to indicate if content is > available If you cannot change your schema, then the other two options are: 1. Use the DOM association feature. This approach is shown in the 'mixed' example. 2. Customize the generated class to extract the content and provide the desired API. This approach is shown in the 'custom/mixed' example. Boris From gerard_odonnell at yahoo.co.uk Thu Feb 20 08:31:27 2014 From: gerard_odonnell at yahoo.co.uk (Gerard O'Donnell) Date: Fri Feb 21 00:14:26 2014 Subject: [xsd-users] xmldsig# signatures Message-ID: <1392903087.30182.YahooMailNeo@web172803.mail.ir2.yahoo.com> Hello Boris, my company is iGate, and we've been tasked with coding a web service client for an ebms system. This isn't my area of expertise, so I'm struggling a bit. I'm sending this from my Yahoo account BTW as I'm on vacation at the moment, my igate address is gerard.odonnell@igate.com. As a first step, I've used XSD to generate serialisation classes for: Soap 1.2? ??? ??? http://www.w3.org/2003/05/soap-envelope Ebms??? ??? ??? ??? http://docs.oasis-open.org/ebxml-msg/ebms/v3.0/ns/core/200704/ WS Security??? ??? http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd, http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd Xmldsig??? ??? ??? http://www.w3.org/2000/09/xmldsig# A2C??? ??? ??? ??? ??? http://jcq.org.uk/a2c These schemas use ##any elements - would I be right in saying that I need to modify the XSD files locally, and replace ##any with references to the elements that the web service will actually use? So for Soap 1.2, I have added the ebms and wse namespaces into the Soap xs:schema, and replaced ##any with references to ebms and ws-security: ??? ??? ??? ??? ??? ??? ??? ??? ??? ??? ??? ??? ??? ??? ??? ??? ??? I've done this and can serialise & deserialise an example soap message for the web service: ? ??? ????? ... ??? ??? ????? ??????? ????????? ... ????????? ... ??????? ??????? ... ??????? ????????? ??????????? ... ????????? ??????? ????? ??? ? ? From there though I'm stuck - does CodeSynthesis offer any help with calculating the SignatureValue and DigestValues for xmldsig? If we were to use something like Microsoft's CryptoXML API to populate the Signature, do the CodeSynthesis-generated serialisation classes allow us to insert that Signature into the Soap XML? Thanks, Ger O'Donnell iGate 10 Lower Mount Street, Dublin 2, Ireland From boris at codesynthesis.com Fri Feb 21 01:05:14 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Feb 21 01:03:08 2014 Subject: [xsd-users] xmldsig# signatures In-Reply-To: <1392903087.30182.YahooMailNeo@web172803.mail.ir2.yahoo.com> References: <1392903087.30182.YahooMailNeo@web172803.mail.ir2.yahoo.com> Message-ID: Hi Gerard, Gerard O'Donnell writes: > These schemas use ##any elements - would I be right in saying that > I need to modify the XSD files locally, and replace ##any with > references to the elements that the web service will actually use? No, I would suggest that you do not modify the schema and instead "assemble" your complete document from individually serialized fragments, as shown in the 'messaging' example. > From there though I'm stuck - does CodeSynthesis offer any help with > calculating the SignatureValue and DigestValues for xmldsig? No, XSD does not provide this functionality out of the box. The recommended way is to serialize the document into DOM and then use something like Apache XML Security to add the signature: http://santuario.apache.org/ Boris From fred at essentel.com Fri Feb 21 14:05:35 2014 From: fred at essentel.com (Fred OLeary) Date: Sat Feb 22 00:33:25 2014 Subject: [xsd-users] Parsing mixed content In-Reply-To: Message-ID: Thanks Boris! On 2/16/14 8:07 AM, "Boris Kolpackov" wrote: >Hi Fred, > >Fred OLeary writes: > >> Our XSD for this is of the form: >> >> >use="optional"/> >> > >The simplest solution is to change (I would even say "fix") your schema: > > > > > > > > > > >> I would like the generated classes to contain methods to retreive the >> content if is present and hopefully a method to indicate if content is >> available > >If you cannot change your schema, then the other two options are: > >1. Use the DOM association feature. This approach is shown in the > 'mixed' example. > >2. Customize the generated class to extract the content and provide > the desired API. This approach is shown in the 'custom/mixed' > example. > >Boris > From gerard_odonnell at yahoo.co.uk Fri Feb 21 14:45:24 2014 From: gerard_odonnell at yahoo.co.uk (Gerard O'Donnell) Date: Sat Feb 22 00:33:25 2014 Subject: [xsd-users] xmldsig# signatures In-Reply-To: References: <1392903087.30182.YahooMailNeo@web172803.mail.ir2.yahoo.com> Message-ID: <1393011924.42616.YahooMailNeo@web172806.mail.ir2.yahoo.com> Thanks for the rapid response Boris, I'll try that out. ________________________________ From: Boris Kolpackov To: Gerard O'Donnell Cc: "xsd-users@codesynthesis.com" Sent: Friday, 21 February 2014, 6:05:14 Subject: Re: [xsd-users] xmldsig# signatures Hi Gerard, Gerard O'Donnell writes: > These schemas use ##any elements - would I be right in saying that > I need to modify the XSD files locally, and replace ##any with > references to the elements that the web service will actually use? No, I would suggest that you do not modify the schema and instead "assemble" your complete document from individually serialized fragments, as shown in the 'messaging' example. > From there though I'm stuck - does CodeSynthesis offer any help with > calculating the SignatureValue and DigestValues for xmldsig? No, XSD does not provide this functionality out of the box. The recommended way is to serialize the document into DOM and then use something like Apache XML Security to add the signature: http://santuario.apache.org/ Boris From mathew.benson at gmail.com Wed Feb 26 05:15:18 2014 From: mathew.benson at gmail.com (Mathew Benson) Date: Wed Feb 26 05:15:27 2014 Subject: [xsd-users] Pointers to references Message-ID: <362490BF-B8AD-4ABE-883C-CBA09B00EAD4@gmail.com> I'm using a standard xsd file that includes references to other elements. The auto code returns the name of the referenced element. Is it possible to configure the auto coder or tweak the code to store a pointer to the referenced element? As it is, I have to search the tree for the referenced element. Sent from my iPhone From boris at codesynthesis.com Fri Feb 28 08:01:00 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Feb 28 08:04:11 2014 Subject: [xsd-users] Pointers to references In-Reply-To: <362490BF-B8AD-4ABE-883C-CBA09B00EAD4@gmail.com> References: <362490BF-B8AD-4ABE-883C-CBA09B00EAD4@gmail.com> Message-ID: Hi Mathew, Mathew Benson writes: > I'm using a standard xsd file that includes references to other elements. > The auto code returns the name of the referenced element. Is it possible to > configure the auto coder or tweak the code to store a pointer to the > referenced element? As it is, I have to search the tree for the referenced > element. I have no idea what you are talking about. What is a "standard xsd file", "auto code", "auto coder"? Show the corresponding schema fragment and the C++ code that you want to write. Boris From mathew.benson at gmail.com Fri Feb 28 09:52:01 2014 From: mathew.benson at gmail.com (Mathew Benson) Date: Fri Feb 28 09:52:11 2014 Subject: [xsd-users] Pointers to references In-Reply-To: References: <362490BF-B8AD-4ABE-883C-CBA09B00EAD4@gmail.com> Message-ID: <065F4DF4-55C5-48EB-909C-02E2DB4C0A64@gmail.com> I'm using the XTCE (XML Telemetry and Command Exchange) v1.1 schema documented by the Consultative Committee for Space Data Systems (CCSDS) at http://public.ccsds.org/publications/archive/660x0g1.pdf. I run xsd cxx-tree on the schema file which generates a bunch of C++ code. I load my XTCE file and can pull out data just fine. The code generated by xsd saved me a massive amount of time and increased the quality of my code. The problem is that many of the XTCE elements have properties that are "references" to other XTCE elements. It refers to these elements by a unique name, I.e.: . I have to search the data structure for the element name. My improved solution is to add pointers to the generated classes for those elements, and do some post processing after unmarshalling the input file. The post processing sets those pointers. So after initialization, I can just follow the pointer, rather than searching the tree for the referenced element every time. The previous example would look like: AnotherElement.h class AnotherElement { ... private: SomeElement *_ptrSomeElement; public: SomeElement *getSomeElement(void); ... } AnotherElement.cpp SomeElement *AnotherElement::getSomeElement(){ return _ptrSomeElement; } main.cpp AnotherElement *objectA; SomeElement *objectB; ... objectB = objectA->getSomeElement(); I'm new to xsd and was wondering if it was capable of doing that post processing itself. I thought it would since it correctly self validates the input XML and tells me when mathew XML references an element that does not exist. I also don't know the best way to use xsd. Is it generating skeletal code that I can/should modify, or should I treat it as autocode that can be regenerated and should not be modified directly? Does that help? I can post a fragment shortly. Sent from my iPhone > On Feb 28, 2014, at 7:01 AM, Boris Kolpackov wrote: > > Hi Mathew, > > Mathew Benson writes: > >> I'm using a standard xsd file that includes references to other elements. >> The auto code returns the name of the referenced element. Is it possible to >> configure the auto coder or tweak the code to store a pointer to the >> referenced element? As it is, I have to search the tree for the referenced >> element. > > I have no idea what you are talking about. What is a "standard xsd file", > "auto code", "auto coder"? Show the corresponding schema fragment and the > C++ code that you want to write. > > Boris From boris at codesynthesis.com Fri Feb 28 10:05:01 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Feb 28 10:08:10 2014 Subject: [xsd-users] Pointers to references In-Reply-To: <065F4DF4-55C5-48EB-909C-02E2DB4C0A64@gmail.com> References: <362490BF-B8AD-4ABE-883C-CBA09B00EAD4@gmail.com> <065F4DF4-55C5-48EB-909C-02E2DB4C0A64@gmail.com> Message-ID: Hi Mathew, Mathew Benson writes: > > . If your schema uses xs:ID type for 'name' and xs:IDREF for 'someElementRef', then, yes, XSD will resolve such references for you. See: http://codesynthesis.com/projects/xsd/documentation/cxx/tree/manual/#2.5.5 > Is it generating skeletal code that I can/should modify, or should I > treat it as autocode that can be regenerated and should not be modified > directly? The generated code should not be modified but there is a mechanism for customizing the generated classes. For more information on type customization see the C++/Tree Mapping Customization Guide: http://wiki.codesynthesis.com/Tree/Customization_guide As well as the examples in the examples/cxx/tree/custom/ directory. So if ID/IDREF does not apply (i.e., your schema uses key/keyref), then the next best thing is to customize the generated classes to add the pointers and to initialize them in the parsing constructors. Boris From mathew.benson at gmail.com Fri Feb 28 11:38:50 2014 From: mathew.benson at gmail.com (Mathew Benson) Date: Fri Feb 28 11:38:58 2014 Subject: [xsd-users] Pointers to references In-Reply-To: References: <362490BF-B8AD-4ABE-883C-CBA09B00EAD4@gmail.com> <065F4DF4-55C5-48EB-909C-02E2DB4C0A64@gmail.com> Message-ID: Ok. Thanks. Since I'm using an already documented and standardized schema, I'm leery about modifying the schema file. I just wish they would have considered that when the schema was defined. On Fri, Feb 28, 2014 at 9:05 AM, Boris Kolpackov wrote: > Hi Mathew, > > Mathew Benson writes: > > > > > . > > If your schema uses xs:ID type for 'name' and xs:IDREF for > 'someElementRef', then, yes, XSD will resolve such references > for you. See: > > http://codesynthesis.com/projects/xsd/documentation/cxx/tree/manual/#2.5.5 > > > > Is it generating skeletal code that I can/should modify, or should I > > treat it as autocode that can be regenerated and should not be modified > > directly? > > The generated code should not be modified but there is a mechanism > for customizing the generated classes. For more information on type > customization see the C++/Tree Mapping Customization Guide: > > http://wiki.codesynthesis.com/Tree/Customization_guide > > As well as the examples in the examples/cxx/tree/custom/ directory. > > So if ID/IDREF does not apply (i.e., your schema uses key/keyref), > then the next best thing is to customize the generated classes to > add the pointers and to initialize them in the parsing constructors. > > Boris > From timur.gamerov at gmail.com Fri Feb 28 14:51:59 2014 From: timur.gamerov at gmail.com (Timur Gamerov) Date: Fri Feb 28 14:52:10 2014 Subject: [xsd-users] latest xsd+dep Message-ID: <5310E8DF.3020006@gmail.com> Would anybody tell me how to cook xsd+dep from git repository? My way to build XSD from git: git clone git://scm.codesynthesis.com/build/build.git git clone git://scm.codesynthesis.com/libcutl/libcutl.git git clone git://scm.codesynthesis.com/libxsd-frontend/libxsd-frontend.git git clone git://scm.codesynthesis.com/cli/cli.git git clone git://scm.codesynthesis.com/xsd/xsd.git mkdir x86_64-pc-linux-gnu cd x86_64-pc-linux-gnu/ make verbose=1 install_prefix=. -f ../build/makefile -j4 install export MAKEFLAGS="-I include" ; make verbose=1 install_prefix=../the-xsd -f ../xsd/makefile -j4 install Then input paths like ../libcutl, /home/timur/toolbuild/xerces-c-3.1.1, ../cli, ../libxsd-frontend repeatly Is that good way to build the xsd? How to use xsd+dep from git? -- Timur Gamerov From mathew.benson at gmail.com Fri Feb 28 21:16:00 2014 From: mathew.benson at gmail.com (Mathew Benson) Date: Fri Feb 28 21:16:08 2014 Subject: [xsd-users] Pointers to references In-Reply-To: References: <362490BF-B8AD-4ABE-883C-CBA09B00EAD4@gmail.com> <065F4DF4-55C5-48EB-909C-02E2DB4C0A64@gmail.com> Message-ID: Thanks for the tip. I dug into it more and found that it was much easier to just change the xsd file a bit to enforce the relationship, while still keeping the basic schema the same and without writing any code. I know the relationship is now enforced because if I set a IDREF without an ID, it will fail to unmarshall the xml file. However, I'm still struggling at how to get to the object from inside code. Below is a snippet of my input xml file: Long description for the header. All long descriptions work this way. 272 0 160 Could you tell me the best way to retrieve 'argumentTypeRef="IpAddressType"' given I have a pointer/reference to the '' object? I'm trying to iterate through all the command objects and enumerate all the contained elements and properties. On Fri, Feb 28, 2014 at 10:38 AM, Mathew Benson wrote: > Ok. Thanks. Since I'm using an already documented and standardized > schema, I'm leery about modifying the schema file. I just wish they would > have considered that when the schema was defined. > > > On Fri, Feb 28, 2014 at 9:05 AM, Boris Kolpackov wrote: > >> Hi Mathew, >> >> Mathew Benson writes: >> >> > >> > . >> >> If your schema uses xs:ID type for 'name' and xs:IDREF for >> 'someElementRef', then, yes, XSD will resolve such references >> for you. See: >> >> http://codesynthesis.com/projects/xsd/documentation/cxx/tree/manual/#2.5.5 >> >> >> > Is it generating skeletal code that I can/should modify, or should I >> > treat it as autocode that can be regenerated and should not be modified >> > directly? >> >> The generated code should not be modified but there is a mechanism >> for customizing the generated classes. For more information on type >> customization see the C++/Tree Mapping Customization Guide: >> >> http://wiki.codesynthesis.com/Tree/Customization_guide >> >> As well as the examples in the examples/cxx/tree/custom/ directory. >> >> So if ID/IDREF does not apply (i.e., your schema uses key/keyref), >> then the next best thing is to customize the generated classes to >> add the pointers and to initialize them in the parsing constructors. >> >> Boris >> > > From Wesley.Peters at tachyon.com Fri Feb 28 13:20:43 2014 From: Wesley.Peters at tachyon.com (Wesley Peters) Date: Sat Mar 1 00:25:45 2014 Subject: [xsd-users] Pointers to references In-Reply-To: <779373584.772.1393599812271.JavaMail.root@birdun> Message-ID: Boris, I'd just like to thank you again for your detailed answers to questions from the XSD userbase. Your products are innovative and helpful and your customer service is fantastic. On 2/28/14 7:05 AM, "Boris Kolpackov" wrote: Hi Mathew, Mathew Benson writes: > > . If your schema uses xs:ID type for 'name' and xs:IDREF for 'someElementRef', then, yes, XSD will resolve such references for you. See: http://codesynthesis.com/projects/xsd/documentation/cxx/tree/manual/#2.5.5 > Is it generating skeletal code that I can/should modify, or should I > treat it as autocode that can be regenerated and should not be modified > directly? The generated code should not be modified but there is a mechanism for customizing the generated classes. For more information on type customization see the C++/Tree Mapping Customization Guide: http://wiki.codesynthesis.com/Tree/Customization_guide As well as the examples in the examples/cxx/tree/custom/ directory. So if ID/IDREF does not apply (i.e., your schema uses key/keyref), then the next best thing is to customize the generated classes to add the pointers and to initialize them in the parsing constructors. Boris ------------------------------------------------------------- Mail1 junk filter. Click here http://mail1.tachyon.net:5272/FrontController?operation=mbeu&f=00001_-128_20140228_628592.eml&chkBayesian=1&pr=1&mt=1&ma=s to mark email as junk. ------------------------------------------------------------- ________________________________ Confidentiality Notice: The information contained in this electronic e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and is confidential and/or privileged. If you and we have a confidentiality agreement or other non-disclosure obligations between us, this Notice shall be deemed to mark and identify the content of this email and any attachments as confidential and proprietary. If any reader of this communication is not the intended recipient, unauthorized use, disclosure or copying is strictly prohibited, and may be unlawful. If you have received this communication in error, please immediately notify the sender by return e-mail, and delete the original message and all copies from your system. Thank you. IRS Circular 230 Disclosure: To ensure compliance with requirements imposed by the IRS, please be advised that any U.S. federal tax advice contained in this communication (including any attachments) is not intended or written to be used or relied upon, and cannot be used or relied upon, for the purpose of (i) avoiding penalties under the Internal Revenue Code, or (ii) promoting, marketing or recommending to another party any transaction or matter addressed herein. E-mail is susceptible to data corruption, interception, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.