From Jonathan.Haws at sdl.usu.edu Thu Jan 9 22:47:41 2014 From: Jonathan.Haws at sdl.usu.edu (Jonathan Haws) Date: Thu Jan 9 22:47:51 2014 Subject: [xsde-users] ios_base::failure exceptions when trying to parse Message-ID: <1f718d3d537046a6bb468bf049678a0b@Perses.usurf.usu.edu> I have a new schema that I am trying to parse. The tool generates code that compiles just fine, however when I try to parse a document I get exceptions. Here is my parsing code: void seiwg_platformstatusreport_p(const char * const xml) { try { /* Parse */ PlatformStatusReport_paggr psr_p; xml_schema::document_pimpl doc_p(psr_p.root_parser(), psr_p.root_namespace(), psr_p.root_name()); psr_p.pre(); doc_p.parse(xml); // <<------EXCEPTION THROWN HERE!!! auto_ptr psr(psr_p.post()); } catch (const xml_schema::parser_exception& e) { cerr << xml << ":" << e.line() << ":" << e.column() << ": " << e.text() << endl; return; } catch (const xml_schema::serializer_exception& e) { cerr << "error: " << e.text() << endl; return; } catch (const std::ios_base::failure&) // <<------THIS EXCEPTION IS THROWN!! { cerr << xml << ": unable to open or read/write failure" << endl; return; } catch (...) { cerr << xml << ": unhandled exception" << endl; return; } return; } /* seiwg_platformstatusreport_p() */ And here is the document: When I generate code, here is the command used: xsde cxx-hybrid --generate-inline --generate-parser --generate-serializer \ --generate-aggregate --root-element PlatformStatusReport --output-dir SEIWG \ SEIWG/PlatformStatusReport.xsd Any ideas why I am seeing those ios_base exceptions? I can't even step into the doc_p.parse() routine to try and find out what is up. It's almost like the doc_p is not getting created correctly. Unfortunately I am unable to share the XSD files to show the schema, so hopefully this is a simple issue that is easily resolved. Thanks! Jon -- Jonathan R. Haws Embedded Engineer Space Dynamics Laboratory (435) 713-3489 jhaws@sdl.usu.edu From boris at codesynthesis.com Fri Jan 10 03:25:11 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Jan 10 03:27:36 2014 Subject: [xsde-users] ios_base::failure exceptions when trying to parse In-Reply-To: <1f718d3d537046a6bb468bf049678a0b@Perses.usurf.usu.edu> References: <1f718d3d537046a6bb468bf049678a0b@Perses.usurf.usu.edu> Message-ID: Hi Jonathan, Jonathan Haws writes: > void seiwg_platformstatusreport_p(const char * const xml) > > doc_p.parse(xml); // <<------EXCEPTION THROWN HERE!!! Does the xml variable contain the file name of the XML document or the XML document itself? This function expects it to be the file name. If it is the XML itself, then you should do something like this instead: doc_p.parse(xml, strlen (xml), true); See the documentation for details. Boris From Jonathan.Haws at sdl.usu.edu Fri Jan 10 09:07:11 2014 From: Jonathan.Haws at sdl.usu.edu (Jonathan Haws) Date: Fri Jan 10 09:07:20 2014 Subject: [xsde-users] ios_base::failure exceptions when trying to parse In-Reply-To: References: <1f718d3d537046a6bb468bf049678a0b@Perses.usurf.usu.edu>, Message-ID: <0g4ugvfcc3jo8t65c9pkml12.1389362828197@email.android.com> Ah yes, that's right. I had forgot about that. It's been a while since I've used this tool! Thanks, Boris! Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Boris Kolpackov Date:01/10/2014 1:27 AM (GMT-07:00) To: Jonathan Haws Cc: xsde-users@codesynthesis.com Subject: Re: [xsde-users] ios_base::failure exceptions when trying to parse Hi Jonathan, Jonathan Haws writes: > void seiwg_platformstatusreport_p(const char * const xml) > > doc_p.parse(xml); // <<------EXCEPTION THROWN HERE!!! Does the xml variable contain the file name of the XML document or the XML document itself? This function expects it to be the file name. If it is the XML itself, then you should do something like this instead: doc_p.parse(xml, strlen (xml), true); See the documentation for details. Boris From Jonathan.Haws at sdl.usu.edu Sat Jan 11 17:08:26 2014 From: Jonathan.Haws at sdl.usu.edu (Jonathan Haws) Date: Sat Jan 11 17:08:37 2014 Subject: [xsde-users] Generate aggregate that handles multiple root elements Message-ID: <872c3bb5efdb452c90fab3b7ae7c8fff@Perses.usurf.usu.edu> Is it possible to take a schema that defines multiple valid root elements and generate an aggregate that will parse/serialize based on what it sees? For example, if I have a schema that defines root elements A, B, and C can I generate a parser that will accept: As well as, And, And determine what the root element is for me and parse the document? I would also like to be able to serialize and not worry about the root element and just have some routines that look something like: Serialize(A&); Serialize(B&); Serialize(C&); Serializing would be the easy part if I had to do it myself. Does that make sense? Can XSD/e generate something like this or am I on my own to write a class that handles all that for me? If I am on my own, what is the easiest way to determine what the root element is since I cannot simply parse it? I would rather not manually parse it using string functions if I can avoid that. I did look at the multiroot example and didn't seem to see an answer in there... Thanks! Jonathan -- Jonathan R. Haws Embedded Engineer Space Dynamics Laboratory (435) 713-3489 jhaws@sdl.usu.edu From boris at codesynthesis.com Mon Jan 13 10:03:11 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Mon Jan 13 10:05:38 2014 Subject: [xsde-users] Generate aggregate that handles multiple root elements In-Reply-To: <872c3bb5efdb452c90fab3b7ae7c8fff@Perses.usurf.usu.edu> References: <872c3bb5efdb452c90fab3b7ae7c8fff@Perses.usurf.usu.edu> Message-ID: Hi Jonathan, Jonathan Haws writes: > Is it possible to take a schema that defines multiple valid root elements > and generate an aggregate that will parse/serialize based on what it sees? > > For example, if I have a schema that defines root elements A, B, and C can I > generate a parser that will accept: > > > > > > > As well as, > > > > > > And, > > > > And determine what the root element is for me and parse the document? The problem with supporting something like is that the returned types are different. Instead, XSD/e allows you to determine the root element and supply a corresponding parser. See the 'multiroot' example for how to set this up. > I would also like to be able to serialize and not worry about the > root element and just have some routines that look something like: > > Serialize(A&); > Serialize(B&); > Serialize(C&); > > Serializing would be the easy part if I had to do it myself. The problem with supporting something like this is that, in the general case, there could be two elements with the same type. So in this case you will have to do it yourself. Boris From Jonathan.Haws at sdl.usu.edu Wed Jan 15 21:32:48 2014 From: Jonathan.Haws at sdl.usu.edu (Jonathan Haws) Date: Wed Jan 15 21:32:57 2014 Subject: [xsde-users] Parsing a TCP stream Message-ID: <0fdaf8aa31a64780891c5c016e8d2b6a@Perses.usurf.usu.edu> I am looking for some tools that will parse a TCP stream into full XML documents. I have searched online and found that some use the boost::asio libraries. I thought would ask here what the recommended tools are. Bottom line is I am receiving XML documents from a TCP socket and need to know where one document ends and another begins. Is there a way to automatically do this using things built into XSD/e? If not, what is the best way to go about doing that? Thanks! Jonathan From boris at codesynthesis.com Thu Jan 16 07:26:42 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Thu Jan 16 07:29:06 2014 Subject: [xsde-users] Parsing a TCP stream In-Reply-To: <0fdaf8aa31a64780891c5c016e8d2b6a@Perses.usurf.usu.edu> References: <0fdaf8aa31a64780891c5c016e8d2b6a@Perses.usurf.usu.edu> Message-ID: Hi Jonathan, Jonathan Haws writes: > I am looking for some tools that will parse a TCP stream into full XML > documents. I have searched online and found that some use the boost::asio > libraries. > > I thought would ask here what the recommended tools are. I don't think there could be a single "recommended" way to do this. It all depends on your requirements. For some application using raw sockets could be a perfectly reasonable approach. While other may requires something as involved as ASIO. Generally, I think it has much more to do with the kind of communication you will be doing: multiple sockets or single socket, single/multithread-handling of connections, blocking/non-blocking/asynchronous I/O, etc. As a generally guide, I would suggest that you keep it simple, at least at the beginning (e.g., blocking raw socket). > Bottom line is I am receiving XML documents from a TCP socket and > need to know where one document ends and another begins. Is there a > way to automatically do this using things built into XSD/e? No, there is no automatic (more like auto-magical) way. > If not, what is the best way to go about doing that? In XML, NULL ('\0', U+0000) is an invalid character (both in XML 1.0 and 1.1) which makes it a convenient document delimiter. Boris From Jonathan.Haws at sdl.usu.edu Thu Jan 16 08:30:30 2014 From: Jonathan.Haws at sdl.usu.edu (Jonathan Haws) Date: Thu Jan 16 08:30:38 2014 Subject: [xsde-users] Parsing a TCP stream In-Reply-To: References: <0fdaf8aa31a64780891c5c016e8d2b6a@Perses.usurf.usu.edu> Message-ID: <503431cad5ae4a6a81b6b1411fa45178@Perses.usurf.usu.edu> > > I am looking for some tools that will parse a TCP stream into full XML > > documents. I have searched online and found that some use the > > boost::asio libraries. > > > > I thought would ask here what the recommended tools are. > > I don't think there could be a single "recommended" way to do this. It all depends on your requirements. For some application using raw sockets could be a perfectly reasonable approach. > While other may requires something as involved as ASIO. Generally, I think it has much more to do with the kind of communication you will be doing: multiple sockets or single socket, > single/multithread-handling of connections, blocking/non-blocking/asynchronous I/O, etc. > > As a generally guide, I would suggest that you keep it simple, at least at the beginning (e.g., blocking raw socket). My setup will be a single thread managing a single connection, most likely blocking until it receives data. The problem I run up against is the fact that a single read may or may not get the whole document. Does that narrow things down any on various approaches? My experience with TCP communication has always included some form of formatted header that gives me all the information I need to either read more data or process it. Unfortunately XML does not have that -- all it has is a closing tag for the root element, but since my schema has multiple options for root elements, some of which can also be children, that complicates this significantly to simply check for a closing tag at the end of the buffer... > > Bottom line is I am receiving XML documents from a TCP socket and need > > to know where one document ends and another begins. Is there a way to > > automatically do this using things built into XSD/e? > > No, there is no automatic (more like auto-magical) way. Dang, I was hoping there was some way that I could just feed it a stringstream or something similar and just have it parse to the end of the root element. > > If not, what is the best way to go about doing that? > > In XML, NULL ('\0', U+0000) is an invalid character (both in XML 1.0 and 1.1) which makes it a convenient document delimiter. That seems like a nice way to handle it, however the tools I have to verify my code against the defined schema (provided to me by the customer) doesn't send a NULL character, so that tells me that it isn't an options to base my parsing off that. Too bad... From boris at codesynthesis.com Fri Jan 17 13:37:46 2014 From: boris at codesynthesis.com (Boris Kolpackov) Date: Fri Jan 17 13:40:10 2014 Subject: [xsde-users] Parsing a TCP stream In-Reply-To: <503431cad5ae4a6a81b6b1411fa45178@Perses.usurf.usu.edu> References: <0fdaf8aa31a64780891c5c016e8d2b6a@Perses.usurf.usu.edu> <503431cad5ae4a6a81b6b1411fa45178@Perses.usurf.usu.edu> Message-ID: Hi Jonathan, Jonathan Haws writes: > My setup will be a single thread managing a single connection, most likely > blocking until it receives data. The problem I run up against is the fact > that a single read may or may not get the whole document. Does that narrow > things down any on various approaches? Yes, you will need to somehow separate documents. That's why I suggested using the NULL byte as a document separator. This way you will know where one document ends and the next begins. > Dang, I was hoping there was some way that I could just feed it a > stringstream or something similar and just have it parse to the end > of the root element. XSD/e parser (and all other XML parser that I have heard of) operate in terms of XML documents, not "XML documents and some extra stuff that should be ignored". In fact, if you try to parse such a buffer with XSD/e, you will get an error saying that there is garbage after the closing tag (which is illegal per the XML spec). > That seems like a nice way to handle it, however the tools I have to verify > my code against the defined schema (provided to me by the customer) doesn't > send a NULL character, so that tells me that it isn't an options to base my > parsing off that. Too bad... I am not sure we are on the same page here, so just to clarify: the NULL character is not part of the XML vocabulary or schema that describes it. Rather, it is a protocol-level delimiter that is used to determine where one document ends and the next begins. If you have control over the protocol, then it is very easy to implement. If the protocol is defined by someone else, then you will have to implement some ad-hoc (and most likely not very robust) approach of determining where the document ends. Boris From Jonathan.Haws at sdl.usu.edu Fri Jan 17 13:46:19 2014 From: Jonathan.Haws at sdl.usu.edu (Jonathan Haws) Date: Fri Jan 17 13:46:33 2014 Subject: [xsde-users] Parsing a TCP stream In-Reply-To: References: <0fdaf8aa31a64780891c5c016e8d2b6a@Perses.usurf.usu.edu> <503431cad5ae4a6a81b6b1411fa45178@Perses.usurf.usu.edu>, Message-ID: <416fdac2b3014dc098588d05f3fa1169@Perses.usurf.usu.edu> > I am not sure we are on the same page here, so just to clarify: the > NULL character is not part of the XML vocabulary or schema that > describes it. Rather, it is a protocol-level delimiter that is used > to determine where one document ends and the next begins. If you have > control over the protocol, then it is very easy to implement. If the > protocol is defined by someone else, then you will have to implement > some ad-hoc (and most likely not very robust) approach of determining > where the document ends. That is my problem - I don't have control over the protocol and those that do decided not to use the NULL character to mark the end of a document. I captured some of their data from their verification tool and it did not have the NULL character, that is why I mentioned I wouldn't be able to use it; thus nearly requiring me to implement an ad-hoc method (yes, not robust; which is why I was hoping there was a better way!). I am trying to get a hold of someone who created the protocol to see what they have done in the past, but so far have not had any luck finding anyone... Thanks for the help, Boris! You are always very helpful!