Convert node to stream/string to extract data before noteToDocument step

, ,
1

We are using wM 6.5SP2 on a Windows 2003 platform. We have a service that has been working for a couple of years now where we receive XML documents via HTTP POST so I bring them as node into the flow and then do noteToDocument to convert them to the XML document. In a new project we are expecting to receive now several different XML formats. We still would like to route all of them through the same flow service so ideally I would like to examine the first few characters of the incoming document to determine which xml format it is, based on that convert the document to our common format and THEN process it nodeToDocument and then through the flow.

In testing I use the getfile command to test my flow and have it working but I am stumped how to do that in the live environment when the incoming data comes as a node as defined in the service input. When I do the getFile command I can easily convert the data using the PSUtilities.stream:streamToString service and then examine a substring of the resulting string to do my determination.

I browsed through Built-In Service reference guide and services available to me but I can’t seem to find much to convert node to stream so I could then use the streamToString service in the live environment too.

Does anybody have any suggestions?

Thanks

Mike

2

You can introspect whats coming in the input with flow:getTransportInfo service but one thing is you cannot save/able to see streams in the pipeline…

But as far the when your external world/TP’s send the XMLData request them to set content-type = “text/xml” always and so your input *node to the service works as expected…

If you were receiving streams data the pipeline var would be *inputStream (object)…

Either way you need to fully test it out before putting your common XML gateway service to live.

HTH,
RMG

3

Create a java service with input as node object and the outputs as string outStr (for string result) and outStream (for stream result). This should convert an xmlnode to string and Byte array stream.

import you will need,

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.io.StringWriter;
import java.io.UnsupportedEncodingException;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.w3c.dom.Node;

Code:

IDataCursor pipelineCursor = pipeline.getCursor();
        Node node = (Node) IDataUtil.get( pipelineCursor, "node" );
        pipelineCursor.destroy();
        
        StringWriter sw = new StringWriter();
            try {
                
              Transformer t = TransformerFactory.newInstance().newTransformer();
              t.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
              t.setOutputProperty(OutputKeys.INDENT, "yes");
              t.transform(new DOMSource(node), new StreamResult(sw));
              // convert to string debug...........
              String x=sw.toString();
              IDataUtil.put( pipelineCursor, "outStr", x);
             
              InputStream outStream = new ByteArrayInputStream(x.getBytes("UTF-8"));
              IDataUtil.put( pipelineCursor, "outStream", outStream);
              
            } catch (TransformerException e) {              
              IDataUtil.put( pipelineCursor, "outStr", e);
          } catch (UnsupportedEncodingException e) {
              IDataUtil.put( pipelineCursor, "outStr", e);
            }

Hope this helps!
Akshith

4

Neither streams nor Java services are necessary.

You can query the node to determine the document type being processed. For example, use the pub.xml:queryXMLNode service to examine the label of the root tag. That may be sufficient for your purposes.

Trading Networks does this sort of thing out of the box so you might consider that.

If determining the doc type becomes too involved (e.g. looking at the root tag and other elements is needed) then it is probably better to not have just one entry point. Have an entry point for each doc type/partner. The amount of duplicate code you’re trying to avoid by using a single entry point isn’t very much.

Lastly, you stated “determine which xml format it is, based on that convert the document to our common format and THEN process it nodeToDocument and then through the flow”. The concept is good but I’m not sure about the implementation plan. How do you plan to convert to “our common format?” What would that entail? If you’re thinking string manipulation you probably want to reconsider.

What you may want to consider, depending upon the complexity of converting to the common format, is to have different entry services for each different XML document type. Those services convert to the common format–then call the one service that accepts that common format and does whatever processing is needed.

Hope this helps.

5

Thank you reamon, rmg and akki_84 for your replies. I apologize for not responding sooner but somehow I did not get auto-notification of the replies.

I will have to try the queryXMLNode command out. I know I gave it one quick attempt a week ago but using the built-in-reference documentation I could not get it to work.

I have had some success with my flow though. I am able to route transmissions from four different sources already through just one flow service by using the XMLNodeToDocument service without specifying a specific document reference and then analyzing the data. I do have an issue though with one source.

Their files have the three digit BOM characters at the beginning of the file → <?xml version="1.0" encoding="UTF-8"?>. And when I try to run the XMLNodeToDocument wM6.5 doesn’t seem to be able to convert the data to document and a couple steps later in the flow I get: com.wm.app.b2b.server.ServiceException - java.lang.StringIndexOutOfBoundsException: String index out of range: -2

If I use the getFile command to pick up the document I can convert the filestream to string and then find and replace away the three characters and then process the file OK. Unfortunately I have not found a way to do this when the file comes in as a node like it will be in the live environment. Scratching my head on this one… after posting this I will try the query command out and see if I can get it to work on that file in question. Oh and when testing this by sending the file via HTTP POST to my service I do specify content type text/xml.

Thanks
Mike

6

Ok I wanted to give you guys an update before I leave tonight. Akki - I created teh Java service and tried ocnverting the incoming node to string but the service for the Files with BOM characters still fails. After doing some googling and research on wm forum I discovered one reference that stated that Java 1.4 was the last Java that did NOT support BOM characters at beginning of file. So that must be my problem. So I am now looking into upgrading my JVM from 1.4.2 to Java 5 which based on threads here in the forum is supported by wm 6.5 SP2.

I changed the %JAVA_DIR% pointing it to the Java 5 jre dir in bin\server.bat but a little further in that file there is a reference to a \lib\i18n.jar which does not exist in Java 5 so I tried to start IS without that reference or with the reference but it will not start. Error is %%80 file exists - could it mean the lockfile? I check before I try to start the service and the lockfile is not present. When I then try to start the IS the lockfile does get created and then the start fails with the error. Does it get created too fast???

I did some checking here on the forum and found a couple of threads discussing JAva 5/1.5 and one also mentioned changing conifg/server.cnf and adding watt.server.compile lines to point to new javac. So I did that. Result is unfortunately still the same.

Thanks for reading this thread and any suggestions are welcome. i would like to get this resolved. it would be great for me to come up with a solution to receive my files from six different sources all in one flow service.

Thanks for your time.

Mike

7

Mike,

I believe IS 6.5 does not support java 1.5.

http://techcommunity.softwareag.com/ecosystem/documentation/webmethods/wmsuites/wMdoc/_webM_suite_65/_Cross-Product%20Documentation/webMethods%20System%20Requirements%206.x.pdf

Use the skip method in the BOMInputStrean API from Apache Commons utils and remove those BOM characters.

http://commons.apache.org/io/apidocs/org/apache/commons/io/input/BOMInputStream.html

8

You may need to consume the BOM marker yourself. Unfortunately, that likely means you’ll need to ask your partner to post using a content-type of something other than text/xml so that the IS content handler doesn’t processing the incoming data into a node. In other words, by the time your service is invoked with text/xml, it is too late to do anything about the BOM.

9

Akki_84 - actually the read_me file that is installed with wMIS 6.5SP2 on our server states that SP2 is the first IS version that is compatible with Java 1.5:

>> Note: Integration Server 6.5 with Service Pack 2 can also use Sun JDK 1.5. (This is the

standard JDK if you install Integration Server with Service Pack 2 into a webMethods 7.0
environment.) For more information, see 1-1AQOH6, 1-1AQOFZ in 2.0 Known Issues.

\EnterpriseOne\wsg\IntegrationServer\IS_6-5_SP2_Readme.html

We got our wMIS 6.5SP2 from Oracle so I don’t know if this is the standard read-me that also comes with the SoftwareAG version of the product or not.

I may not need to go that route though afterall. We are in contact with the distributor who is sending us the XML order file with the embedded BOM characters and it looks like they might be able to strip them off before sending us the file. So my problem might be solved.

Thanks for your input and thanks for the information about the Apache code to strip BOM off. I may still try that to see if I can get it to work anyways. I haven’t done much with compiling custom Java services in this environment so I may just widen my horizons and give it a try.

Reamon,

thank you also again for your input. Having them change the content type is also a good idea but if I can get them to not send the offending characters at all then I may not have to do that.

Thanks
Mike