I believe you are on the right approach of the handling large XML streams the flow steps you mentioned below.
Also some parameters for processing the large XML data would be based on your IS/JVM memory heap configurations etc… Could slower the run-time performance you are seeing.
So is your test environment is better configured to handle these large XML payloads and thresholds?
but i do not own a terracotta license. Can i still use advantages of Enhanced Xml Node ?
When i try to activate only Cache (no BigMemory) under IS → enhanced XML parsing i got
the message “License key file D:\SoftwareAG\webMethods96\common\conf\terracotta-license.key does not exist or cannot be read”
We are using SAP GTS. We must upload XML Sanction Party Lists (SPL) provided by a Data Provider into the System.
We would like to use a standard webservice from SAP.
The main problem is that the SPL File looks like this.
The webservice expects the same structure, except the fact that it need the additional element CONTROLLER.
(see screenshot)
Also all empty fields have to be set to zero. But the pipeline “forget” these empty elements.
So i have to through the Xml document and set the CONTROLLER element and some validations on the fields.
Yesterday i replaced appendToDocumentList with addList from PSUtilities. But there is no real performance boost.
So i’m searching for the clue what makes my current implementation so slow.
For a 19mb XML the service runs 10min. if i execute savePipeLineToFile at the end of the service the file has about 700mb.
Yes I believe appendToDocumentList does takes lot of memory to hold the list temporary and original list and probably writing the XML to the disk or using customer java streamer could be fast again its all depends on the IS handle for this kind of read/write large XML payloads.
Let us see what other expert users chime on this particular issue.
hope you have create a WSD (type Consumer) for the SAP WebService WSDL.
Then you will find some DocumentTypes in it with the required structures.
Here you can also check if the candidates for “empty” fields must exist or not and if they can be set to null (xsi:nil) or not.
When you have read in the payload as XML String convert it to the matching DocType reference by using build-in services under pub.xml:*.
Map this Document to Input-DocType of the WebService and aad the missing structures.
You can use pub.schema:validate to check if the document matches the doctype
Then call the WebService with this Input-Document.
This is approach should be quite fast excpet for reading in the payload which depends on the size of file.
See IS Build-In-Services guide for informations about the mentioned services.
Can you share a screenshot of your flow?
We will try to find out then where the service spents the time.
I was under assumption Inbound receives the XML payload and parse it for the mapping to a different target document reference structure and if this is not correct then Holger’s advise will do the trick of invoking the SAP WSDL and process the XML data to the WS end point.
i found my issue. I had used 3 Xml Iterators for the same file.
First i selected the with the iterator the first segment, then the last one.
After that i read all other segments. (approx. 13.000).
I changed this. Now i am using 1 iterator with selection criteria for
the needed 3 segments.
Before i had runtime approx 30min for a 36mb xml file. After the change
the runtime is 45 seconds. That is very high performance boost.
Although i could not explain technically why this happened.
So only the problem with the empty WSDL date fieds remain.
In Designer i could not change the date field to not required or optional.
Even if is set them to 0000-00-00 the webservice is not called cause the
validation fails.
So what it is trick to allow empty date fields? I attached the WSDL file.
Thanks a lot for your great support and help so far.
