Processing multiple EDI ISA segments in a single file

, ,
1

Hello All:

I do have a scenario where multiple EDI ISA segments are stored in a single file in VAN.

Let us consider for example, different inbounds EDI transactions are embedded in a single file, if i do a actual processing using http client to sent the document to wm.tn.receive service, I am not able to process it.

But if there is only one inbound EDI transaction in the file, then i am able to process the document.

Has somebody come across this situation and please advice regarding the same

thanks,
Ravi

2

Hi,
This totally depends on how you process the EDI file. When you create the EDI record all you have to do is to loop over /Values/ISA and create your own record structure as you need and process it.
We are doing it here successfully.
Hope this helps.
Thanks

3

I know that TN expects all ISA id’s in the interchanges to be linked to the user ID you are signing on with via the profile of the ID your using to send data to tn.receive. If any ISA id’s are not in the profile, they will receive an error. There are two ways around this that I know of, one is to use an ID that has TNAdministrator access (Not good if you are allowing
partners to access from across the internet, they may have access to Administer your TN) or as Rob Eamon pointed out to me the other day, this Ezine article supplies another alternative…

[url=“wmusers.com”]wmusers.com

4

Thanks for the help ,

I have done a work around i have explained the steps below. Please provide your feedback whether it is a proper mechanism.

  • Get Edi file from FTP server
  • convert the byte –> string using bytestoString flow
  • Created a custom java servive to parse the string to string array using “ISA*” as delimiter
  • Loop through the String array
  • submit the inarray string value to the Trading networks using wm.tn.receive through wmpublic.client.http

would like to know everyones feedback on this.

Regards
Ravi

5

Points to be aware of:

  • This will not support large documents, since the entire file is loaded into memory at once.
  • Using “ISA*” as the literal will not support documents that use other field delimiters.
  • Searching for ISA assumes that ISA will not appear in the data, which might be inaccurate.
  • Invoking wm.tn:receive through pub.client:http is an unecessary extra step. Name the string edidata in the pipeline and call wm.tn:receive directly.

That last bullet may actually get you past the security restrictions as security is handled differently for local invokes. Try loading your file, assign it to a pipeline var named edidata, then invoke wm.tn:receive, passing edidata to the node parameter.

6

Rob ,

Do you suggest a alternative for this approach for handling the large document for this scenario.

Will the wm.tn.reciev work without the Content-Type value set to “application/EDIstream”

Regards
ravi

7

The large document handling docs describe how to deal with large documents.

Naming the pipeline var edidata will do the right processing. If you invoke tn:receive, there is no content-type to worry about–there is no http interaction.

8

Rob,

I understood the details regarding the submit to the wm.tn.receive .

Regarding the Large document handling:

The Large document handling would be efficient when the transaction set handled by TN is very large to be handled.

But i think in this scenario. The Entire file from FTP is split into invidual transcations and fed to the wm.tn.receive service.(Individual EDI trasactions identified as ISA segments). we can implement the large document handling for each of the EDI transaction submitted to the TN.

Like if multiple 204,856, are there in the file. Then each 204 and 856 are seperated and submitted to the TN using wm.tn.recieve. Then for each of these large document handling can be setup.

How can we handle the file before submission to the TN. I have done a seperation using delimiter(ISA*). Is there a efficient way to handle large file containing EDI ISA before dividing and submitting to TN.

Reagrds

9

Couple of items:

wm.tn:receive can indeed handle multiple interchanges (ISA-IEA pairs) in a single submission. The documentation describes how it is handled.

An example: 2 ISAs, each with one group (GS-GE), each with one transaction set (ST-SE).

In TN, there will be 3 X12 Envelope (one for the entire doc, one each for the individual interchanges), 2 X12 Group and 2 X12 where is something like 4010 and is something like 850, 210, whatever.

I did a simple test service to verify the behavior:

getFile
…filename=e:/mydir/multiISA.txt
…loadAs=bytes
bytesToString
…bytes=body/bytes from getFile
…map output string to pipeline var named edidata
receive
…node=edidata

The docs were logged in TN as expected.

However, if I pass the same string as the post body to wm.tn:receive via a pub.client:http call, only the first interchange is processed. Not a Good Thing.

Also, the example doesn’t handle large docs–the entire file is loaded into memory. The only technique I could uncover that looked like it would work, causing the data to go through the wM content handlers that would support large docs, was to invoke wm.tn:receive via an http call.

So I tried a variety of settings for the pub.client:http call with no success. Here’s the contents of the service I was fiddling with:

pub.file:getFile
…filename=e:\mydir\myfile\multiISA.txt
…loadAs=stream
pub.client:http
…url=http://localhost:5555/invoke/wm.tn/receive
…method=post
…loadAs=bytes (note this is for the response, not the post)
…data/stream=body/stream from getFile
…auth/type=basic
…auth/user=
…auth/pass=
…headers/Content-type=application/EDIstream
pub.string:bytesToString
…bytes=body/bytes (which now has the response)
…encoding=UTF-8

The result is a 403 Service Error. The response string is empty.

The user/pass I used was an administrator account, so none of the restrictions of the logged in user vs. the doc sender apply.

Anyone have any thoughts on what to try next? Can any wM person jump in with some insight?

10

Great Rob,

It works for me. I am able to process multiple EDI Documnet from a single file. That was a great help. The TN firts identifies the documnet as X12 envelope as you specified:

I have done the following and i am able to retrieve the segregated transaction from the single file(i am having my file i a FTP location)

  1. FTP
  2. MAP the content from FTP to edidata variable
  3. wm.tn.receive (Map the edidata to the receive service input “node” )

In the TN i have imported the documnet type 850 and added a processing rule to narrow down on 850.

In the processing action i have added the service action to perform.

The Flow service i invoked was flow1 with the following input

  1. ByteToString to convert the Byte array to a string
  2. smtp service to send each idenified ISA transaction

for a single file having 12 transactions send in one shot to the wm.tn.receive. The documnet got identified as X12 Envelope and the processing rule for 850 fired and i got 12 mails containg the different transactions.

Thanks a lot.

11

After further twiddling, here’s some more information. The key element is the content-type, of which application/edistream has some special considerations.

I believe that the description of content-type in section “The EDI Content Handler and EDI Content Types” in Chapter 7 of the EDI Module: Trading Networks Component guide is wrong. This section appears to be a simple a copy of “The EDI Content Handler” section in Chapter 3 of the EDI Module: Core Component guide with minor edits.

The EDI guide describes application/edistream as a way to use memory efficiently. Perfect. The EDI for TN guide doesn’t add that using application/edistream when submitting to wm.tn:receive 1) doesn’t work; 2) isn’t necessary. Using application/edistream is only necessary when using the EDI Core Component by itself.

I offer the following evidence to support the claim.

Doing this:

pub.file:getFile
…filename=e:\mydir\myfile\multiISA.txt
…loadAs=stream
pub.client:http
…url=http://localhost:5555/invoke/wm.tn/receive
…method=post
…loadAs=bytes (note this is for the response, not the post)
…data/stream=body/stream from getFile
…auth/type=basic
…auth/user=[user]
…auth/pass=[pass]
…headers/Content-type=application/EDI
pub.string:bytesToString
…bytes=body/bytes (which now has the response)
…encoding=UTF-8

works just fine. Note that the content type is application/EDI, not application/EDIstream as it was in the previous post. I haven’t configured the large doc handling stuff yet but that’s next on the agenda.

The content handlers for application/edi, application/x12, et. al. all put “edidata” var into the pipeline. For all but application/edistream, edidata is a String. So http/ftp clients simply set the content-type to one of the EDI content handler-supported types and viola, edidata shows up in the pipeline. wm.tn:receive supports this.

If the content-type is application/edistream, edidata is not a String but is a java.io.InputStream object. wm.tn:receive does not support this and errors out.

So how can EDI documents with content-type of application/edistream be passed to wm.tn:receive? My best guess is that they can’t. Nor do they need to be. Using the “normal” content-types, the TN large doc handling ought to kick in just fine.

If someone needs to send a content-type of application/edistream, one needs to write a “front-end” service to simply redirect the edidata input stream to the pub.client:http data/stream. A simple service named receiveX12Stream does this easily:

pub.client:http
…url=http://localhost:5555/invoke/wm.tn/receive
…method=post
…loadAs=bytes (note this is for the response, not the post)
…data/stream=edidata
…auth/type=basic
…auth/user=[user]
…auth/pass=[pass]
…headers/Content-type=application/EDI

So application/edistream submitters would invoke not wm.tn:receive but this receiveX12Stream service instead, which re-types the content and passes it on. Of course one can add in some checks to make sure the content-type is right and do the right thing. Ultimately, wm.tn:receive needs to be able to accept InputStream objects.

Sorry for the lengthy post. Hopefully this helps someone. It helped me!! :wink:

12

Ravi: So if you ever need to support large docs, do your getFile, loadAs stream, and invoke wm.tn:receive via pub.client:http with the content-type set to application/edi. That will give you large doc handling (if TN has been configured to do so).

13

Rob,

I discovered what you’re referring to a few months back. I hinted at it when I responded to Sonam’s Ezine but I thought it was me. I spent alot of time trying to get tn.receive to accept the ftp EDI file directly.

My solution was exactly what you said. I created a service SubmitEDI that has an http in it with the content type set to Application/EDIStream. I map the input object edidata to data/string.

Chris

14

Rob,
Thats what i am planning to do, set up the trading networks to do large document handling and simply submit the doc to the TN to handle using wm.tn.receive without the http service.

The Ft
p serviice get will get file and map the content to edidata and passit directly to wm.tn.receive. I am not going to use the wm.clinet.http for this.

Regards
Ravi

15

From my testing, you have to go through the http service to do the submission to wm.tn:receive. If you load the file as bytes, you load the entire thing. Since wm.tn:receive can’t accept a stream object directly, you have to go through the pub.client:http service, which will get the content handlers involved and will get the TN large doc handling facilities engaged. My earlier comment that you don’t have to go through http is only if you don’t need large doc handling.

16

Rob,

I am able to do the processing i will use http for large document handling as you suggested.

While processing i found a interesting factor (Interesting and new to me ) While Submitting a EDI ISA envelope with multiple ST SE segments. The TN segregates the SE-ST paisr as individual transaction with ISA envelope around each.

If there are 10 ST /SE pairs in a ISA envelope then according to my processing rule for the EDI document the processing service gets invoked 10 times. This is fine as it treats each one as seperate transaction

I am going to proceess this EDI trasaction in a Flow service and publish to the EAI via WMenterprise. The EAI will treat this as individual transaction(each ST and SE pair). Is there a way by which the trnsactions in a ISA be processed together.

I understant that it is possible to track them using the GS groupid . Is there any way developer/ TN has a functionality to help this.

Regards
Ravi

17

Right, TN does the de-enveloping for you all the way down to the transaction set (tset) level. If you want to process all tsets in a group, then hook your processing rule to the group envelope instead of the tset. The challenge here is that you’ll then need to determine which tset you have (810, 201, whatever) and invoke the proper mapping service yourself. A little bit more work but doable using the EDI core services to help.

18

Rob,

I have to write multiple transaction set of the same transaction to a file.

EDIdata => FileAdapter

I am doing this using the tset itself but trying to figure out the end of the processing of all the transactions using the wm.tn.view to retrieve the related documents of the particular transaction. Cross verify the related transaction for System status =“Done” or “Done /W Errors” . If all the transaction set have been processed then the flag is set to indicate the end of processing all the transaction and the file is closed.

Regards
Ravi

19

Please keep us posted on the results.

20

I have written a following flow service to handle the document procesing status and it works in the developer tool.

input : bizdoc–> internalid

  1. wm.tn.doc:view (get the related docs)

find the ISA envelope using the doc:view to find all the ISA sets of a file.

After getting all the ISA enveloep id pass through doc.view to retrieve the the Groupid and then the transaction document using the document internal id i find the systemstatus of the document if it is anyone of the status other then DONE/ DONE W/Errors then i set the processing flag to true else set it to false.

But Rob i am facing a problem if i exceute the service from the devleoper then it works if i execute the same service as a processing rule thriugh TN it does not execute.

Is there any ACl restriction when invoking as service.

Regards
Ravi