Processing Inbound X12 Architecture Question

, ,
1

Hello all.

I would like to pose an architectural question to you, in the hopes that your feedback might lead me to making a design decision.

I am currently working on a webMethods EDI project for a company that will receive X12 documents from a VAN via EDIINT/AS2 into Trading Networks 6.0.1. The X12 data will then be:

  1. Mapped to a Canonical model
  2. Published to the Broker
  3. A Subscribing Flow service maps the Canonical data to a flat file format.
  4. Deliver flat file data to a Mainframe via FTP.

The design decision that I’m struggling with is deciding which of the following two options is best:

OPTION 1
I can process the inbound X12 data at the Transaction level. This means the following:

  1. Set the splitOption for all EDITPAs to “Transaction”.
  2. Create a processing rule for the Interchange envelope that will invoke a service that will generate a Functional Acknowledgement and send it back to the Trading Partner.
  3. Ignore the Group envelope.
  4. Create a processing rule for each Transaction type (810, 831, 832) which will invoke the appropriate service that will map the transaction content to a canonical model, and then publish.

The downside, in my opinion, to processing at the Transaction level, is that Trading Networks will become cluttered with documents. For example, If a Trading Partner sends a single X12 Envelope with one GS and 100 invoices inside, the Transaction Analysis Window will show 103 entries: 1 EDIINT document, 1 X12 Envelope, 1 X12 Group, and 100 X12 810s. Also, because we’re processing at the Transaction level, each individual transaction will be mapped and published separately, resulting in 100 publishes to the Broker. Now on to Option 2.

OPTION 2:
I can process the inbound X12 data at the Group level. This means the following:

  1. Set the splitOption for all EDITPAs to “Group”.
  2. Create a processing rule for the Interchange envelope that will invoke a service that will generate a Functional Acknowledgement and send it back to the Trading Partner.
  3. Create a processing rule for the Group envelope that will loop over the GS segment, look inside each ST to determine the transaction type (e.g. 810, 831, etc), and then call another transaction specific flow service that will map to the canonical model and append the result to a list of canonicals. This list will then be published to the Broker.
  4. Transactions are not recorded in the Transaction Analysis Window because the splitOption is set to “Group”.

The upside to this path is that Trading Networks will only record entries in the Transaction Analysis Window for X12 Envelopes and X12 Groups. If a Trading Partner sends an X12 Envelope with one GS and 2000 invoices (which happens all the time here), only three entries will appear in the Transaction Analysis Window: 1 for the EDIINT document, one for the X12 Envelope, and one for the Group Envelope. Also, because we’re processing at the group level, all documents of the same transaction type will be mapped and published together, which will result in far fewer publishes to the Broker.

The downside here though, is that each individual Publish to the Broker can potentially contain a lot of data (3-5 MB). In the past, I’ve experienced very poor performance on the Broker when it tries to process really large documents.

What I’m really looking for here is for some of you that have implemented similar projects, to please share your experiences regarded which level is best to process at: Transaction, Group, or Interchange.

All comments, thoughts, suggestions are welcome and appreciated. Thanks.

2

Chris - a couple of thoughts. Just to let you know, my experience is not directly similar to yours, as I have not used the EDI adapter in WM - but I am approaching this from a general TN point of view…

I think Option 1 is conceptually better.

a) I would think publishing the individual transactions is most relevent to your business. I would think having this available in TN would be very valuable information.

b) You are letting TN do the work for you recognizing the transaction type. Make use of this facility - this is a huge out-of-the-box value that TN brings to the table. Taken to extreme - option 2 would result in a complex processing rule - having to manage the different transaction types you wish to support - effectively duplicating what TN already does for you.

c) The issue of too many documents in TN should be manageable through automated cleanup and archiving processes.

HTH

3

Hi Haithem,

I just got back from lunch, during which I mostly convinced myself that option #1 is in fact better, for all the reasons you just mentioned. Reading your response and seeing that it validated my lunch-induced clarity is encouraging! Another advantage that option #1 gives is the ability to resubmit individual transactions from the TN Console. Processing at the Group level would end up being much more of an “all-or-nothing” resubmit.

Thanks for the insight.

4

Chris
I have always used option 1 for similar reasons to your lunch-induced clarity.

LOL

5

It is amazing what a grilled chicken sandwich with bacon can accomplish…

6

Chris,
I do have a question for you. Would you ever need to reject an entire EDI transmission from a trading partner? Meaning if one of the ST’s within the document fail validation, you will want to fail the entire GS? Within option 1 you are not providing that capability.
Marie

7

Marie,

That’s a great point, but fortunately the client only wants to fail documents at the transaction (ST) level. If a GS contains 10 transactions and 2 are invalid, the other 8 should still continue on to the destination resource. So for this particular client, option #1 is a good fit.

8

After much discussion, we decided to go with option 1, as well. While we don’t have quite the number of STs you have in your GSs, we went with this option for all the same reasons. Some of the things we ran into, though, were mostly related to outbound processing.

We are processing all inbound EDI based on the transaction, but we are generating FAs and archiving everything based on the ISA. The GS is ignored here, as well.

For outbound, we are (for the most part) creating a new ISA for each transaction we have. So, we are sending EDIINT out based on the ISA. One thing we ran into, though is using different GS03 values (GS receiver ID) for each transaction group type. We ended up adding a series of extended fields in the partner profiles for GS03 values of each transaction group. Then, before we wrap the ST that we created in a GS and ISA, we look up to the profile to see if the GS03 is available there. If it’s not, we just use the envelope receiver that exists in that TPA.

Another thing we had to do was to create formatServices for EDI R0, R1, R2, etc datatype. All of the Rx values are formatted as decimal (which is incorrect, since it always places 2 decimal places on the end. So, 12 -> 12.00.

The TPAs opened up so much more than what 4.6 had available (which was nothing). I’m just hoping that wM adds at least these two features in the future versions.

9

I have a question for outbound EDI architecture question:

I will get a file looks like this:

00000396696352226 01AN
…
00000396696352227 01AN

00000996696352226 01AN

00000996696352226 01AN

000003 is trading partner … 01AN records means a new invoice …
000009 is trading partner … 01AN records means a new invoice …

000003 and 000009 are using different maps.

So this file shows 2 trading partners with 4 invoices.

Option 1: preprocess the whole files, make it 4 invoices then load into webMethods … send it to TN to send to different process maps

Option 2: Load one files (with 4 invoices) then process inside of webMethods

I think Option 1 is better, please advice. It will be a performance issues if the file is big (with 500 invoices) right.

John

10

Jason,

The big file can be a performance issue depending on a number of factors like server configuration, memory, processor speed etc. You also need to define big. How big a file is your 500 invoices. Total bytes or line items would be a good gauge to determine how big it really is. The one thing that you would need to do is configure the EDI and TN modules to use LargeDocs and by the way if you’re using 6.1 the docs are missing the tnBigDocThreshold parameter in the instructions.
I myself prefer option 2 since each partner would only receive one Envelope vs. option 2 where each invoice would likely be in a separate envelope.

HTH

11

Jason,

If you are expecting lot of Invoices merged in onefile then Option2 would be better and it will be easy to handle each INV at a time and process to TN and reduces the complexity while testing and ofcourse in the maintainence too.

Please register your self in this site,welcome wmusers.com

Jut my thoughts,