B2b standard to reconcile documents transferred

, ,
1

Is there a B2B standard to reconcile transferred documents? Somewhat like the “Here are your document transfer stats for May 2003” type of email generated by VANs and B2B Exchanges, but automated.

There are cases when we or our partner has a serious problem and we basically have to sit down, identify the problem time period, and manually reconcile each document transferred during that period.

So I’m looking for something that enables two companies, say ‘X’ and ‘Y’, to have the following conversation fully-automated?

X: “We sent you ‘n’ POs with unique ids: {…}. Please verify”
Y: “We confirm receipt of ‘n-2’ POs with unique ids: {}. We did not receive 2 PO with ids: {}”
X: "Acknowledged. Escalating to administrator”

2

EDI uses “Functional Acknowledgements” for this.

X: Send ‘n’ POs with unique IDs.
Y: Got these POs: {…}
X: Reconcile Y’s list with the original list.
X: Escalate missing IDs to administrator.

I believe version 6 of the EDI for TN adapter provides a level of reconciliation in that it will mark the outbound docs with an indicator that an ack was received. For docs with no ack indicator, you’ll likely need to write something to identify those.

For previous versions of the adapter, you get to do all the work yourself. Same for non-EDI traffic (I’m not aware of any other adapters/components that perform reconciliation–please chime in if you know of one).

All of this would happen “on the fly” rather than “on demand” as you’re asking about.

Not sure this info is helpful but thought I’d toss it out there anyway.

3

Thanks Rob - that is helpful

> Same for non-EDI traffic (I’m not aware of any other …

In our integrations we mostly do XML over HTTPS. We treat the HTTP 200 OK response as a sync ack for the document transfer. So failed deliveries in the TN task pane are how we “escalate to administrator”. Would you agree this process is similar in functionality to sync acks in EDI?

However, this is not an explicit ack, and we have come across problems with customer who sent us a 200 response, and then had a failure on their system lose the document. Till recently (TNS FIX 30) TN would do the same - send back a HTTP 200 from wm.tn:receive when the TN database was offline.

> All of this would happen “on the fly” rather than “on
> demand” as you’re asking about.

Yes, I was hoping for an industry standard. But neither xCBL nor cXML seem to support one. Maybe ebXML?

In fact a problem is that within each document, something must be unique and this “something” can vary from customer to customer. Not too long ago, we had a customer who insisted on sending a “service order” – defined as a sort of “repeated order” on their system with different orders having the same PO number. This meant our TN duplicate checking, which is based on the customer’s PO number being unique, could not be used for their orders.

I guess, a reconcilition protocol may well take the following form

X: Sent ‘n’ POs in ‘t’ timeframe,
unique IDs: {}
ID path: “/a/b[0]/vv”

Y:
Confirm ‘n-1’ POs
Cannot find 1 PO with ID {}

X: Thanks, escalating missing IDs to administrator

4

Sonam,

We experienced this some time back with mainframe systems and the requirement to provide reconciliation of the transmission of the message.

We also used the HTTP status of 200 to infer that the message made it safely. This backfired on us because it only provides visibility to the landing zone on the receiving system and didn’t actually provide end-to-end visibility.

The solution we implemented at the end of the exercise was to wait for the processing of the message by the receiving system. Once the message processed in the backend legacy system, it generated an “ack” (edi 864) containing either an ack message or an error message depending on the actual system status.

From your perspective, it seems that you are looking to integrate business logic in with your transport mechanism (hence the use of 200 to generate an ack message) when it seems that the requirement is at a biz level. The ebXML standard provides a really good implementation for this, but it will require work on all sides to completely implement.

Also, look at the AS2 (EDIINT), to find the Message Disposition Notice (MDN) which is equivalent to the EDI 997 message but contains greater detail.

For what it’s worth, we used the EDI 864 TEXT Message to provide the business level acknowledgement because it’s open text without any qualifications.

5

Using only the HTTP return code is risky, as has been pointed out. An explicit ack, via an immediate response doc or subsequent doc (this is the EDI approach via 997 docs) is really the only reliable way. The receiver of a doc should return a 200 only if it can guarantee that it has accepted responsibility for the doc one way or another.

We use a front-end service for TN receive, which receives the doc, invokes receive (or routeXml depending on what we’re doing), then checks the Persisted? flag in the resulting bizdoc. If not persisted, we return an error.

There are two levels of acks here, as Ray pointed out. A comm level ack and biz level ack. EDI 997s are essentially a comm level ack, though they also provide info on syntax checks. Receiving a 997 indicates only that the doc has been received for processing. Not that it has been accepted. Not that it can be processed without error. This is where reconciliation or “non-repudiation” usually takes place. “I have a record of sending that doc to you and I also have the functional ack from you saying that you got it.” EDI has both levels. It seems that cXML, xCBL, et. al. focus on the biz level (PO acks, INV acks, etc.).

I would offer that you keep comm acks and biz acks separate and always design for both.

6

Thanks Ray and Rob - So there goes my search for an “asynchronous document reconciliation standard” Well, OK.

As far as our use of HTTP 200 goes, the HTTP protocol does defines the meaning of a HTTP 200 response to mean ‘submission went well’ (or something like that). So we tell our partners to send us a 200 to acknowledging receipt of our document. It’s worked well so far.

> We use a front-end service for TN receive, which receives
> the doc, invokes receive (or routeXml depending on what
> we’re doing), then checks the Persisted? flag in the
> resulting bizdoc. If not persisted, we return an error.

We do something similar to what you do - we recognize the document, submit it through routeBizdoc, then check if the sender /sender/CorporationName was ‘Unknown’ or null.

Also, with TN FIX 30, TN now causes all TN services (incl. wm.tn:receive) to throw an exception (and hence a HTTP 403 error) if the bizdoc isn’t persisted.

> got it." EDI has both levels. It seems that cXML, xCBL, et.
> al. focus on the biz level (PO acks, INV acks, etc.).

Yes, xCBL has a seperate PO Ack document sent seperately after the PO is received and processed by the backend; but cXML does not. But both standards, (cXML natively; and xCBL when it is transported in CommerceOne envelopes) allow for a proper ack document in the sync response to the HTTP post.

> This is where reconciliation or “non-repudiation” usually
> takes place. "I have a record of sending that doc to you
> and I also have the functional ack from you saying that
> you
Hmm. Public key crypto is required for non-repudiation to stand up in court. But if buyer and seller both agree and trust each other, a HTTP 200 can basically serve the same purpose as a sync ack document – bytes are bytes after all.

7

Sonam,

If all you care about is: “Did my document get there?”, then using 200 is perfectly acceptable. However, once it’s there does the receiving system take any action? It would be nice to know if the 20,000 feet of lumber that’s been ordered is on the way! It would even be nicer to know that the PO submitted was rejected due to insufficient credit and that a phone call is required. A 200 won’t suffice if this is the case.

Really, it’s the business that decides what is necessary and up to us to make it happen.

EDI is perhaps one of the most mature data constructs in use today. Coupled with a VAN, it’s hard to beat.

AS1 is EDI via mailbox paradygms using a VAN (Value Added Network such as GEIS). AS2 is the bi-directional transmission of EDI structured documents using TCP/IP. There are some key changes in the business logic for those who are switching to AS2.
A good read on this technology is the mandate that the World’s largest retailer made to their 10,000+ Vendors. United States based Walmart has mandated that ALL vendors that wish to continue to do business with them will adopt the AS2 standard. Nice to have the big hammer.

Cheers.

Ray

8

I have seen some cases where businesses wait so that the latest technology can be incorporated into the computer systems that they have.
I think this is not very right. Businesses should be driving the technology viewpoint. Technology should serve business, not otherwise.
And when business takes upper hand then whatever works for the parties involved should be sufficient.
And in Sonam’s case, you can create a scheduled service that sends an email at the month end to the contact person at the client’s side including the number of files received and the details.
Have a good day!

9

This thread started out with my question about an automated reconciliation standard – it’s really forked off into a different direction since then.

Ray:

> If all you care about is: “Did my document get there?”,
> then using 200 is perfectly acceptable. However, once it’s
> there does the receiving system take any action?

But obviously there are processes in place to handle that – for instance, an xCBL 3.0 OrderResponse document generated by the backend. As for the controversy with HTTP 200 - hey, it can serve as an ack as long as both parties agree. And once the remote party has acknowledged receipt, the rest – processing/ async ack-ing - is their responsibility.

I do agree with you and Rob about the maturity of EDI generally – I recently had a running tiff on the xCBL discussion lists about xCBL.org releasing xCBL 4.0 incompatible with xCBL 3.x. B2B standards were supposed to have built upon the maturity of EDI – not change so fast. I also agree that the VAN/postbox paradigm is valuable – especially when servers at the either end are unreliable, as well as for consolidating management of transaction flow. We exchange EDI, as well as xCBL, via GEIS and a few other VANs. However, there are business factors that pull in an opposing direction from VANs and balance must be reached.

Jones:

> I have seen some cases where businesses wait so that
> the latest technology can be incorporated into the
…
> Technology should serve business, not otherwise.

I agree – in fact, we are all corporate servants of some sort here.

> you can create a scheduled service that sends an email at
> the month
Since we get weekly “transfer summaries” from the VANs we deal with, an email transaction report was the first thing that came to mind (see my comment at the top of this thread).

However, these email reports are useless until someone reports a problem (generally a user, or accounts). Until we know what to look for, no one has the time to go through a transfer report line by line and manually reconcile with the backend in the off chance of a problem being there. The obvious solution is to automate the email checking process, but unless there is a standard, we have to build a custom parser for each party generating a report.

But as it happens, there isn’t a standard - so I guess emailed reports are the only option now. [ Grin ]