We are using EDIINT AS2 with Integration Server and Trading Networks 4.6 (but are in the process of upgrading to 6.1) and are having an issue with both inbound and outbound transactions for a particular trading partner. There are a few scenarios that can result in a backlog of data resulting in multiple transactions being submitted into Trading Networks almost at the same time. Since Trading Networks is multi-threaded, we sometimes have a transaction that was submitted 2nd actually finish processing 1st. This can be a problem as order/sequence of both sets of inbound and outbound transactions are important. I’m not sure if we want to actually single thread this if we could since the transactions are also very time critical.
For outbound transactions, we have put a workaround in to submit the documents to TN with a pause/delay but this doesn’t guarantee that they will process in the correct order but seems to be working ok. We are also looking into whether the trading partner could submit the documents to TN with a pause as well but are wondering if anyone else has experienced this and if you have any suggestions or better workarounds.
We have the same issue with a RosettaNet process using TN 4.6, where if we receive the transactions within 2-3 seconds of each other the 2nd one can pass the first one and get sent to the backend first. This also gives us problems because the order is important. When I researched it, I was surprised to find that there wasn’t a solution for this and didn’t get any feedback to a similar post that I made 6 months ago. Fortunately for us, this situation occurs very rarely so we haven’t had to worry too much about a solution. I’d like to hear if you find a solution for this. Good luck.
You could use something like a priority queue at each step in your processing workflow. This way you could ensure the order of processing at each stage. You’d have to work out where to store the queue (repository, database, session).
There’s also a sequencer pattern which you could think consider using - which checks the ordered sequence of messages. Fowler has an example in his Enterprise Integration Patterns ([url=“books”]http://www.martinfowler.com/books.html[/url]) . This would probably be overkill, however, as it’s usually used for EAI related matters (machines sending across the network).
or …
To enforce a single threaded model:
You could also spawn a thread for each document being processed and then block on all subsequent processing until it’s completed (see servcies in pub.sync). This seems like a pretty terrible way of doing things, as you point out. I wouldn’t do it, personally.
Code some tricky threading logic to handle the sequencing of messages.
Hi Shawn - you said the sequence of the transactions is important but you also said: “a few scenarios that can result in a backlog of data resulting in multiple transactions being submitted into Trading Networks almost at the same time.”
If transaction order is essential, you can only guarentee it at the client submitting the document. That is because only the client knows which order documents must be submitted in - your server has no way of knowing if certain interim transactions are missing.
I think this is what you’d need to do: (1) First ensure your processing rules are all be synchronous - this sends back a response only after the document is completely processed. Any cases of error should send back a response indicating failure. (2) Next get your trading partner to design their submitting client to submit the documents in order, and wait for a success response from your server before submitting the next document. No pause is needed between submissions, but the document submission process must be serial and the client must retry any failure before proceeding to the next document.
If enforcing these constraints on your trading partner is not feasible, you can approximate what you want by just using TN as a data store, and having a continuous process sort the received documents and dispatch time-delayed documents to your backend. For eg: this process could select unsent documents received at least 15 minutes ago (this takes care of processing unevenness), sort these documents, and dispatch them serially to your backend (using the same constraints as above: wait for a response, then send the next one).
Thanks for the feedback. Here’s a little more information regarding backlogging of data and the order/time criticality.
For outbound transactions, we actually use TIBCO Certified Messaging on our backend to get data to/from the webM IS/TN server so even if there is backlog (due to the network or app being down, etc.) we still have the messages being submitted into TN in the appropriate order but due to the multi-threading we get the described issue.
For inbound transactions, the same scenarios can occur (trading partners app being down, Internet being delayed, etc.) to result in multiple transactions submitted to TN at about the same time. TIBCO will get the documents to the backend system in the appropriate order as long as they are published from webM to TIBCO in the appropriate order. Again, the multi-threading in webM can result in docs getting out of order.
From a timing standpoint, there are scenarios where we need to send an outbound document to the Trading Partner, have it processed and get another document back in 5 minutes or less.
I think we’ll look into the synchronous approach some more and see what that would mean for us and the trading partner. We have talked about single-threading but preferred not to do even if we could in case one transaction errored and held up not only related subsequent transactions, but un-related transactions. Wouldn’t synchronous processing result in the same sort of backlog if errors were incurred?
It seems the only true way to handle this is to have the sending application put some sort of incremental sequence number in the data and have the receiving application look to see what it was before processing. This moves the responsibility out of webM and on to the application. This may or may not be possible - we haven’t discussed it at a detailed level yet. We’ll also look into the Fowler method as described above.
Happy to hear even more thoughts. I guess I’m surprised others haven’t seen this more frequently. Unfortunately for us, it’s becoming too frequent for this particluar set of integrations.
Shawn - about making the processing rules synchronous: open a processing rule in TN Console, select the ‘Action’ tab. Here you presumably have already set a service in ‘perform the following action’ - just select the ‘synchronously’ radio button also.
If one processing rules triggers another, you must set all the relevant processing rules to ‘synchronous’. So for eg, if you have this scenario:
Incoming document A -> rule R1 matches and translates it to ->
Intermediate document B -> rule R2 matches and translates it to ->
Final document C -> rules R3 matches and dispatches it to backend
In this case, rules R1, R2 and R3 must all be set to ‘synchronous’ because they are in the ‘processing path’ the document takes. When your partner posts, he will get a 200/OK only after R1, R2 and R3 have all finished, and document C has been generated and dispatched.
> I think we’ll look into the synchronous approach some more and see
> what that would mean for us and the trading partner. We have talked
> about single-threading but preferred not to do even if we could in
> case one transaction errored and held up not only related subsequent
> transactions, but un-related transactions. Wouldn’t synchronous
> processing result in the same sort of backlog if errors were
> incurred?
Yes, it would. You could mitigate by separating out unrelated transactions into different groups of transactions - that way, if a hold-up occured in one group, it would not hold up other groups. But if one document in a set of related documents has an error, and delivery order for these documents must be maintained, by definition you cannot submit the other documents until the error is resolved.
Synchronous processing/single-threading is often can be quite an acceptable alternative. The important thing is no delays inserted between posts - the server just makes sure processing is synchronous, and the client ensures it wait for the acknowledgement before posting the next document from the group.
Just curious, if a trading partner had a similar requirement and you were using webM to send them documents. How would you setup webM to only submit docs to them once a successful return code (AS2) was sent back to webM? We aren’t currently executing a service, we just use a delivery method to send the document to the partner.
> How would you setup webM to only submit docs to them once a
> successful return code (AS2) was sent back to webM? We aren’t
> currently executing a service, we just use a delivery method to send
> the document to the partner.
I assume you’re using an inbuilt TN delivery service – For eg: the ‘HTTPS delivery service’ submits a document, waits for a 200/OK, then retries if it didn’t receive a 200.
TN delivery services are multithreaded. If I recall correctly, the following WM directive controls the number of delivery threads: watt.tx.jobThreads=n
You could set this param to ‘1’. However, this is not a good solution since its a global setting and narrows down the pipe for all partners.
While you can write custom TN delivery services, these are also controlled by the thread parameter above.
How about this? : Convert from a ‘push’ solution to a ‘pull’ solution.
Instead of submitting documents to your partner’s server, have your partner invoke a service on your server - lets say, this is a service called ‘getNextTransaction(ticketID)’. Each time ‘getNextTransaction’ runs, it returns two things to the remote caller:
The next transaction enqueued for that partner
A ‘followingTicketID’ value that identifies the following transaction
Your server would also track the transactions the remote partner had successfully retrieved. This service could also be called with a null input - it would correctly return the next enqueued transaction/followingTicketID to the partner. (This is useful for startup, and in some cases where the remote server crashes and loses state somehow).
You can implement a symmetrical solution for inbound documents -have your partner write a similar remote interface, and ‘pull’ documents from them, rather then having them submit it to your server.
Further to this, a refinement of the pull solution above could be to seperate document-ID retrieval from document retrieval as below:
SERVICE # 1: getNextDocumentID
Input: null
Output:
nextDocumentID - string uniquely identifying next document to retrieve
Description:
This service returns an ID that uniquely identifies the next document that must be retrieved. By ‘next’, we mean that this service issues the IDs in ‘correct document order’ and maintains the ‘state’ of which ID it sent back the last time it was run. This service completes relatively quickly (compared to the getDocument service).
SERVICE # 2: getDocument
Input:
documentID - the document ID to retrieve
OR
null - nothing passed in
Output:
nextTransactionID - the content of the ‘next document’
Description:
Given a document ID, this service retrieves a document. This service completes relatively slowly compared to the service above. However, the ‘pulling’ server has the option of either invoking this service serially or running multiple retrieval threads in parallel. In the latter case, the pulling server will need to re-order the retrieved documents, which should not be a problem since it has an ordered list generated by calling getNextDocumentID.
This service maintains state of ‘documents retrieved’ as well (note, this is different from the state maintained by service #1). In case input is null, this service returns the next document that hasn’t been transmitted (this is needed for initialization, and maybe for crash recovery).
The above assumes proper authentication and isolation of different partners from each other, etc.
I have several TIBCO opportunities available in various locations around the country. Both contract, contract to hire and permanent positions available. The positions range from developers to management positions. If interested, please send me your updated resume in word format, along with contact details and the best time to call you. carol.bell@globalitresources.com
Actually this forum is not related to TIBCO world,purely webMethods related.So i believe please post the same in the TIBCO forum you will get the quick replies.
I will do that. I’m new to this user group, and am just learning how to post here. Thank you for your patience and advise. Although I don’t have any at the moment, I often do get positions in requiring webMethods.