Integration Server - Terracotta transactions

Michele_Biancucci · March 31, 2016, 5:45pm

Hello everyone,
I’m using webMethods Integration Server 9.9 and Terracotta suite 4.3.1. I’m trying to put elements in a Terracotta cache using transactions in order to ensure that either all elements are written (commit) or not (rollback).
I have defined a cache manager and a corresponding cache using Integration Server web console; I’m also using pub.cache:get and pub.cache:put to respectively read and write from/to cache. Finally, I’m using pub.art.transaction:startTransaction, pub.art.transaction:commitTransaction and pub.art.transaction:rollbackTransaction to control a single transaction.
However I don’t seem to get it working since the transaction doesn’t actually start at all:

Launch started: 2016-03-31 16:18:13.187
Configuration name: TestTerracottaCache
Configuration location: /home/octo/Development/workspace99/.metadata/.plugins/org.eclipse.debug.core/.launches/TestTerracottaCache.launch
 
com.wm.app.b2b.server.ServiceException: net.sf.ehcache.transaction.TransactionException: transaction not started
	at pub.cacheimpl.put(cacheimpl.java:164)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.wm.app.b2b.server.JavaService.baseInvoke(JavaService.java:407)
	at com.wm.app.b2b.server.invoke.InvokeManager.process(InvokeManager.java:643)
	at com.wm.app.b2b.server.util.tspace.ReservationProcessor.process(ReservationProcessor.java:39)
	at com.wm.app.b2b.server.invoke.StatisticsProcessor.process(StatisticsProcessor.java:49)
	at com.wm.app.b2b.server.invoke.ServiceCompletionImpl.process(ServiceCompletionImpl.java:243)
	at com.wm.app.b2b.server.invoke.ValidateProcessor.process(ValidateProcessor.java:49)
	at com.wm.app.b2b.server.invoke.PipelineProcessor.process(PipelineProcessor.java:171)
	at com.wm.app.b2b.server.ACLManager.process(ACLManager.java:303)
	at com.wm.app.b2b.server.invoke.DispatchProcessor.process(DispatchProcessor.java:34)
	at com.wm.app.b2b.server.AuditLogManager.process(AuditLogManager.java:371)
	at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:547)
	at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:384)
	at com.wm.app.b2b.server.ServiceManager.invoke(ServiceManager.java:238)
	at com.wm.app.b2b.server.BaseService.invoke(BaseService.java:225)
	at com.wm.lang.flow.FlowInvoke.invoke(FlowInvoke.java:254)
	at com.wm.lang.flow.FlowState.invokeNode(FlowState.java:511)
	at com.wm.lang.flow.FlowState.stepIncremental(FlowState.java:428)
	at com.wm.lang.flow.FlowState.invoke(FlowState.java:331)
	at wm.server.flowdebugger.stepFlow(flowdebugger.java:935)
	at wm.server.flowdebugger.execute(flowdebugger.java:465)
	at sun.reflect.GeneratedMethodAccessor170.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.wm.app.b2b.server.JavaService.baseInvoke(JavaService.java:417)
	at com.wm.app.b2b.server.invoke.InvokeManager.process(InvokeManager.java:643)
	at com.wm.app.b2b.server.util.tspace.ReservationProcessor.process(ReservationProcessor.java:39)
	at com.wm.app.b2b.server.invoke.StatisticsProcessor.process(StatisticsProcessor.java:49)
	at com.wm.app.b2b.server.invoke.ServiceCompletionImpl.process(ServiceCompletionImpl.java:243)
	at com.wm.app.b2b.server.invoke.ValidateProcessor.process(ValidateProcessor.java:49)
	at com.wm.app.b2b.server.invoke.PipelineProcessor.process(PipelineProcessor.java:171)
	at com.wm.app.b2b.server.ACLManager.process(ACLManager.java:303)
	at com.wm.app.b2b.server.invoke.DispatchProcessor.process(DispatchProcessor.java:34)
	at com.wm.app.b2b.server.AuditLogManager.process(AuditLogManager.java:371)
	at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:547)
	at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:384)
	at com.wm.app.b2b.server.ServiceManager.invoke(ServiceManager.java:238)
	at com.wm.app.b2b.server.comm.DefaultServerRequestHandler.handleMessage(DefaultServerRequestHandler.java:119)
	at com.wm.app.b2b.server.HTTPMessageHandler.process(HTTPMessageHandler.java:156)
	at com.wm.app.b2b.server.HTTPDispatch.handleRequest(HTTPDispatch.java:178)
	at com.wm.app.b2b.server.Dispatch.run(Dispatch.java:386)
	at com.wm.util.pool.PooledThread.run(PooledThread.java:127)
	at java.lang.Thread.run(Thread.java:745)
Caused by: net.sf.ehcache.transaction.TransactionException: transaction not started
	at net.sf.ehcache.transaction.local.LocalTransactionStore.getCurrentTransactionContext(LocalTransactionStore.java:101)
	at net.sf.ehcache.transaction.local.LocalTransactionStore.assertNotTimedOut(LocalTransactionStore.java:107)
	at net.sf.ehcache.transaction.local.LocalTransactionStore.put(LocalTransactionStore.java:202)
	at net.sf.ehcache.store.AbstractCopyingCacheStore.put(AbstractCopyingCacheStore.java:78)
	at net.sf.ehcache.store.TerracottaTransactionalCopyingCacheStore.put(TerracottaTransactionalCopyingCacheStore.java:31)
	at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.put(NonStopStoreWrapper.java:820)
	at net.sf.ehcache.Cache.putInternal(Cache.java:1617)
	at net.sf.ehcache.Cache.put(Cache.java:1543)
	at net.sf.ehcache.Cache.put(Cache.java:1508)
	at pub.cacheimpl.put(cacheimpl.java:160)
	... 46 more

Launch completed: 2016-03-31 16:18:13.187

I didn’t find any reference on both documentation regarding Integration Server and Terracotta.
Is there any documentation I’m missing out?
Is there any guide on how to design transactions using Integration Server and Terracotta?

Thanks in advance for your help.
Best Regards

Mahesh_K_Sreenivas · March 31, 2016, 5:53pm

As per my understanding you really do not have to call the transactions (start, commit and rollback) for putting and getting to/from cache. Just put and get will do it.

Michele_Biancucci · April 1, 2016, 11:59am

Thank for your help.
What I actually need is to make sure that I can transactionally write into multiple caches and/or into multiple sources like databases or jms queues/topics. How can I handle put/write errors or failures without transactions? Either all writes are successful (commit) or rollback to the previous safe state. The use of transactions guarantee this particular behavior.

Syed_Faraz_Ahmed1 · April 3, 2016, 2:12pm

I don’t think there will be any such requirement like transnational writing in Cache.

Yes, For db you can make use of WmART services (Start, Commit, Rollback).

“Either all writes are successful (commit) or rollback to the previous safe state. The use of transactions guarantee this particular behavior.”

For this you can make us of DB and put all the insertion operation with in transaction boundary and rollback and commit at once.

Note: For using this operation using webMethods you need to use Transaction Type as “Local_Transaction”.

Regards,
Syed Faraz Ahmed

Michele_Biancucci · April 4, 2016, 11:27am

let me explain better. The functional requirement is that the data must be in terracotta caches. The problem is that the flow can be for example:

put n elements in different caches
put elements in a cache or different caches, remove elements from cache than send a jms message

So, if a single put fails all puts (even previous) must be rollbacked or if a remove fails, all previous interaction must be rollbacked. Futhermore if a jms message send fails, the previous puts must be rollbacked. From this, the need to use transactions.

rmg · April 4, 2016, 5:31pm

Michele,

Yes even I agree with the below and but not sure if the transactional logic can help the Terracotta cache mechanism in your requirement outlined above.

Yes, For db you can make use of WmART services (Start, Commit, Rollback).

Please extra clarify.

HTH,
RMG

Michele_Biancucci · April 4, 2016, 6:30pm

I have a flow like the one in Attachments. So, if a single put in the LOOP fails all puts (even previous) must be rollbacked, and if jms message send fails, the previous puts must be rollbacked.

I read that Terracotta supports Transaction in fact i added transactionalMode=“local” in cache configuration file. More in detail:

I have defined a cache manager and a corresponding cache using Integration Server web console. I have also added the “transactionalMode” parameter in

“ehcache-cache-manager.xml” file placed in “/IntegrationServer/instances/default/config/Caching”

the following is how the configuration file looks:


<cache         
        name="testcache4"         
        maxEntriesLocalHeap="10000"         
        copyOnWrite="true"         
        copyOnRead="true"         
        cacheLoaderTimeoutMillis="30000"         
        transactionalMode="local">        
        <persistence             
            strategy="distributed"/>        
        <terracotta             
            consistency="STRONG">            
            <nonstop/>        
        </terracotta>        
        <searchable/>    
    </cache>

The cache manager started then the configuration is ok.

Then i use pub.cache:put to write to cache like in Attachmemnts image, surrounded by pub.art.transaction:startTransaction, pub.art.transaction:commitTransaction and pub.art.transaction:rollbackTransaction to control a single transaction.

However I don’t seem to get it working since the pub.cache.put raise the following exception:


1.	com.wm.app.b2b.server.ServiceException: net.sf.ehcache.transaction.TransactionException: transaction not started  
2.	    at pub.cacheimpl.put(cacheimpl.java:164)  
3.	    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
4.	    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
5.	    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
6.	    at java.lang.reflect.Method.invoke(Method.java:497)  
7.	    at com.wm.app.b2b.server.JavaService.baseInvoke(JavaService.java:407)  
8.	    at com.wm.app.b2b.server.invoke.InvokeManager.process(InvokeManager.java:643)  
9.	    at com.wm.app.b2b.server.util.tspace.ReservationProcessor.process(ReservationProcessor.java:39)  
10.	    at com.wm.app.b2b.server.invoke.StatisticsProcessor.process(StatisticsProcessor.java:49)  
11.	    at com.wm.app.b2b.server.invoke.ServiceCompletionImpl.process(ServiceCompletionImpl.java:243)  
12.	    at com.wm.app.b2b.server.invoke.ValidateProcessor.process(ValidateProcessor.java:49)  
13.	    at com.wm.app.b2b.server.invoke.PipelineProcessor.process(PipelineProcessor.java:171)  
14.	    at com.wm.app.b2b.server.ACLManager.process(ACLManager.java:303)  
15.	    at com.wm.app.b2b.server.invoke.DispatchProcessor.process(DispatchProcessor.java:34)  
16.	    at com.wm.app.b2b.server.AuditLogManager.process(AuditLogManager.java:371)  
17.	    at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:547)  
18.	    at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:384)  
19.	    at com.wm.app.b2b.server.ServiceManager.invoke(ServiceManager.java:238)  
20.	    at com.wm.app.b2b.server.BaseService.invoke(BaseService.java:225)  
21.	    at com.wm.lang.flow.FlowInvoke.invoke(FlowInvoke.java:254)  
22.	    at com.wm.lang.flow.FlowState.invokeNode(FlowState.java:511)  
23.	    at com.wm.lang.flow.FlowState.stepIncremental(FlowState.java:428)  
24.	    at com.wm.lang.flow.FlowState.invoke(FlowState.java:331)  
25.	    at wm.server.flowdebugger.stepFlow(flowdebugger.java:935)  
26.	    at wm.server.flowdebugger.execute(flowdebugger.java:465)  
27.	    at sun.reflect.GeneratedMethodAccessor170.invoke(Unknown Source)  
28.	    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
29.	    at java.lang.reflect.Method.invoke(Method.java:497)  
30.	    at com.wm.app.b2b.server.JavaService.baseInvoke(JavaService.java:417)  
31.	    at com.wm.app.b2b.server.invoke.InvokeManager.process(InvokeManager.java:643)  
32.	    at com.wm.app.b2b.server.util.tspace.ReservationProcessor.process(ReservationProcessor.java:39)  
33.	    at com.wm.app.b2b.server.invoke.StatisticsProcessor.process(StatisticsProcessor.java:49)  
34.	    at com.wm.app.b2b.server.invoke.ServiceCompletionImpl.process(ServiceCompletionImpl.java:243)  
35.	    at com.wm.app.b2b.server.invoke.ValidateProcessor.process(ValidateProcessor.java:49)  
36.	    at com.wm.app.b2b.server.invoke.PipelineProcessor.process(PipelineProcessor.java:171)  
37.	    at com.wm.app.b2b.server.ACLManager.process(ACLManager.java:303)  
38.	    at com.wm.app.b2b.server.invoke.DispatchProcessor.process(DispatchProcessor.java:34)  
39.	    at com.wm.app.b2b.server.AuditLogManager.process(AuditLogManager.java:371)  
40.	    at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:547)  
41.	    at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:384)  
42.	    at com.wm.app.b2b.server.ServiceManager.invoke(ServiceManager.java:238)  
43.	    at com.wm.app.b2b.server.comm.DefaultServerRequestHandler.handleMessage(DefaultServerRequestHandler.java:119)  
44.	    at com.wm.app.b2b.server.HTTPMessageHandler.process(HTTPMessageHandler.java:156)  
45.	    at com.wm.app.b2b.server.HTTPDispatch.handleRequest(HTTPDispatch.java:178)  
46.	    at com.wm.app.b2b.server.Dispatch.run(Dispatch.java:386)  
47.	    at com.wm.util.pool.PooledThread.run(PooledThread.java:127)  
48.	    at java.lang.Thread.run(Thread.java:745)  
49.	Caused by: net.sf.ehcache.transaction.TransactionException: transaction not started  
50.	    at net.sf.ehcache.transaction.local.LocalTransactionStore.getCurrentTransactionContext(LocalTransactionStore.java:101)  
51.	    at net.sf.ehcache.transaction.local.LocalTransactionStore.assertNotTimedOut(LocalTransactionStore.java:107)  
52.	    at net.sf.ehcache.transaction.local.LocalTransactionStore.put(LocalTransactionStore.java:202)  
53.	    at net.sf.ehcache.store.AbstractCopyingCacheStore.put(AbstractCopyingCacheStore.java:78)  
54.	    at net.sf.ehcache.store.TerracottaTransactionalCopyingCacheStore.put(TerracottaTransactionalCopyingCacheStore.java:31)  
55.	    at org.terracotta.modules.ehcache.store.nonstop.NonStopStoreWrapper.put(NonStopStoreWrapper.java:820)  
56.	    at net.sf.ehcache.Cache.putInternal(Cache.java:1617)  
57.	    at net.sf.ehcache.Cache.put(Cache.java:1543)  
58.	    at net.sf.ehcache.Cache.put(Cache.java:1508)  
59.	    at pub.cacheimpl.put(cacheimpl.java:160)  
60.	    ... 46 more

It seems that pub.cache:put doesn’t detect the transaction.
There is a method to force pub.cache:put to detect the Integration Server Transaction Manager?

Thanks

Holger_von_Thomsen · April 4, 2016, 7:01pm

Hi,

please change the order of steps in your catch sequence.
Otherwise you will not receive the proper error.

pubt.flow:getLastError ist always to be called first in a catch sequence.

Additionally move the commitTransaction Step inside the try sequence.
currently it is placed between the try and the catch sequence.

Regards,
Holger

Mahesh_K_Sreenivas · April 4, 2016, 7:42pm

startTx out side the main sequence.
I am going through your requirement will keep you posted. Stay tuned.

Prasad_Pokala · April 4, 2016, 9:20pm

pub art transactions does not support transaction boundaries for ehcache yet. I have developed a custom implementation of the same few years back. Please find attached and follow the instructions in README file. Let me know in case you face any issues with it.

The same can be downloaded @ Utility to manage transaction boundaries on Ehcache puts and gets – Integrate
EhcacheTransactions.zip (582 KB)

rmg · April 4, 2016, 10:49pm

@Prasad,

Thanks for sharing the new info and it certainly helps the user base here

Thx
RMG

Michele_Biancucci · April 5, 2016, 5:19pm

@Prasad

Thanks for your help. I’m testing your library right now.

EDIT:

I tested the transaction for “local” and “xa” and it seems to work. I have a problem with the test flow in attachements:

I configurated all to use “xa” transaction, but when i simulate exception with EXIT, the put in cache is rollbacked, however the message is sent anyway to the jms topic.

Any suggestions?

Thanks

Holger_von_Thomsen · April 5, 2016, 8:05pm

Hi Michele,

can you share the configuration of your JMS-Connection-Alias please?

Regards,
Holger

Prasad_Pokala · April 5, 2016, 9:52pm

Please use pub.ehcache:startXATransaction service instead of plain startTransaction in case of XA. I had to develop two sepeate services for both the local and XA transaction types due to the nature how ehcache deals with transactions. I am sure JMS send will be rollbacked too if you use startXATransaction. Also make sure you are setting transaction type as XA in JMS connection alias too.

Best Regards
Prasad Pokala

Prasad_Pokala · April 6, 2016, 7:39am

Apologies… my mistake… I have installed older version of my package on IS and referring to that . The latest one I shared with you guys have only one service “pub.ehcache.transaction:startTransaction”. You just have to set “transactionType” as “xa” and make sure transactionType in JMS connection alias is XA too.

Best Regards
Prasad Pokala

Michele_Biancucci · April 6, 2016, 10:37am

Hi,

sorry my fault, i forgot to set “xa” transaction type in JMS connection alias. Now everything seems ok.

Thanks

Prasad_Pokala · April 6, 2016, 11:02am

Awesome