Sometimes the searchTask API call fails with a Read Timeout Exception generated on the IS logs. Going through the MWS logs, i do see the call is logged and is completed.
2016-08-09 22:00:05 EDT (task:INFO) - Query All Tasks Task Search Query [max:200] searchTerms: {#{currentTask.taskData.myTask.employeeID} = 12345; name = My Task; status = active}
2016-08-09 22:01:08 EDT (Framework:INFO) [WS:360061] - WebService Request: "command/searchtasks_ws" completed in 62721
This, to me means that MWS is able to fetch the task (if any) from the database, and return the same back to the service call. On the IS however, i see the timeout exception:
electric.util.WrappedException: java.net.SocketTimeoutException: Read timed out
at com.webmethods.caf.wsclient.proxy.impl.WSClientDynamicProxy.getCompatibleException(WSClientDynamicProxy.java:496)
at com.webmethods.caf.wsclient.proxy.impl.WSClientDynamicProxy.invoke(WSClientDynamicProxy.java:257)
at $Proxy32.searchTask(Unknown Source)
at pub.task.taskclient.searchTasks(taskclient.java:194)
at sun.reflect.GeneratedMethodAccessor18415.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.wm.app.b2b.server.JavaService.baseInvoke(JavaService.java:443)
at com.wm.app.b2b.server.invoke.InvokeManager.process(InvokeManager.java:643)
at com.wm.app.b2b.server.util.tspace.ReservationProcessor.process(ReservationProcessor.java:41)
at com.wm.app.b2b.server.invoke.StatisticsProcessor.process(StatisticsProcessor.java:44)
at com.wm.app.b2b.server.invoke.ServiceCompletionImpl.process(ServiceCompletionImpl.java:243)
at com.wm.app.b2b.server.invoke.ValidateProcessor.process(ValidateProcessor.java:51)
at com.wm.app.b2b.server.invoke.PipelineProcessor.process(PipelineProcessor.java:171)
at com.wm.app.b2b.server.ACLManager.process(ACLManager.java:276)
at com.wm.app.b2b.server.invoke.DispatchProcessor.process(DispatchProcessor.java:30)
at com.wm.app.b2b.server.AuditLogManager.process(AuditLogManager.java:363)
at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:547)
at com.wm.app.b2b.server.invoke.InvokeManager.invoke(InvokeManager.java:386)
at com.wm.app.b2b.server.ServiceManager.invoke(ServiceManager.java:234)
at com.wm.app.b2b.server.BaseService.invoke(BaseService.java:194)
at com.wm.lang.flow.FlowInvoke.invoke(FlowInvoke.java:324)
at com.wm.lang.flow.FlowState.invokeNode(FlowState.java:584)
at com.wm.lang.flow.FlowState.step(FlowState.java:444)
at com.wm.lang.flow.FlowState.invoke(FlowState.java:409)
at com.wm.ap...<truncated>
I tried increasing the timeout parameter on the WmTaskClient package to 2 mins (from the default 1 min), but the issue still persists, (it does not occur all the time, but it does occur). Is there some other configuration parameter i should be looking at ?
There is a KB article which points to same error as in your post. See if this helps:
If MWS has the JVM option -Dtask.prt.audit.enabled=true when a task which belongs to a business process is created, MWS will audit records on the ProcessAudit database.
ProcessAudit database is unknown to MWS, it will try to obtain the database info with the following order:
1- If db.xml contains a datasource named ProcessAudit it will be used.
2- If 1 is not found, check if same schema for MWS contains ProcessAudit data, by executing the SQL: SELECT INSTANCEID FROM PRA_PROCESS WHERE INSTANCEID IS NULL
3- If 1 and 2 has no results MWS will execute service WmMonitor/wm.task.taskclient:getProcessAuditJDBCConfig from the IS defined under MWS > Applications > Administration > My webMethods > System Settings > TaskEngine
During the migration db.xml will be updated to same URL as MWS, if ProcessAudit is defined you will need to double check if the JDBC URL and username used are correct for the new ProcessAudit database, otherwise change the value manually with proper information.
pub.task.taskclient:searchTasks service returns data based on the search criteria. If the search criteria relatively returns smaller set of data, it may be quick but if it returns huge number of records, this service will take more time. Also, this API examines all business data in all the tasks in the inbox. Increasing timeout would be a temporary solution as it may affect when returned dataset is huge.
pub.task.taskclient:searchTasksIndexed is an API that searches only on indexed fields. In your task data, you will specify which fields to be indexed (again, not all but the fields that really need to be indexed based on use-case). This API will work much better as it uses DB index.
This goes back to implementation phase but if that is an option, you should consider.
To start with, I suggest you look at MWS performance (one of the built-in hidden tool)
By enabling this portlet, you can see many parameters one of which is the Task API’s time taken, DB query time taken etc., This would give you an indication to find root cause
Thanks for asking. I’m fine. Hope you are doing well. Long time
Appreciate your response and found definitely useful. In future during upgrade program, I will take it up to change the API to index one for better performance as suggested.
Also I verified few options in MWS diagnosis but will go through more on the same and see if there is an option.
I feel somewhere the memory/thread is not getting released and causing this random occurrences like once in 4 days. So I will forward the support team all this analysis.
Whats the wsclient-socketTimeout? You can try and increase it until adopting the better performing API. You can check what is the setting at global level under: MWS > Folders > Administrative Folders > Administration Dashboard > Configuration > CAF Application Runtime Configuration → Configure Global Defaults
If you suspect a memory issue heapdump is the best option to validate the suspicion.
Have a nice day!
You could try calling the TaskClient API directly in the IS with the same parameters to check if the time it takes to return the answer is as long as the one you are experiencing in the MWS.
Also, raising the log level for the WmMonitor sections might help you measuring the response time.
I have seen that sometimes WmMonitor does return rather quickly but the amount of the data retrieved has the MWS taking a long time to render.
Monitoring the MWS internals (for instance, enabling its JMX port and using the Java Mission Control or VisualJVM tools) might help you understand what is going on.
Probably increasing the MWS’s JVM max reserved memory might help, but it is better to check what the above tools tell you.
