We are facing tough time in our production environment with the integration servers hanging frequently. Every time it hangs, we see the below exception in the server.log:
“(58009/-4499) [jcc][t4][2055][11259][3.61.65] The database manager is not able to accept new requests, has terminated all requests in progress, or has terminated this particular request due to unexpected error conditions detected at the target system. ERRORCODE=-4499, SQLSTATE=58009”
Any suggestions for the above will be helpful. Let me know if any extra info is needed.
We’re not heavily using the Audit DB and there are no issues detected with the Audit DB, rather we see that the above is occurring with regard to DB2 (Application Database). But DB2 team confirms that there is nothing wrong on their side during all these times.
Server logs do not show any useful information (than the above information corresponding to each Query failure).
Bad part is that, the frequency of the server hanging is so high that, it happens at least once or twice in a week. I’m still trying to understand whether the error message I posted above is a cause of the issue or a symptom of the issue.
We’re in the verges to move onto 9.5 environment, however, wanted to make sure we don’t get into this problem in the upgraded environment.
I will be glad to provide any other information that you may need to get into the right direction of root cause.
IS712 happens in many cases if the env wasn’t set right and not update to date patch level/config params tuning etc…
But can you set these entries if you are not using audit db and sessions db turnoff in the extended and restart IS?
Always test this in your pre-prod env.
watt.server.auditlog.gd=false
watt.server.auditlog.session=false
watt.server.jca.connectionPool.thresholdWaitingRequest=20 (tune this value based on your 712 environment volumes to start with)
Hope you are not seeing any memory/java core heap dumps in the IntergrationServer directory.please check it also.
Hi Siva,
Next time you see this hung state, take a thread dump by providing the process id of IS instance which will be of great help to find the root cause. Also check how many connections are active/inactive with db by checking with db team, and see if there is any limit on the number of connections that can be established with the application db you are connection with.
I guess,it might be due to missing of jar files especially, db2jcc_license_cisuz.jar. Kindly also request SQL server DBA to put a trace for a couple of days to find how many requests are arriving to DB and how is db response at all intervals. Kindly update forum with latest findings.