1/ First of all, increase you log level (caution, it will consume more CPU power). You may found some interesting “No thread available” messages.
2/ Have a look on the messages issued by the JVM : we are launching our I.S. using nohup. Having a look on the nohup.log helped a lot to find out memory outage or JVM crashes.
3/ I’ve built a custom monitoring tracking several I.S. figures in flat file (Memory, # of threads, all JDBC pools, Broker …). A backIt helped a lot to find out why we ran out of resources.
4/ if not already done, don’t use the JVM provided by wM but prefere use the latest JVM provided by your operating system vendor.
kill -3 = thread dump (the I.S. is not stopped)
kill = clean kill
kill -9 = abrutly kill the I.S. (use it only if the I.S is not responsive)