Service thread usage in two similar IS setups

,
1

We are running WM 6.5 SP3 on two different IS instances on RedHat Linux (say A and B). The work done in these instances is similar, however, instace A has some scheduled tasks that instance B does not.

There is a significant difference between number of threads running on A vs. B (B has a lot more).

A shows:

                    Current Peak

Service Threads 9 131
System Threads 165 208

B shows:

Service Threads 28 556
System Threads 347 380

I took a thread dump from each and browsed through them. The significant difference between the two seems to be the extra “Service Thread Pool” items that look like this:

“Service Thread Pool” prio=1 tid=0x08404988 nid=0x5fbe in Object.wait() [0x602bf000…0x602bf198]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:429)
at com.wm.util.pool.PooledThread.run(PooledThread.java:103)
- locked <0x877bd808> (a com.wm.app.b2b.server.TMPooledThread)
at java.lang.Thread.run(Thread.java:534)

How can I tell what these threads are doing or investigate why these two IS instances have such a significant difference in number of threads?
ThreadDump492-20081006.log (79.6 KB)
ThreadDump491-20081006.log (189 KB)

2

Mehdi: Once a server thread is allocated to do some work and completes, it is returned to the server’s thread pool and remains in a wait state, ready to be used by a future service invocation. In this way, the size of the thread pool always represents a “high water mark” of server activity. The snippet from the thread dump that you included shows just such a thread. It is waiting in the server’s thread pool.

The presence of many threads in the thread pool does not necessarily indicate a problem. the sever is designed to work this way. However, if your two servers are clustered and you expect them to share the load equally, this could indicate that your load balancing is not distributing requests equitably. If not clustered, it just indicates that server B experienced an event (such as a burst of client requests) that triggered the creation of many threads.

If the size of the thread pool is fairly constant, you won’t be able to tell where these threads came from. If can catch it while the thread pool is growing, take a couple thread dumps then, and check the running services on the Administrator browser interface. This should give you an idea of the cause of the growth.

HTH,
m.