Has anyone experienced the server hanging before

Please tell me someone else has experienced these symptoms:

  • IS stops responding to all requests
  • java.exe is in task manager but using NO cpu
  • cant connect from developer, administrator, SAP txns get stayed stuck in Executing state
  • only solution is a full reboot

has anyone experienced this symptoms and if so, what was the cause and resolution??

What WM Version? Java version?

What events trigger the hang?

There was a similiar in 4.01. A patch is available on the Advantage site for this issue.

HTH,
Jim Palmer

4.6 using IBM JVM 1.3.0

no specific events trigger the hang… cant get thread dumps from console(i dont have the -Xrs switch in server.bat) when this occurs…
it occurs at random times and during random transactions… it could be SAP, could be DB i have nothing in logs to point me in either direction and ive had both at 10 since this started happening…

ill check for that patch, but we’re on 4.6 and that should be included in that release…

We JUST had a major hang. It was due to too many threads on our SQL Server Trading Networks database. We had to reboot the database server and that took care of the problem.

That patch is part of 4.6. Wish I had more for you. I think I saw some forum conversation on the SAP adaptor but I can’t be sure.

Good Luck,
Jim P

Check the number of threads alocated (may be need to increase that a bit) and the heap size…

not sure if that helps… but just want you to try that out…

Let us know…

Sharath.

  1. Do you have a firewall between SAP-gateway and IS? If this firewall has an automatic cut off after time out you will get many lost threads in the gateway and IS.

  2. This did only happen with java.exe getting huger: Do you have the repository cleaned very frequently? If not, the repository begins to block IS.

Regards
Josep

I have the amount of threads up to 500 which should be more than enough. Also, dont new requests queue up when this occurs???

We do not clean the repository that often which may be related. I will enable some sort of repository cleansing and see if that helps…

We had similar symptons (w/ plenty hangups), but no SAP adapter use. After analysis found out some threads not being closed/killed by IBM JVM. Changed to Sun JVM 1.3 and the issue never happened

IS hang can be casused by many things but casue is usually same or related. In most cases it is due to exhausted thread pool and all requests are blocked! In Versions 4.6 or earlier you cannot tell exactly how may threads are active. Admin using JVM get active thread API but it does not return a true value. Only reflection that has been true at some point in time. Your IS may infact be out of threads and increasing them will only extend the life but will hang later. Also having threadpool too large like 500-1000 may imact server on some JVM with garbage collection. You need to investigate what porcess hanged threads! Another possibility is IS JVM process is ok but listener manager has crashed that will look like a crash, you cannot connect with anything to IS but process is ok. You can chek this by setting some test scheduled job to run. And chek if its still running after IS hanged.
However scheduler can also cause hang for same resource exhaustion reason. There are severl patches even for 6.0.1. to fix this porblem. I know 4.6 had same issue. If you run scheduler jobs alot make sure you have patch to start jobs on different sessions!

If you’re going to use 6.0.1, and you want to do the test with scheduled jobs mentioned above by Igor, check also the patches for scheduled jobs since there are bugs known that prevent the schedular to pick up new tasks.

We are also using 4.0.1 and 4.6 versions.

Are there patches available for those versions for the scheduler issues such as tasks hanging, not running or new tasks not being picked up?

If there are, please provide details?

I would suggest the following in order:
putting GC on scheduler.
Increase the number of threads for scheduled jobs.
ensure that you have the latest service packs installed on the 4.x versions.
check for any patches/fixes for 4.x versions from wm-support.

HTH.

Thanks for the reply.

A quick question - when I check the resource settings for assigned threads the options are:-

  1. server thread pool
  2. scheduler thread pool

Is there a way to split/change the assigned threads to scheduled jobs and system tasks?

Cheers,

Steve.

We were provided a patch by webMethods back when we were on 4.6 to split the scheduler thread pool and user schedule tasks so that they would have their own thread pools. This patch never really solved the problem of the server getting hung up for us.

I didnt quite get your question.

The Thread pools are different. Just ensure that you increase the thread pool size in ratio.

For ex. defaults are
Server Thread Pool
Maximum Threads 75
Minimum Threads 10

Scheduler Thread Pool
Maximum Threads 10

You can change them to (increase in approx ratio of 150%)
Server Thread Pool
Maximum Threads 200
Minimum Threads 25

Scheduler Thread Pool
Maximum Threads 25

Hope you got the answer to your question.

Hi there,

Thanks for the answer.

It seems that our settings may not be correct as follows:-

Server Thread Pool
Maximum Threads 75
Minimum Threads 10

Scheduler Thread Pool
Maximum Threads 30

What would your recommendation be?

Is there also a limit based on say memory assigned to the BC application?

Thanks,

Steve.

You can change the settings as suggested here:
Server Thread Pool
Maximum Threads 225
Minimum Threads 30

Scheduler Thread Pool
Maximum Threads 30

and see if it benefits you. also, as per my previous post in this thread, do have GC on a scheduler.

how much memory you are running on your IS server? did you find if you are at latest SP on your 4.x server?

Thanks again for the settings - this makes more sense to me than the current values.

We don’t have GC set on the scheduler - how is this set? Do I need to create a new service or is their a default service to use?

We are using MS Win2000 (5.00.2195 SP 4) with 2GB RAM. The BC is version 4.0.1, build 698 + Fixes 1-57, with 1.4GB RAM assigned.

Steve,

For scheduling GC,you have to create a custom java service and put this line System.gc();

And in the ISAdminConsole create One-Time and SimpleIntervalTasks and schedule this service for every 30 seconds interval.

HTH,