WebMehods IS startup long time issue

Hi experts,
I has one issue about WebMehods 6.1 IS startup.
There has old version webMethods about 6.1 and the OS version is Windows 2003.
At 5/22 , i want to shutdown ths IS ,but the shutdown command was not worked in half an hour.Then i forced to shutdown the webMethods services at windows task manager.
I try to startup the webmethods6.1 IS service,then i wait and wait …and at last i find it need a long time to startup IS service successful about two hours.Then i check the serverlog and find there need 106 mins between one step(wm.prt.admin:startup) and next step(wm.prt.admin:brokerPush).
I think this case is not normal,why these two step need 106 mins,please help me resolve this issue.

The servrlogs as follow:
2015-05-22 18:30:39 CST [ISS.0028.0012C] WmDB: Startup service (wm.server.db:startup)
2015-05-22 18:30:39 CST [ISS.0028.0012C] WmPKI: Startup service (wm.server.pki.admin:init)
2015-05-22 18:30:39 CST [ISS.0028.0012C] WmAdmin: Startup service (wm.administrator.repo:centralRepoRegister)
2015-05-22 18:30:39 CST [ISS.0028.0012C] WmAdmin: Startup service (wm.administrator.admin:initialize)
2015-05-22 18:30:39 CST [ISS.0028.0012C] WmPRT: Startup service (wm.prt.admin:startup)
2015-05-22 20:16:08 CST [ISS.0028.0012C] WmPRT: Startup service (wm.prt.admin:brokerPush)

2015-05-22 20:16:08 CST [ISS.0028.0012C] WmIPRoot: Startup service (wm.ip.util:init)
2015-05-22 20:16:08 CST [ISS.0028.0012C] WmRNIF11TRP: Startup service (wm.ip.rnif11.util:init)
2015-05-22 20:16:08 CST [ISS.0028.0012C] WmRosettaNet: Startup service (wm.ip.rn.util:init)
2015-05-22 20:16:08 CST [ISS.0028.0012C] WmRNLargeDocSample: Startup service (wm.b2b.rn.largeDocSample.init:startup)
2015-05-22 20:16:08 CST [ISS.0028.0012C] WmRNIF20TRP: Startup service (wm.ip.rnif20.util:init)
2015-05-22 20:16:08 CST [ISS.0028.0012C] WmRNIF20TRP: Startup service (wm.ip.rnif20.mime:init)

Thanks very much!!

Hi Jianhua,
Good day

Can please try this one .Let me know is it helpful or not .

Hangs After the Line [ISS.0098.9013] Dispatcher Is Not Initialized
This error might occur for the following reasons:
A package startup service, or a scheduled service that uses Broker publishing, is invoked before the Integration Server and the Dispatcher have fully started.
Problems exist with the dispatch.cnf file.
If the dispatch.cnf file is corrupted, back up the file, delete the original file, and restart Integration Server. Integration Server creates a new dispatch.cnf file during the startup sequence.
The Dispatcher has fully started only after the following line is written in the server log:
2005-09-20 08:50:05 CEST [ISS.0025.0036C] Dispatcher started
To resolve this issue, do the following:
For Integration Server 6.0.1, install the appropriate fix.
For 6.1, try these workarounds:
Create a startup service that first calls a Java service that sleeps for a period of time that is longer than the time is takes Integration Server to start up. The startup service can then call the service that you want to run. This will ensure that the service does not run before Integration Server has started.
Write a Java service that monitors the server log and then runs your service after the “Initialization Completed in XXX seconds” line has been written to the log.

Hi ,Murail :
I think it’s not dispatcher.conf error.
I have change the webMethods IS logs level to 5, and then restart IS again and found the more detail database operated error logs between one step(wm.prt.admin:startup) and other step(wm.prt.admin:brokerPush).

The server logs as below:
2015-06-02 12:53:14 CST [ISS.0028.0012C] WmPRT: Startup service (wm.prt.admin:startup)
2015-06-02 12:53:14 CST [PRT.0101.0153V1] Default PRT transport is BROKER
2015-06-02 14:56:59 CST [PRT.0101.0512W] *** Database Storage problem. Exception: [wm-cjdbc33-0069][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 86) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
2015-06-02 14:57:04 CST [PRT.0101.0512W] *** Database Storage problem. Exception: [wm-cjdbc33-0069][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 88) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

2015-06-02 15:33:46 CST [PRT.0101.0512W] *** Database Storage problem. Exception: [wm-cjdbc33-0069][SQLServer JDBC Driver][SQLServer]Violation of PRIMARY KEY constraint ‘PK__PRTQUEUECONTENT__544C7222’. Cannot insert duplicate key in object ‘PRTQUEUECONTENT’.
2015-06-02 15:33:51 CST [PRT.0101.0512W] *** Database Storage problem. Exception: [wm-cjdbc33-0069][SQLServer JDBC Driver][SQLServer]Violation of PRIMARY KEY constraint ‘PK__PRTQUEUECONTENT__544C7222’. Cannot insert duplicate key in object ‘PRTQUEUECONTENT’.
2015-06-02 15:33:56 CST [PRT.0101.0512W] *** Database Storage problem. Exception: [wm-cjdbc33-0069][SQLServer JDBC Driver][SQLServer]Violation of PRIMARY KEY constraint ‘PK__PRTQUEUECONTENT__544C7222’. Cannot insert duplicate key in object ‘PRTQUEUECONTENT’.
2015-06-02 15:34:01 CST [PRT.0101.0512W] *** Database Storage problem. Exception: [wm-cjdbc33-0069][SQLServer JDBC Driver][SQLServer]Violation of PRIMARY KEY constraint ‘PK__PRTQUEUECONTENT__544C7222’. Cannot insert duplicate key in object ‘PRTQUEUECONTENT’.
2015-06-02 15:34:04 CST [ISS.0028.0012C] WmPRT: Startup service (wm.prt.admin:brokerPush)
2015-06-02 15:34:04 CST [ISS.0028.0012C] WmIPRoot: Startup service (wm.ip.util:init)

Can you help to analyse these database error of table ‘PRTQUEUECONTENT’ and which reason can caused that?

Xu,
This looks like there is some issue in database retry logic in the event of deadlock. It is very unlikely that you would get any support on this, as the version you are facing this issue is 6.1.

If you are not using BPM, you can try disabling WmPRT package, and restart/shutdown IS and see if it prevents.

regards
Senthil

Oops… I just saw the other log messages wherein it displays RosettaNet package startup service information, which means you have process models, and you cannot disable this package.

Did you try killing the process instance id? Is it a non-production environment? I understand you have problems while shutting server down, but what are you trying to do here?

-Senthil

Hi Jianhua,
Please apply latest fix WmPRT package and Let me know weather solution worked or not .

Thanks,
Muralidhar reddy

Hi,Senthilkumar
Yes,this server is production environment,and it used to send and receice the RosettaNet message.
Because this webMethods server install as application server and the OS is windows 2003,I can kill the process in windows task manager.

Did you observe any hanging threads quite often which might be responsible for this kind of situation.Do you have logs which gives this info ?

Thanks,