My Integration server keeps restarting even after I stop the server.
My requirement is to keep the server down forever as we don’t require it for now.
But as it keeps restarting , there is extra manual work of stopping the server daily. Since this is not the case with just one server, I need to find out the reason as it will save my lot of time of daily checking on those servers and stopping them.
can you explain how you stop your server as there are several possibilities to do so with different outcome?
Some of them will auitomatically restart the server and some of them will not.
Is the server running as a Windows service or as an application?
Can you provide the wrapper.logs from profiles/IS_default/logs?
Interesting and What is your server version (hosted OS on) and you said shutting down manually and can you tell or do you know when that server is starting up per the IS/wrapper logs?
Please post here with more analysis details to resolve this kind of issue.
Hello Holger,
Thank you for coming back on this.
As I already mentioned, I wanted the server to be always down. So I am just stopping the server. It is getting started itself(dont know how).
I am stoppping the server via unix box where we first remove the LOCKFILE generated in the IntegrationServer directory. Then kill the instance running on specific port(port which needs to be down).
It gets start itself.
Runnning as an application.
Could not find any wrapper.log under profiles/IS_default/logs.
Found under /common/runtime/logs but with ‘0’ file size.
Interesting and What is your server version (hosted OS on) and you said shutting down manually and can you tell or do you know when that server is starting up per the IS/wrapper logs?
Please post here with more analysis details to resolve this kind of issue.
in this case the wrapper.log is located under profiles/IS/logs.
Removing the LOCKFILE and killing the Unix process is not the recommended way to shutdown the IntegrationServer.
You should either use shutdown.sh from IntegrationServer/bin, the shutdown link in IS Admin UI or just remove the .anchor file from profiles/IS/bin.
This informs the wrapper the IntegrationServer should not be restarted after shutting down.
LOCKFILE will be automatically removed at the end of the shutdown procedure.
Wrapper will even take care of the fact that the IS could not be shutdown correctly by killing the IS Unix process after certain time.
When just killing the Unix process for the IS the wrapper assumes that the IS crashed accidentally due to some JVM error and restarts it immediately.
I would also take a look at your cron jobs. It’s not uncommon for people to create auxiliary scripts to monitor the Integration Server, and if the script detects that the server is down, it will automatically start it back up.
in this case the wrapper.log is located under profiles/IS/logs.
Removing the LOCKFILE and killing the Unix process is not the recommended way to shutdown the IntegrationServer.
You should either use shutdown.sh from IntegrationServer/bin, the shutdown link in IS Admin UI or just remove the .anchor file from profiles/IS/bin.
This informs the wrapper the IntegrationServer should not be restarted after shutting down.
LOCKFILE will be automatically removed at the end of the shutdown procedure.
Wrapper will even take care of the fact that the IS could not be shutdown correctly by killing the IS Unix process after certain time.
When just killing the Unix process for the IS the wrapper assumes that the IS crashed accidentally due to some JVM error and restarts it immediately.
Regards,
Holger
Hi Holger,
Appologies for the confusion here.
The procedure I mentioned is done in case of RRT servers and that also when the shutting down is not happening properly.
For prod servers I follow the one you mentioned i.e. running the shutdown.sh script under bin path.
I had checked under the path /profiles/IS/logs also, but did not find any wrapper.log being generated.
Infact I even tried to check the path configgured for wrapper.log to be generated in wrraper.conf file and that also directs the same path i.e. /profiles/IS/logs/wrapper.log . But when I got there, I dont find any such file.
Also I tried running a find cmd to check all the possible location for wrapper.log file and thats when I found it under the path common/runtime/logs/ but again when I go there find the file with name wrapper.log but with 0 byte size.
do you have the nohup.out available for this IntegrationServer?
Eventually this one provides us more informations.
Are you able to identiy why the shutdowns is not happening properly?
When using the shutdown.sh from either IntegrationServer/bin, profiles/IS/bin or just deleting the anchor file under profiles/IS/bin the shutdown should be completed successfully.
Since you’re not finding the wrapper.log file, I’m guessing you’re not running the server inside the Tanuki wrapper. However, just in case you are, you should know that, as Holger has indicated, the wrapper will take care of restarting the server if it thinks the server crashed.
This is typically handled/configured by the Tanuki wrapper’s on_exit property. You should be able to find this property in your wrapper.conf and custom_wrapper.conf files. Play around with these properties to see if they are the reason why your server is restarting automatically. If you find out that after tweaking these properties, the server stopped restarting automatically then one of two things:
(a) The server is still not being shutdown properly
(b) There is a bug
When I am trying to stop down the server using shutdown.sh script from bin path, below server.logs are been generated and also I could not see server going down.
Please have a look:
bash-3.2$ tail -f server.log
2017-07-01 23:45:29 BST [ISS.0098.0042I] Successfully reconnected to Broker. Sto
2017-07-01 23:45:59 BST [ISS.0098.0041C] Unable to connect to Broker. Starting t
2017-07-01 23:46:09 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:46:09 BST [ISS.0098.0042I] Successfully reconnected to Broker. Sto
2017-07-01 23:46:39 BST [ISS.0098.0041C] Unable to connect to Broker. Starting t
2017-07-01 23:46:49 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:46:49 BST [ISS.0098.0042I] Successfully reconnected to Broker. Sto
2017-07-01 23:47:19 BST [ISS.0098.0041C] Unable to connect to Broker. Starting t
2017-07-01 23:47:29 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:47:29 BST [ISS.0098.0042I] Successfully reconnected to Broker. Sto
2017-07-01 23:47:59 BST [ISS.0098.0041C] Unable to connect to Broker. Starting t
2017-07-01 23:48:09 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:48:09 BST [ISS.0098.0042I] Successfully reconnected to Broker. Sto
2017-07-01 23:48:39 BST [ISS.0098.0041C] Unable to connect to Broker. Starting t
2017-07-01 23:48:49 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:48:49 BST [ISS.0098.0042I] Successfully reconnected to Broker. Sto
2017-07-01 23:49:19 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:49:29 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:49:29 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:49:59 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:50:09 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:50:09 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:50:39 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:50:49 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:50:49 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:51:19 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:51:29 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:51:29 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:51:59 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:52:09 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:52:09 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:52:39 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:52:49 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:52:49 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:53:19 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:53:29 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:53:29 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:53:59 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:54:09 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:54:09 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:54:39 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:54:49 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:54:49 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:55:19 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:55:29 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:55:29 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:55:59 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:56:09 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:56:09 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:56:39 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:56:49 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:56:49 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:57:19 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:57:29 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:57:29 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:57:59 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:58:09 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:58:09 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:58:39 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:58:49 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:58:49 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:59:19 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll
2017-07-01 23:59:29 BST [ISS.0025.0025I] Broker Synchronizer initialized
2017-07-01 23:59:29 BST [ISS.0098.0042I] Successfully reconnected to Broker. Stopped polling
2017-07-01 23:59:59 BST [ISS.0098.0041C] Unable to connect to Broker. Starting to poll