Cluster Provider initialization exception during the startup of My webMethods Server

Description:

This article is about Cluster Provider initialization exception during the startup of My webMethods Server. The article applies to My webMethods Server versions 10+.

The startup of a My webMethods Server cluster node might fail with the following stack trace:

 2021-01-25 15:53:47 ICT (c.w.p.s.c.i.ClusterProvider:ERROR) [Start Level: Equinox Container: cb90d8aa-84ac-47e9-9818-239e0be842db] [   ]  [  ]- null
com.webmethods.portal.system.init.InitializationException: null
 at com.webmethods.portal.system.cluster.impl.ClusterProvider.init(ClusterProvider.java:389) ~[com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
 at com.webmethods.portal.system.impl.BaseProvider.activate(BaseProvider.java:146) [com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
 at com.webmethods.portal.system.cluster.impl.ClusterProvider.activate(ClusterProvider.java:204) [com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_242]
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_242]
 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_242]
 at 
org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1657) [org.eclipse.osgi_3.13.300.0000-0004.jar:3.13.300.0000-0004]
 at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1558) [org.eclipse.osgi_3.13.300.0000-0004.jar:3.13.300.0000-0004]
 at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:233) [org.eclipse.osgi_3.13.300.0000-0004.jar:3.13.300.0000-0004]
 at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:343) [org.eclipse.osgi_3.13.300.0000-0004.jar:3.13.300.0000-0004]
 Caused by: com.webmethods.portal.PortalException
 at com.webmethods.portal.system.cluster.impl.ClusterProvider.registerLocalClusterNode(ClusterProvider.java:754) ~[com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
 at com.webmethods.portal.system.cluster.impl.ClusterProvider.init(ClusterProvider.java:364) ~[com.webmethods.caf.server_10.5.0.0004-0065.jar:10.5.0.0004-0065]
 ... 135 more
 2021-01-25 15:53:47 ICT (c.w.r.p.ShutdownManager:INFO) [tHREAD-13][  ] [  ] - Shutdown Manager running all shutdown hooks.
 2021-01-25 15:53:47 ICT (c.w.r.p.ShutdownManager:INFO) [tHREAD-13][  ] [  ] - Portal state : 4
 [ProxyLogManager] reset() -> org.apache.logging.log4j.jul.LogManager@79638fc5
 2021-01-25 15:53:47 ICT (c.w.p.s.i.i.ClusterPhaseProvider:INFO) [MWS Boostrap Lock Hook][  ] [  ]   -Released cluster bootstrap lock
 [Mon Jan 25 15:53:48 ICT 2021] Bootstrap : Stopping...
 
 !ENTRY com.softwareag.platform.hook.shutdown 1 0 2021-01-25 15:53:48.304
 !MESSAGE Stopping OSGi framework.
 2021-01-25 15:53:48 ICT (o.e.e.logger:INFO) [Wrapper-Shutdown-Hook] [  ] [  ] -Stopping OSGi framework.
 
 !ENTRY com.softwareag.platform.hook.shutdown 1 0 2021-01-25 15:53:48.305
 !MESSAGE Framework shutdown timeout set to 120 seconds.Please also check tanuki-wrapper shutdown property, if specified.
 2021-01-25 15:53:48 ICT (o.e.e.logger:INFO) [Wrapper-Shutdown-Hook] [  ] [  ] -Framework shutdown timeout set to 120 seconds. Please also check tanuki-wrapper shutdown property, id specified.

Root Cause:

This issue occurs when the values of the NODE_NAME parameter for the instance do not match throughout all required configuration files for My webMethods Server. The NODE_NAME mismatch is usually a result of manual editing of one or more (but not all) of the configuration files.

My webMethods Server stores the node name of a cluster node in the cluster.xml file in the My webMethods Server database, and in the following configuration files on the file system:

  1. <SoftwareAG_directory>/MWS/server//config/cluster.node.properties(bak) - the value in this file gets copied to the cluster.xml configuration file in the My webMethods Server database at startup.
  2. <SoftwareAG_directory>/profiles/ MWS_/configuration/custom_wrapper.conf (set.NODE_NAME=) - the value in this file gets verified at startup, together with the value in the configuration file in 1. If the values do not match, the server node will not start.
  3. <SoftwareAG_directory>/server//bin/instance.properties - the value in this file gets copied to the custom_wrapper.conf configuration file (2.) when creating or updating the My webMethods Server profile.

When updating the instance profile, for example, with the installation of a test patch or official fix, or when using the update command, My webMethods overwrites the NODE_NAME values in the custom_wrapper.conf and cluster.xml configuration files.
If you modified part of the NODE_NAME values manually, and the values in all configuration files no longer match, the My webMethods Server cluster node will fail to initialize with the exception above.

Resolution:

My webMethods Server provides a mechanism for updating the name of a cluster node, using the command line. In order to avoid the issue, described in this article, always use the My webMethods Server command line to modify the NODE_NAME of a cluster node, as follows:

  1. Go to <SoftwareAG_directory>/MWS/bin/mws.sh|bat
  2. Run the mws update command, as follows:
    mws -s update -Dnode.name=<newNodeName>
1 Like

I see Node Name under all 3 files, still I see same error ? what would be the next step. Attaching my log files…Iam using 10.11 trail version.

<SoftwareAG_directory>/MWS/server//config/cluster.node.properties(bak)

#Sat, 14 May 2022 08:33:42 -0600
name=Sandeep-Helen-Cathy-node54779
portal.http.port=8585
roles=autodeploy,search,taskengine,notification

portal.https.port=0

<SoftwareAG_directory>/profiles/ MWS_/configuration/custom_wrapper.conf (set.NODE_NAME=)

#encoding=UTF-8

#********************************************************************

Application Paths

#********************************************************************

Application Paths

set.SERVER_NAME=default
set.NODE_NAME=Sandeep-Helen-Cathy-node54779
set.SERVER_HOME=C:\SoftwareAGnew\MWS\server/default
set.PORTAL_HOME=%SERVER_HOME%/…/…
set.WM_HOME=%PORTAL_HOME%/…
#set.JAVA_HOME=${wrapper.java.home}
set.FILE_ENCODING=UTF-8

<SoftwareAG_directory>/server//bin/instance.properties
platform.log (59.7 KB)
sag-osgi.log (119.6 KB)
wrapper.log (198.6 KB)

#Sat, 14 May 2022 08:33:42 -0600

server.template=
components.include=/.cdp **/.pdp /.skin **/.war /.dccb **/.lp /*.war_lp
components.exclude=samples/
extras/
development/tools/
test/
templates/** content/preview/** alignspace/** default/collaboration/** default/internet/wm_webclip*.* services/task/**/. services/deployer/wm_task*.* system/basetypes/wm_xt_basictask.pdp
node.name=Sandeep-Helen-Cathy-node54779
debug.port=10033