IS won't start correctly after fix installation

Renan_Lopes1 · February 11, 2025, 5:06pm

Product/components used and version/fix level:

Integration Server 10.3 Fix #31 (Development environment)

Detailed explanation of the problem:

Hi there!
We’re trying to install the fix #31 for our I.S 10.3. We’re currently on Fix #19.
However, after using UpdateManagerCMD.sh to install the fix #31, our IS is displaying some exceptions when starting.
Also, we can’t navigate the IS Admin page, it’s only displaying the “Server > Statistics” page and we can’t see the menus:

Please, do you know if need to do any other steps after this fix installation?

When re-opening the UpdateManagerCMD.sh and checking the available fixes, we have the following list:
eaid00-available-fixes.txt (10.6 KB)

Error messages / full error message screenshot / log file:

Fist exception

2025-02-11 11:51:20 GMT-03:00 [ISS.0017.0001E] (tid=256) An error was encountered parsing SoftwareAG-IS-IData.xml: net.sf.ehcache.CacheException: com.terracottatech.frs.recovery.RecoveryException: Caught an error recovering from log at /WebM/IS01/IntegrationServer/instances/default/./cacheStore/SoftwareAG-IS-IData/$
        at net.sf.ehcache.store.restartability.EhcacheRestartability.startStore(EhcacheRestartability.java:270)
        at net.sf.ehcache.store.restartability.EhcacheRestartability.constructRestartability(EhcacheRestartability.java:217)
        at net.sf.ehcache.store.restartability.EhcacheRestartability.getRestartablePortability(EhcacheRestartability.java:118)
        at net.sf.ehcache.store.heap.RestartableHeapStore$RestartableBackingMapFactory.<init>(RestartableHeapStore.java:97)
        at net.sf.ehcache.store.heap.RestartableHeapStore.create(RestartableHeapStore.java:47)
        at net.sf.ehcache.EnterpriseFeaturesManager.createRestartableStore(EnterpriseFeaturesManager.java:256)
        at net.sf.ehcache.EnterpriseFeaturesManager.basicCreateStore(EnterpriseFeaturesManager.java:165)
        at net.sf.ehcache.EnterpriseFeaturesManager.createStore(EnterpriseFeaturesManager.java:123)
        at net.sf.ehcache.Cache.initialise(Cache.java:1223)
        at net.sf.ehcache.CacheManager.initializeEhcache(CacheManager.java:1362)
        at net.sf.ehcache.CacheManager.addCacheNoCheck(CacheManager.java:1428)
        at net.sf.ehcache.CacheManager.addConfiguredCaches(CacheManager.java:824)
        at net.sf.ehcache.CacheManager.doInit(CacheManager.java:502)
        at net.sf.ehcache.CacheManager.init(CacheManager.java:398)
        at net.sf.ehcache.CacheManager.<init>(CacheManager.java:269)
        at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1108)
        at net.sf.ehcache.CacheManager.newInstance(CacheManager.java:1084)
        at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:350)
        at com.wm.app.b2b.server.cache.config.CacheManagerConfig.readConfiguration(CacheManagerConfig.java:404)
        at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:40)
        at com.wm.app.b2b.server.cache.config.CacheManagerReader.call(CacheManagerReader.java:20)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: com.terracottatech.frs.recovery.RecoveryException: Caught an error recovering from log at /WebM/IS01/IntegrationServer/instances/default/./cacheStore/SoftwareAG-IS-IData/persistence/metadata
        at com.terracottatech.frs.recovery.RecoveryManagerImpl$ReplayFilter.checkError(RecoveryManagerImpl.java:267)
        at com.terracottatech.frs.recovery.RecoveryManagerImpl.recover(RecoveryManagerImpl.java:103)
        at com.terracottatech.frs.RestartStoreImpl.startup(RestartStoreImpl.java:117)
        at net.sf.ehcache.store.restartability.EhcacheRestartability.startStore(EhcacheRestartability.java:268)
        ... 24 more
Caused by: java.lang.AssertionError
        at com.terracottatech.frs.object.RestartableMap$MapObjectManagerStripe.replayPut(RestartableMap.java:438)
        at com.terracottatech.frs.object.AbstractObjectManager.replayPut(AbstractObjectManager.java:41)
        at com.terracottatech.frs.PutAction.replay(PutAction.java:138)
        at com.terracottatech.frs.recovery.RecoveryManagerImpl$ReplayElement.replay(RecoveryManagerImpl.java:322)
        at com.terracottatech.frs.recovery.RecoveryManagerImpl$ReplayFilter.lambda$null$2(RecoveryManagerImpl.java:253)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
        at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
        at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool.helpComplete(ForkJoinPool.java:1870)
        at java.util.concurrent.ForkJoinPool.awaitJoin(ForkJoinPool.java:2045)
        at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:404)
        at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
        at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
        at com.terracottatech.frs.recovery.RecoveryManagerImpl$ReplayFilter.lambda$replayBatch$3(RecoveryManagerImpl.java:247)
        at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1424)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
. See the Integration Server error log for the full stack trace.

Second Exception:

2025-02-11 11:51:26 GMT-03:00 [agent.load.tartI] Loading registry config file /WebM/IS01/IntegrationServer/instances/default/config/fabric-agent.xml
2025-02-11 11:51:26 GMT-03:00 [agent.load.rrorI] Error processing registry config file /WebM/IS01/IntegrationServer/instances/default/config/fabric-agent.xml
java.io.FileNotFoundException: /WebM/IS01/IntegrationServer/instances/default/config/fabric-agent.xml (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at java.io.FileInputStream.<init>(FileInputStream.java:93)
        at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
        at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
        at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
        at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
        at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
        at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
        at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:177)
        at com.webmethods.fabric.agent.FabricAgentLoader.load(FabricAgentLoader.java:96)
        at com.webmethods.fabric.agent.FabricAgent.load(FabricAgent.java:64)
        at com.wm.app.b2b.server.FabricAgentManager.init(FabricAgentManager.java:45)
        at com.wm.app.b2b.server.Server.run(Server.java:707)
2025-02-11 11:51:26 GMT-03:00 [ISS.0025.0054E] (tid=74) Fabric Agent initialization failed.

Third exception:
When accessing https://[my-is-ip]:5100/WmRoot/menu.dsp I can see a 500 Internal Server Error page.
But I can’t see any logs anywhere containing the reason for this error 500

Question related to a free trial, or to a production (customer) instance?

Production (customer) instance on DEV environment

Holger_von_Thomsen · February 11, 2025, 5:31pm

Hi,

you can try to restore the fabric-agent.xml file from config/backup to config directory.

Did you check the Readme for IS_10.3_Core_Fix31, if it requires additional dependent fixes to function properly?
If this is the case, apply these fixes additionally,

On the other side you should check, if it is possible to perform a side-by-side migration to 10.15 as 10.3 is quite old and out of support.

Regards,
Holger

Renan_Lopes1 · February 11, 2025, 5:46pm

Hello Holger!

I did as you suggested and the second exception disappeared

Did you check the Readme for IS_10.3_Core_Fix31, if it requires additional dependent fixes to function properly?
If this is the case, apply these fixes additionally,

Yes, I did and I found the following fixes are required:
IS_10.3_Migration_Fix1 OK - we are on fix #2
WSS_10.3_Core_Fix3 OK - we are on fix #14
TPS_10.3_XMLTools_Fix3 OK - we are on fix Shared Bundles 10.3 XMLTools Fix 3
OGI_10.3_Migration_Fix1 – Not found in the available fix list
TPP_10.3_Misc_Fix2 OK - We are on Shared Libraries Misc 10.3 Fix 3
WSC_10.3_Audit_Fix5 OK - We are on Shared Components 10.3 Audit Fix 5
MTA_10.3_Fix1 – Not found in the available fix list

Please, what’s the meaning of “OGI” and “MTA”?

Regarding the migration to 10.15 we won’t be able to do it yet because we’ll have a lot of costs by buying a new server, buying a new O.S license, a new CPU, etc.
However, our managers are aware that we’ll need to upgrade our environment soon and they’re managing resources to do this year.

Thanks again!

Holger_von_Thomsen · February 11, 2025, 6:11pm

Hi Renan,

OGI stands for the “OSGi Platform”.
Latest fix would be OGI_10.3_Migration_Fix9 dated 04.05.2023.

MTA stands for the “webMethods Metering (MTA)” product.
Latest Fix for this would be MTA_10.3_Fix14.

Regards,
Holger

Renan_Lopes1 · February 11, 2025, 7:09pm

Holger,

Oh, I managed to install the MTA fix but the OGI fix OGI_10.3_Migration_Fix9 is not appearing in our available fix list.

As we have an image backup, I decided to install all the available fixes. All of them were succesfully installed, except the “Web Services Stack OSGi 10.3.0 Fix 14” with the following error:

Error: Action: osgiInstall; Args: enableOnRepatch:true;installIU:com.softwareag.wsstack.osgi.feature.patch.feature.group;profileRepo:*;publishPatchTo:common/runtime/bundles/wss/eclipse;repository:repo;; Status: ERROR; Error Message:Error occurred in executing action "MWS_default" units not installed: [com.softwareag.wsstack.osgi.feature.patch.feature.group 10.3.0.0014-0606]
Error: Action: osgiInstall; Error Detail:
Error: Failed action executions for fix name: Web Services Stack OSGi 10.3.0 Fix 14 wMFix.WSSOSGi 10.3.0.0014-0606 Status : Error
Error: Action: osgiInstall; Args: enableOnRepatch:true;installIU:com.softwareag.wsstack.osgi.feature.patch.feature.group;profileRepo:*;publishPatchTo:common/runtime/bundles/wss/eclipse;repository:repo;; Status: ERROR; Error Message:Error occurred in executing action "MWS_default" units not installed: [com.softwareag.wsstack.osgi.feature.patch.feature.group 10.3.0.0014-0606]
Error: Action: osgiInstall; Error Detail:

Error: SUM_ERROR>Can not install fix: Web Services Stack OSGi 10.3.0 Fix 14 WSS_10.3.0_OSGi_Fix14 wMFix.WSSOSGi 10.3.0.0014-0606
Post Install Messages:
The fixes you installed require fixes for other products. These fixes are not installed, because you do not have the relevant products in your product installation. If you have the indicated products in another product installation, install the following fixes there:
Web Services Stack Web Application 10.3.0 Fix 14
 If you do not have the indicated products installed on any product installation, ignore this message and finish your update

Unfortunately, the IS Admin Page continue with the issue:

I can see some errors in the browser console log:

Please is there anything else I can do? How can I be able to check the WmRoot 500 Internal Server Error logs?