IS 6.0.2 Fails to complete initial load of TN data in Oracle

, ,
1

We have a problem with our production install of IS 6.0.2 which I am hoping you have seen before, or know how to get around.

We have established 4 discrete IS environments � Development, System Test, User Acceptance Testing, and Production. Each of these is a separate install of IS, has a separate Broker, and a separate Oracle instance accommodating the Xref, ISCoreaudit, ProcessAudit, and TN functional pool.

We are running Oracle 8i on a hpux11i platform. We have Trading Networks, Modeller, and additional components including WmebXML.

The Development, System Test and User Acceptance Test environments were all created on a single server. The installation and configuration went very smoothly, and all environments functioned.

We then performed the Production implementation on a separate server, but did not have a chance to start IS as we were waiting on our Database Administrators to create the file system and empty schemas. Due to a server upgrade we then migrated the 3 non-Production environments to the same server the Production server is hosted on.
These three environments can start and run successfully.

The problem is with the Production implementation. We started IS and did the usual configuration � Broker names, Extended fields, and jdbc pool and functions. We then shutdown and restarted IS�this is when our problems started.

When IS first starts up and realises it has a database to talk to, it recognises that the schema is empty of records and populates these with the initial processing rules and document types you see when you first start TN. This process, as seen in a log with DEBUG set to 10, appears to load up 11 records into the ProcessingRules table�and then it stops functioning. Almost. Every 120 seconds it writes about 10-15 lines to the server log talking about Repository entries and then goes to sleep for another 120 seconds. It does not get past this point. It does not get as far as starting the HTTPListener on the primary port.

At this point we�ve had to crash the server by giving it a �kill� message. After removing the LOCKFILE and restarting, IS goes through the motions of checking the Repository, and then starts loading records from the database again. It gets to the stage of giving us a message saying �loading Processing Rules��and stops again. No loops, just sits there.

What have we tried? Lots:

  1. Dropped and recreated the Oracle database tables. No luck.

  2. Deleted and recreated Oracle on the file system. No luck.

  3. Substituted the empty Oracle database for one we had prepared on another system. This worked, in that IS started, but it is beside the point. We should be able to load IS with an empty database the way that webMethods intended it…

  4. Took the same substituted database, dropped its tables and recreated them. No luck.

  5. Dropped tables and recreated schemas, then pointed our (working) User Acceptance instance of IS at it to populate. No luck (interesting this�you would think that a working IS install would go through to completion�this has us suspecting our Oracle install�)

  6. Dropped tables and recreated schemas. Disabled the TN functional pool and restarted IS. This worked, but as soon as we added in the TN pool and restarted IS�no luck.

  7. Deleted the WmRepository2 folder under IntegrationServer/config. (in case this was corrupt) No luck.

  8. Started IS without a database. Disabled the WmebXML package. Dropped tables and recreated schemas. Restarted IS. This worked, in that database records were created and IS could restart�but when we restarted WmebXML and restarted IS�back to square one.

So � that is where we are at. Point 3 is the closest we have gotten this thing to run up. My suspicion is that there is something different in terms of Oracle install, or unix patch level, on the particular server we are running on which is causing jdbc problems between IS and the Oracle database, causing these lock ups and failures. I have our systems admin guys looking into this. My next and most annoying step to take is to blow away the entire Production installation � databases, executables, repositories, and packages � and reinstall it from scratch. I really don�t want to have to do that.

If you have seen such behaviour in IS when it is first running up, I�d really appreciate hearing what you believe caused it, and what you did to resolve it.

Thanks in advance,

  • Alistair
2

Folks, a quick follow up after some investigations this morning.

We have discovered that the target server that we are having problems on is running a differently patched version of Oracle than our original non-production server. The target server is running 8.1.7.4.0, and the other server was running 8.1.7.0.0.

It is the target server we are having the problems on. We did a test running an empty Oracle database on the non-prod server, and configured the JDBC pool on the production server to point to it. This worked. We then did the reverse, with the application running up on the old server, and the database on the new server, and it gave us the same bad result.

We have escalated this to our DBAs to ask them what the difference should be in these Oracle patch levels, and what could be causing this problem. If we get it resolved I’ll drop an update here.

3

Folks, a resolution for you.

webMethods this afternoon supplied an update of the DataDirect JDBC Oracle driver, build 35. We replaced the standard out-of-the-box driver that came with IS, and restarted it with an empty database.

This worked. It would appear there are some “fixes” in this updated driver file that allow it to have an improved dialogue with a database running on version 8.1.7.4.0 on hpux 11i 64 bit.

This driver is most likely available from your local webMethods support office. To apply, the jar file is copied into IntegrationServer/libs/jar , replacing the existing file in this directory.