I am currently involved in a project where we are implermenting the following webMethods components:
webMethods Integration Server v6.0.1 (currently on service pack 2)
webMethods Mainframe v6.0.2 using CICS Sockets
Note: Both of these components are implemented on Solaris servers.
The queries that I have are:
When implementing webMethods, what is the best practice approach / experiences gained for implementing logging for the purposes of operational reporting? Is it best to stick with the out-of-the-box audit logging that comes with webMethods, or is it more effective to generate custom services for logging and utilise libraries such as Log4J?
Is there any performance advantages / disadvantages with doing this?
If required to log specific attributes as part of Operational Reporting requirements, that are not logged as part the the standard audit logging (e.g. Customer ID that is part of the pipeline data) what is the best way to approach this.
Similarly for Exception handling and error logging, from experience, what has been the most effective approach to exception handling and the logging of errors. Currently we are proposing to use Try-Catch blocks implemented using sequence steps in flow (as per the webMethods best practice document). Is this the best approach?
What was your final solution on this. We went with custom db tables to record our specific attributes (bp data) and we have rarely relied on audit logging for no more than debugging unhandled expeptions. We kept the wm standard try-catch blocks and also created standard services for exception handling, logging and notifications. We included the use of Manager within our exception handling as well.
I would appreciate hearing your final solution as well.
I’ve used Log4J on a few webMethods projects (and some Java projects before that) with great success. If you’ve used it before (sounds like you have) then you know how flexible and easy-to-use it is, and you can turn the log-level down or even off in production for better performance - I’d highly recommend it! One of the key advantages is that it keeps your application logs separate from the webMethods platform logs - so your application support team can focus on the messages that have meaning to your application.
The WmUsers site (http://www.wmusers.com) has a download wrapper package for Log4J contributed by Mark Carlson which will get you going if you decide to go that path.
On the error logging side of things, one approach I’ve seen a couple of times which seems pretty effective is to publish an “error” document from inside the catch block of services (where the error is determined to require notification). A subscribing trigger picks up the error message and formats it into an email for delivery to the support team. This centralises and decouples your error notification logic from the error detection logic. By using pub/sub for error notification, the error handler can even be running on a separate Integration Server.
So, how did (anyone above) configure your documents and logging for use with webMethods Monitor? Would it be redundant to roll your own inside catch blocks and also log the pipeline document for view in Monitor?
Also, with regard to Monitor and document searches, our inclination is that it is not of much use in a high volume environment because the search features lack user defined fields. We only have date time to rely on, and cannot add our own fields, such as customer id, etc.
Process models allow you to specify custom fields to log during execution of the model. These can then be searched on using the Advanced button on the Monitor search page.
We are using this to track thousands of interface partner documents as they go through our process model. Users are able to use the built-in Monitor advanced search capability to locate specific documents based off the additional fields we log (e.g. document number, type, etc). The Monitor search screen adds fields for each custom logged field and includes operators such as ‘=’, ‘>’, ‘contains’, etc. I’m sure if you take a look at the dsp’s and services that support these pages, you could extend the functionality much further.
In terms of error handling, you can also build error steps into the process model to handle exceptions. Your steps could use a logging utility to write to a log or send an email. The nice feature using Monitor and a model is that you can easily determine the point of failure and resubmit the transaction if necessary.
We implemented our own custom logging framework. It captures both errors and process execution. The main reason was that we’d like to keep our logs for about 12 months and the webMethods audit logs fill the DBs so quickly we need to archive them every 3 months. Services publish a ‘logging’ documents as they execute, which have a logging level similar to Log4J (Info, Warning, etc). We then have a lightweight UI for browsing these logs. This way we can insulate our users away from the webMethods GUI’s. Log entries or steps are made for ‘processes’ which belong to a ‘functional area’. They can also have custom ‘attributes’ attached to them and can be used for searching.
I enjoyed reading through this thread. It’s nice when you realise that lots of others have faced similair issues.
In my organisation we began with the approach to only use the webMethods features - and not to use any add ons. This soon turned out to be a bad idea.
In the past we have used the audit log for recording service execution times - this was flawed for the case of fast services (40mS) where we found the overhead (around 200 mS) just way too much. Also it is really annoying that you can’t put application information into the audit log. Same goes for the error logs. In our case we tail our logs centrally to trigger support calls - since you can’t put application data in it does not help us.
Because of that we generally use the server log for everything and have used external logging either log4J or an internally developed equivalent - esp for greater detail. Good thing about this is the performance.
Personally I prefer a seperate application log file and can’t understand why this isn’t a setting for each package.
And a grumble…
I always wondered why on debugLog you can specify the function name - couldn’t the server figure it out?
Has anyone implemented exception handling where the server exception has to flow up to the client? I am using web services and the webMethods web services throw ServiceException. I want to be able to transfer the original exception to the client but it seems that the SoapException the client receives only includes the ServiceException and only the message of the original exception (MyException). I instanciate ServiceException with ServiceException(MyException).
The ServiceException xml looks like this in the Soap message:
wm.server.soap:rpc
com.wm.app.b2b.server.ServiceException
com.kk.k2.lang.MyException: My exception message
com.wm.app.b2b.server.ServiceException: com.kk.k2.lang.MyException: My exception message
at ....
For a web service call you might want to define an output structure containing an error message that becomes part of the wsdl. So the flow would be
SEQ: parent try catch
SEQ: Try
Do something
Map output data + success = true
SEQ: Catch
get last error
Map output error message data + success = false
this way the web service invoker can look for the true / false and if false contain a stuctures message with any data you want, such as error code, type of error, error message,
There may still be exceptions that go back to the client that are not handled (such as soap fault or timeout). the client needs to be smart enough to handle the various classes of error.
One advantage of this is that “systems” exceptions can be distinguished from “data” exceptions etc. and then be handled differently by the caller.
Sorry guys,
I know the post it’s quite old, but I’m still interested in the topic
I’m trying to compare whta’s the best approach for error logging and tracking.
The functionalities in the last IS release seems quite astonishing, and I don’t think there would be need for an external logging system.
What advantages could a log4j logger bring to the application? What should be discouraged the use of the internal logging ?
I would use the service audit for logging error, exceptions, start and stop, even to log business data (from the log fields screen). The internal monitor allows resubmit of failed services as weel as a great search capability.
I see adavantages in public services, overriding the erroe structure to external call, but what for internal only services?
Any help or comment would be really helpful
Thanks
Fabrizio
You can implement auditing and exception handling as services. Application-level audit logging requires status data about each business transaction. The ability to monitor an application in a business or transactional context requires a monitoring interface within the service to send messages detailing the transaction’s status specific to the service invocation. This requires each service to send a status message at critical steps in the business transaction. You can then build a real-time viewer to correlate status messages (based on the semantics of the message - e.g. transaction ID) with the services within the composite application. This provides an end-to-end view of the business transaction for SLA management, fault tracing and problem determination.
An audit service can then be implemented as a state machine that can consume and record messages based on criteria defined in its configuration. A generic exception handler can also use the audit service to support a centralized view of problems that occur in the enterprise SOA - to support exception-based monitoring. Any should-not-occur condition in the solution is instrumented to send an exception message to the exception handler.
I know this is a late one. We use and have implemented a SOA solution for all our error handling. This allows us to log all types of information independent of environment ( Mainframe,Net. Java, Cobol, SAP ). The framework make use of a “transactionID” to collect and order all incoming log messages. By using this type of approach we can “map” each transaction into a transaction graph ( transaction flow ). All code exist for these environments. .Net client, SAP client, Java Client and the Server side. You can find it at [url]https://github.com/stefan-andersson-se-at-gmail-com[/url] The Wiki contains documentation and a demo.
I want to manage the technical exceptions on the BPM which not expected and could damage the running processes then the goal is to know how to catch them and to handle these types errors occuring on each activity of Workflow.
You can create an installer class, which is invoked at installation time when administrator privileges are available, or you can configure the permissions on the EventLog registry key to allow the ASP.NET process identity (or impersonated identity) to create event sources at run time. The former approach is recommended.