Hi there.
We have applications running on the mainframe (OS/390) and we need to integrate with these applications (from Enterprise Server 5.0.1).
We know all about webMethods Mainframe (for CICS & IMS).
We know all about using JDBC adapter to write to DB2 on the mainframe, however the mainframe application requires a flat file feed. The question is: what is the best way to get the file there ? Use an IO adapter to write the flat file and then FTP it to the mainframe ? What about packed (comp-3) fields ? The mainframe application requires some fields to be in packed (comp-3, comp) format. Should we make the mainframe programmers convert the regular fields in the flat file to packed format (after they receive the file on the mainframe), or should we use custom Java code in our IO adapter to create packed fields ? Obviously we are hoping to reinvent the wheel. Surely someone has run into the same issue(s) ???
The best thing is to create a flat file from the IO Adapter, FTP the file to the Mainframe and have a Cobol/CICS program convert it to required Mainframe format, be it Packed or Zoned.
Creating the packed/zoned fields isn’t too terribly difficult. There should be some code available publicly that can help. The IO Adapter along with some custom methods should do the trick.
A question to ask yourself–where is it easiest to do the work and maintenance of the necessary code, on the mainframe or on a workstation? If the change procedures and such on the mainframe aren’t onerous–and you don’t mind the ribbing you’ll get from the mainframers about not be able to do any “real” work outside the mainframe --then Sharath’s suggestion is the way to go.
We are experiencing a bit of ribbing from the mainframers because they think that webMethods platform should do everything. (“it slices, it dices…” ). My Christmas wish is for more out-of-the-box mainframe capabilities (i.e. read/write VSAM and flat files on the mainframe via an adaptor or the Mainframe server). webMethods is VERY aware of this and have added a feature request…so hopefully someday before I retire we will see it
There is a webMethods CICS Adapter which came with the standard adapter installation in the 4.1 Adapter version. This is not continued in any of the existing Adapter releases.
The CICS Adapter has the capability to read/write VSAM files on the Mainframe. It has two versions - SNA and TCPIP based on the connectivity that is established between webMethods and the Mainframe.
You may try your luck with webMethods, if they can still send you a copy that you could evaluate on.
Are you sure that the CICS Adapter can read/write VSAM files? When I worked with it a few years back, it could interact with any CICS trans (passing stuff in the COMMAREA). I believe they added IMS support at some point. I think the CICS/IMS programs do the actual reading of data but I may be out of touch.
The adapter used Insession’s TransFuse product underneath the covers (but not too far underneath that you didn’t know you were working with TransFuse). The adapter was a cool piece of software that wM seemed concerned about us using in production, asking us about stability and such. It was stable for us!
Thanks for the feedback…again.
This issue is getting very ugly at my company. The mainframers are now saying “I told you so” because we cannot seamlessly read/write (flat & VSAM) files from/to the mainframe. The project managers are getting upset because the webMethods Super-Dooper-Wonder product cannot interact seamlessly with all files on the mainframe (out of the box). And EVERYONE is questioning why we are using webMethods as our Enterprise Integration package.
You and I know that other EAI and ETL products suffer the same limitations…but these people don’t know this. They are just looking for any opportunity to bash the product. Most of these people (mainframers) feel threatened by webMethods and what it can do…so they are looking for any opportunity to say “Ah ha !!! I told you that the new-fangled product you bought isn’t very good”. These comments are filtering back to upper management.
Our company has MANY mainframe applications that will not go away for quiet awhile.
Is anyone else in the same situation ? (Using webMethods for EAI with many mainframe applications) What is your stragedy for getting flat files and/or VSAM data to/from the mainframe ? How are you dealing with the negative people ?
Wayne, I think everyone feels your pain. We have all had to debunk the “Magic Wand” expectations of the webMethods Platform.
The issue comes down to somebody believing that any software could be installed on a network and do things without preparation and without analysis.
Remember – webMethods is a Business Integration company. The webMethods Platform uses technical tools to achieve this objective, but without careful attention to the underlying business details, an integration will fail. The same holds true for any of webMethods competitors and even for a hand-coded, Java integration.
In general, failed business integrations using the webMethods Platform failed not because of the technology behind the integration but because of the business processes (or lack thereof) in front of it.
I plan to discuss these topics more in an upcoming webMethods Ezine article.
Being a mainframer for 32 + years I can say that they are being unrealistic. Even when we connected using LU2 (screen Scraping) we were limited to Displayable characters only (no binary). That’s why we built a mainframe Integration system that’s connect’s to all of our Legacy applications in a standard fashion. It mitigates the differences between the Legacy Applications. Our mainframe system process 2.5 million transaction daily with a sub-second response time. We can connect to this sytem using LU2 or more preferred is ECI or IBM’s MQ Series. In any case I don’t know of any product that reads VSAM directly as does DB2 wusing DB2Connect.
As you pointed out, you won’t find a tool that is able to read VSAM data directly. A couple years ago we had searched extensively for a tool that would provide simple access to VSAM data with no luck. Tony–can you provide an overview of the facilities DB2 provides wrt VSAM data access?
The approaches that have been successfully used that I know of:
ODBC adapter to DB2/MVS via DB2 Connect ODBC
FTP of flatfiles to/from the mainframe, including packed/zoned decimal fields
CICS adapter as described in my earlier post. The CICS programs were PL/I apps developed specifically for integration
MQSeries adapter to provide the pipe between ES and the mainframe
There are undoubtedly other approaches.
I think the people issue can be addressed by getting a mainframer or two involved in crafting a solution. If they know of an easy way to do what needs to be done (whether on the mainframe or whatever) then it should be pursued. The only way that people end up understanding where wM fits (and where it doesn’t) is for them to get familiar with the problem(s) to be solved, and understand various approaches and their drawbacks. Anyone that has the I-have-a-hammer-so-everything-is-a-nail view is going to be at a disadvantage.
We found a unix-based utility (I believe it also works in Windows) called NetConvert that takes mainframe files with fairly complex layouts and data types (e.g. COMP-3). The company is called “The Workstation Group”, www.wrkgrp.com. We never implemented the tool because we had already written COBOL programs, but we felt it would have been a good way to grab mainframe files and make them readable by webMethods.
Thanks for all the great info and thoughts.
Rob, question for you: Is it possible to create a flat file (using IO adapter in ES) with a mixture of ascii text & packed/zoned decimal fields and then ftp this file to the mainframe (without any data becoming corrupted ?) Based on your response above it sounds possible.
We are under the impression that if we ftp a file containing mixture of regular text & packed/zoned to the mainframe then the file will be corrupted. If this is possible what mode do we use ? I’m thinking we cannot use ascii mode with the packed/zoned fields ? If we use binary will it corrupt the non-packed fields ?
Well the CICS Adapter does not perform the Read/Write alone, it does require help through the 4 - 5 Cobol/CICS programs that comes with the CICS Adapter installation and a custom Cobol/CICS program running on the background.
This product still perfomes as what is desired out of it. We have found it to be stable in our Production environment.
Wayne,
You may have to pacify the Mainframe programmers to get this done. I am not sure of the flat file containing ascii and packed can be ftp’ed. Let me check and get back on this.
This is becoming an interesting conversation… lot of knowledge to share…
Sharath–that’s what I remember. There are some programs to install (InSession stuff) on the mainframe that do the handshaking/marshalling with the CICS adapter (more InSession stuff), along with an optional security exit, and then the actual program that CICS adapter (InSession) will invoke. Is that last part what you mean by “…a custom Cobol/CICS program running on the background.”?
I’m just trying to make sure I understand your description/experience–if it was the same as mine, then the CICS Adapter cannot be accurately described as having “…the capability to read/write VSAM files…”. The CICS Adapter can communicate with CICS transactions–which may in turn read/write VSAM, TDQs, TSQs, IMS, KSAM, whatever.
Wayne–I believe the IO adapter can handle this with no problem, though I have not done it myself (the project where packed/zoned was supported was a couple of years ago, prior to the IO adapter).
For the “corruption” issue, I believe it’s more a function of the ASCII/EBCDIC translation than anything else.
There is a guru that has addressed this issue. I don’t believe he participates in this forum so I’ll reach out to him and post what I find out.
You can create a file containing both text (character) data and packed (comp-3) data on the PC then FTP it to the mainframe.
One approach is to write the text data as EBCDIC and FTP the file in BINARY mode so that it is not translated.
You would use an OutputStream (for example) to write the file. To write a String as EBCDIC you use something like stream.write(string.getBytes(“Cp037”)). This converts string to an EBCDIC byte array and writes it to the stream.
Wayne - I’ve emailed you separately an example method to convert a BigDecimal to a packed decimal byte array. I tried to post the code to this forum, but the software here mungs the code.
--then Sharath’s suggestion is the way to go.