Newbie Best Practices for Data migration

Hello,

We just purchased PeopleSoft EnterpriseOne latest version (not installed yet, sorry not sure of the version exactly yet).

We are migrating ERP system - BPCS - on an AS400 to E1 on a new AS400, and we are trying to figure out how best to move data to and from both systems. We don’t really have a need for anything else except ERP system migration, so we really would not use any B2B functions or complex message brokering in WM.

We know XPI ships with the product, but are looking at EAI solutions, especially webMethods.

I see in these forums that there are a lot of issues with the E1 adapter.

We have not decided yet if we are going to just work at the z-file database level or if we want to work at the E1 application level.

I read in another post that we don’t even need the E1 adapter for z-file communications, that we could just use JDBC. Is this correct?

Would anyone share their point of view on how best to work with E1?

Can I just use XPI that ships with the product successfully? Should we purchase webMethods Integration Broker and their E1 adapter?

Keep in mind that this project is only the migration of ERP and we do not need complete data synchronization, nor do we need real-time data flows as well. Batch would be sufficient.

Any input is greatly appreciated, we want to know what we are getting ourselves into and to make sure we are on the right path.

Do you guys think it is better if we just work with the z-files, or should we dare to try the application object layer?

Thanks!

Rodney

Rodney,

I hope this doesn’t sound like heresy on a webMethods forum… and speaking as a webMethods enthusiast, from an architectural point of view, webMethods Integration Server (or any other EAI tool) is not the ideal technology for what you are doing.

You said the project is only the migration of ERP, you do not need complete data synchronization, or real-time data flows, batch would be sufficient.

In order to make possible all the functions that you do not need in this situation, EAI tools sacrifice throughput. What size machine will you be running the data migration on, and how many rows processed per-second is acceptable?

This has all the characteristics of an ETL problem best attacked with a tool like Informatica, Embarcadero, Microsoft DTS, etc.

In a batch bulk-processing race, an ETL will outperform an EAI by factors of 10, 100 or more.

If you work hard enough you can force an EAI to do an ETL job and vice versa. ( If determined you can use a screwdriver as a hammer ).

(Ducks expected volley of tomatoes)

Now having said all of that… I’d be really surprised if a real-time application (EAI) and/or B2B integration scenario doesn’t emerge with your new E1 system, at which point webMethods is an ideal solution. If used correctly, webMethods integrations offer exciting opportunities for Business Activity Monitoring and related business process improvement.

Regards

Mark,

Thank you very much for the prompt reply. Here are the answers to your questions, some more information and more questions.

The data is being migrated from one AS400 machine to another AS400 machine. Depending on the EAI tool, we will either be using a dedicated W2K server machine for the Integration engine, or we will install the engine on the E1 AS400. At this time, I do not know the figures for the amount of data or what is acceptable. The ETL tool that we use now is Cognos DecisionStream and a little of DTS. I know, there is quite a grey are between ETL and EAI tool functionality, we are looking for EAI tools that are the best at ETL.

The issue with ETL tools, like Informatica, is that they are really good at data extraction and creating a relation data warehouse. They are not so good in delivering the data into ERP systems, at least for what I was told and know. This is why we would like an EAI tool.

The ERP data migration and synchronization is what is driving this project and our short-term needs. FOr the long-term we are looking at Migrating our EDI processes to the EAI and we are looking for acquisitions, so an EAI tool will prepare us for these scenarios.

THe thing that confuses me is PeopleSoft E1 is shipped with XPI, a webMethods product, and I am trying to get a handle on if just using XPI is sufficient enough.

If we do decide to go with XPI and maybe upgrade it to the full WM EAI tool, do you feel that the E1 adapter works sufficiently for my needs?

Has anyone had any major, major issues with connecting to the application layer of PS E1 with WM?

I just want to get a handle on the technology from people who have actually used to tool with E1 and see if there are any show-stoppers.

Thanks!

Rodney

I agree with Mark. Neither the wM Integration Suite nor XPI are the right tools for the task you describe. In my opinion, there is no gray area between integration tools and ETL tools–integration tools are appropriate for near real-time interactions while ETL tools are appropriate for bulk data movement and transformation (data conversion/migration). It’s a good possibility that you will not be happy with the results of the data migration if you use wM or XPI. And actually, the most successful conversions I’ve seen didn’t use ETL either–they used custom code completely.

Of course that’s just my opinion, I could be wrong.

Rob,

Thank you very much for the reply and for the advice! I will take the information into consideration. Actually, the project was scoped out with manual custom coding. This is the reason why this project was initiated, to see if we could do it quicker and less expensive using EAI tools. The issue is that after the ERP system is migrated to E1, the old machine goes away and all the money and effort sunk into custom coding is gone. If we went with EAI, the processes would no longer work, but we would have an EAI tool for the future and maybe migrate EDI and set ourselves up for acquisitions.

The reason I say there is a gray area is because some EAI tools can do complex transformations, and some ETl tools (Informatica) indicate that they can do EAI functions.

I wish there was a tool out there that could do both with ease.

I have to find out more about XPI, because if we already have this shipped with E1, then we should use it. I don’t know what the rush would be for us to purchase the full upgrade to WM Fabric and a separate E1 adapter, if we could get by with XPI.

Thanks!

Rodney

Rodney,

For what it’s worth…

This does not look like a gray area to me. As Rob says, you are unlikely to be happy with an EAI tool in this situation.

I have used Informatica for data migration in the past with great success. Good developer productivity, and very high performance.
I didn’t end up throwing away all the code, because after implementation, we still needed a way to bulk-load data into certain tables in the new system.

What problems did you run into?

Regards