Pipeline data being offset during an FTP process and Convert

Flat file transmitted via FTP. I have a flat file with 32 records on an AS400 that we FTP to the IS FTP port that kicks off a service, and the first step in the service is a convertToValues. Our flat file is fixed length with no record id’s. When saving the pipeline directly after the convertToValues, and then restoring after executing the FTP the 13th record appears to have not completely converted and does not list all of the fields, which then throws all of the subsequent records off.

Has anyone ever experienced this?

I’m not specifically familar with the FlatFile stuff, but here are some generic FTP comments:

Is it possible tht the FTP client is running in text mode and therefore converting the file contents? If not as simple as switching to binary mode, two tests come to mind:

  1. Write a Java service that reads the incoming stream and just writes it to disk. You can then compare the disk file with the source file and make sure there isn’t something in the transfer. If there is, try test #2.

  2. FTP from the AS400 client to another FTP Server and then compare the files. If there is still a difference, then the problem is in the FTP client and not IS.


After working with Support for 4 days, it was an encoding issue coming from the AS400 to Unix.