Flat files Via FTP

karupuv · June 22, 2005, 2:36pm

Hi,
i have created the Fixed length Flat file schema in the developer.For this schema document i have to assign values from the incoming flat files via FTP.
Please help me to write a flow service to receive the flat file via FTP.And let me know how can we get the filename of the incoming flatfile.It is urgent.
Thanks,
Viji.

gupta_r.17495 · June 22, 2005, 7:32pm

Vijaya,

FTPing a flatfile say (ff.txt)your service input should be ffdata(object–stream)then your sample flow steps are
streamTobytes(convert ffdata to bytes)
bytesToString(convert bytes to flatfile string)
pub.flatfile:convertToValues(This will parse the flatfile based on the FFSchema you have created…gives output ffValues (IDATA Document))
start Mapping…

To get the incoming flatfile orginal name use the service
pub.flow:getTransportInfo see the output transport/ftp/filename and extract it from the pipeline.

Pls do Search this forum if you have more queries on flatfile processing,many threads discussed on this procedure.

HTH,
RMG

karupuv · June 23, 2005, 9:56am

Hi RMG,
Thanks to ur valuable reply.i have some doubt in this.As u said when we design the flow service to accept the flat file from IS ,it should inlcude all the steps what you have told above.
What is the use "pub.client.ftp " whether it is used to send Flat file to Is or receive the flat file from IS.
I think,i have to include this service by setting the Command as “GET” as a first step in the service which includes all of ur above steps.
If it is not So, clarify me.

Thanks,
Viji.

vikashanil · June 23, 2005, 10:35am

I am using IS and Developer 4.6.
I am very new to webMethos and want to write a small service which can read a XML file and copy the content of the XML file to SQL Server database. I am able to write the service which reads the XML file from d: and copy the record to SQL Server 2000. Now I want to do the same by supplying the XML file thru ftp.
Till now what I did is

I made my service as startup service
When I am putting my XML file to [url]ftp://localhost:7777/ns/DBAccess/LoadTrueFTP[/url] folder, I am able to see the file name, type, loginID using pub.flow:getTransportInfo.
I am using pub.io:streamToBytes to read the content of teh file, but I am unable to do that. the revices I have used is –

pub.io:streamToBytes
pub.string:bytesToString
pub.web:stringToDocument
pub.web:queryDocument
…

With no input defined for service. Please help me how to do this. Remamber I am using version 4.6.

mwr0707 · June 23, 2005, 5:57pm

pub.client:ftp is a client service. This means that you use it to access information from an external FTP server.

The steps that Ram gave are for setting up an FTP port on your Integration Server. This makes your Integration Server appear very much like any other FTP server except you may be using another port instead of the usual 21.

When an external client, for example, FTP at a Windows command-line prompt (And yes, it could even be an Integration Server executing pub.client:ftp) opens a session with your integration server their commands actually execute a service even though it looks to the client like a file is being accessed.

Please take a look at the Administrator and Developer guides as they explain how this works very well.

Regards

gupta_r.17495 · June 23, 2005, 7:29pm

Vijaya,
What is the use "pub.client.ftp " whether it is used to send Flat file to Is or receive the flat file from IS.
I think,i have to include this service by setting the Command as “GET” as a first step in the service which includes all of ur above steps.

yes,using GET it will pull the files from the remote location or shared network folder and then using PUT command place the files directly to the ns/service and follow the above steps.

HTH,
RMG

gupta_r.17495 · June 23, 2005, 7:37pm

Vikash,

When you ftp PUT xml or any files directly to the ns/service just set the service input as contentStream(object)which will be in the pipeline and this makes streamTobytes works as expected.But in IS6.x if you specify content-type “text/xml” in the filepolling section the service input should be node(object).

goodluck,

HTH,
RMG

vikashanil · June 27, 2005, 2:29pm

Hi RMG,
Thanks for your reply. I have added an Object type Input in InputOutput tab with the name contentStream and mapped the ‘stream’ service-in parameter of the streamToBytes service with contentStream. But still I am unable to do the load. Please help me how to make the streamToBytes service work in Developer 4.6. It is urgent.

karupuv · June 27, 2005, 3:27pm

Hi RMG,
Thanks to ur reply.I will use the same steps in my service for receiving ,incoming Flatfile(ff.txt) from IS to the developer.I have created the Fixed length Flat file schema with the format like,
the schema has 3 records PurchaeseOrder,Header,Lines.The purchaseOrder contains header and line records only.The hierarchy of the schema is
----- Purchase Order[0] (Main rec-repeatable)
---------Header (unique)
-----------Account
-----------PO Number
-----------Customer’s PO
-----------Currency Code
-----------Bill to date
---------Lines[0] (repeatable)
-----------POnumber
-----------Line number
-----------ModelNAme
-----------ModelCode
----------- .
----------- .
----------- .

----PurchaseOrder[1]
------- Header
-----------.
-----------.
--------Lines[1]
-----------.
-----------.
The output of this record values must be,

PurchaseOrder[0]HeaderAcc100PO200CusPO200Curcod2013/03/2005Lines[0]PO200Line300Model1Code34Lines[1]…

For this schema,
How the ff.txt file should be looking like?Can you please let me know the content of this file so that i can pass values and test the service.

Thanks in advance,
Viji.

gupta_r.17495 · June 27, 2005, 7:15pm

Vijaya,

Use any texteditor like ultraedit and put the data in the fixed positions including RecordDefinitions as per defined in the schema and make sure the data is the right positions,so that FFSchema parses successfully.

HTH,
RMG

karupuv · June 28, 2005, 3:11pm

Hi RMG,
The attached one is the sample of Flat file Format which to be sent to the Integration Server via FTP . I have created the Fixed length flat file schema in the wMDeveloper by setting the record length of 1169 .
By referring this POFormat document each record occupies 1169 characters Length.The parent record PurchaseOrder include the child records Header and Line.
The problem is this record contains 1156 blanks following the name PurchaseOrder so that it fits the entire width of the record(1169).
When sending the flat file to IS , the user has to design the flat file with 1156 blankspaces after entering the name PurchaseOrder.Is it so?
can u pls send me the sample value flat file(ff.txt) for this schema?flatfile

POFormat.doc (24.6 k)

gupta_r.17495 · June 28, 2005, 8:09pm

Vijaya,
yes the user has to design the flat file with 1156 blankspaces when sending the flatfile and also in the FFSchema you should add Blank FieldDefinition(contains 1156 spaces)under the PurchaseOrder RecordDefinition.So this makes parsing successful.

Hope you are clear on this.

HTH,
RMG

vikashanil · June 28, 2005, 10:48pm

Hi RMG,
Please help me also.
Thanks in advance.
–Vikash

gupta_r.17495 · June 28, 2005, 11:09pm

Vikash,

Please elaborate on your FTP processing flow?

Actually contentStream input should work.For debugging purpose try to do this way…After the StreamTobytes service call getTransportInfo and savepipeline service and later restore it check what is coming in the pipeline.

Note:Stream objects cannot be saved and it will show up with different notation but atleast you will know whats going on when ftp transport puts file on the ns/service.

HTH,
RMG.

vikashanil · June 28, 2005, 11:27pm

Hi RMG,
Thanks for your reply.

I am using developer 4.6 I want to insert the records in SQL Server from an XML/Flat file. I want that the XML/Flat file should should be submitted thru FTP. For this I have configured the ftp thru IS and my ftp server is working on port 7777. At presently I am using single machine for all. Now I have created a Package Vik, under that I have a Folder DBAccess with in that I have 5 services. I setup the startup service to LoadThruFTP as this is my mail service. Within this service I am calling streamToBytes as the first service. After the flow is

pub.io:streamToBytes
getTransportInfo
SavePipelinetofile
pub.string:bytesToString
pub.web:stringToDocument
pub.web:queryDocument
pub.string:bytesToString
pub.web:stringToDocument
pub.web:queryDocument

If I it in the above sequence then no result get stored into the pipeline file.But if i remove the streamToBytes service then the transport information is getting stored into the pipeline file. It seems that the streamToBytes service is not working perfectly. I tried using contentStream also. For this I have added the contentStream into Input to the service as Object type and mapped it to content variable of the service-in of StreamToBytes.
I tried all but no sucess. Please Guide.

Thanks

gupta_r.17495 · June 28, 2005, 11:38pm

What variables you are seeing in the transportinfo pipeline?
First tell me are you ftping the file directly to ns/service directly?
Are you setting content-type=“text/xml” in the ftp process.

HTH,
RMG

vikashanil · June 29, 2005, 12:41am

Hi,
Yes, I am directly ftping to ns/Service.

gupta_r.17495 · June 29, 2005, 12:46am

Try setting node(object)in the service input and eliminate streamToString and use documentToRecord.

HTH,
RMG

karupuv · June 29, 2005, 9:45am

Hai RMG,
Thanks to ur cooperation.Again i have one problem.I hope you will help to me.
How can we set the default value(blank spaces)for a field definition of fixed position 13-1169.In the property panel i have selected the position property in thati have selected the “default
value” checkbox.But it is not assigned.
And also clarify me,i have a field called trailer which should have the total number of records as its value.How can I receive the total records and assign to trailer.
Please reply ,it is very urgent,.
Thanks,
Viji

vikashanil · June 29, 2005, 9:47am

Hi RMG,

Thanks it is working now.

Vikash