We are using Enterprise Server and File I/O adapter to read flat files. But, we are not sure whether the polling operation in the I/O adapter will poll a file that is still be writing in UNIX box(not closed file). Does it only poll the writen-complelely files?
Thanks for your experience and answer!
ZG
It depends on whether or not the file is locked. I’m not sure on the File I/O adapter, but previous adapters (flat file adapter) would fail because it couldn’t open the file.
The safest way to do this is to write the file to one name, then rename it and have the I/O adapter poll for the new name/directory.
The other option would be to create an empty flat file with the ext .st file …so when you are reading the file the .st file would exist …as soon as the reading of your flat file is over delete the .st file . This code you can write in your ATC. This works in Unix as well.
You can use file detection and read the file using your ATC .I mean your ATC code will be triggered after recieving a notification event from the flat file adapter and start reading the flat file using file i/o java apis.
So when some other program tries to read that file …look out for that .st file…if it does not exist then start reading the flat file.
I hope this solves your problem …This we did in one of my previous project…we were facing the same problem.
I know that the I/O Adapter polling operation will detect a file that is being FTP’d before the FTP process is complete. I had to put a custom code step in the component that was triggered by the polling operation to read each line (using buffered java reads) to end of file, testing to see if the trailer record existed yet. If it didn’t exist, I did a thread.sleep for 2 minutes and then looked again. Once the trailer record was found, I allowed the integration to continue.
This worked well for awhile until one day we got a String Index Out Of Range exception in the custom Java code. That was because in a rare instance, my Java read got only 1/2 of a line (FTP process was in the process of writing that line at the same time as the read) and the string function I used to look at a field in the record tried to access past the end of the line. To fix this I added code to test for the length of the line, and if it was not a full record, I assumed the FTP was not complete and went into the sleep logic.
Tom.
Hi All,
I have a situation wherein a service has not completed writing a file and the poller has picked up the incomplete file. Is there any work around for this.
I am using the PSUtilities:file:writeToFile for writing to file, is there a bug with that or is it a bug in the File Polling port of webMethods.
Thanks
Gaurav
We do quite a bit of file poll operations and I have seen this problem before. In our experience, it is dependent on the Operating System not webMethods. Windows is pretty good about “locking” a file until it is completely written. Unix does not seem to lock the file so you can easily get a partial file.
The policy that we have adopted is that you build your script to write the file into a temp directory somewhere. Then the last step of the script is to move or rename the completed file to the target directory where you are polling. A file rename is usually quick enough that the poll operation then picks up the completed file.
To further dava.sabaka’s post: if you ensure the temp directory is on the same disk partition as the polling directory, then the file move will be instantaneous – the moment it appears in the polling directory it will be complete, as Unix is simply moving the directory entry.