File age can be a workable approach. According to the IS Administrator’s Guide, the file age is based on the file’s last modified time. If the file age is set to 3 minutes and the current time is 3 minutes beyond the file age, the file will be processed.
The file polling interval will determine how long the polling waits, not how much longer needs to pass before the file has reached the file age. So if the polling runs every 5 minutes, and the file was 10 seconds from being “old enough” it will be picked up at the next 5 minute poll, not in 10 seconds.
I tried the filepolling part by giving the fileage to 120 secs. I am picking the files from UNIX folders. But when I tried to write few sample files.
I have written first file…after 45 secs the second file and after 45 seconds the third file. Now, I have enabled the filepolling immediately and it waited for 2 mins and picked up all the 3 files in under 3-4 secs. I doubt it will not work and will pick the half files too.
Can you please assist if there is any better approach using coding etc/ may be digging more into filepolling can help?
File age of 120 seconds–by the time the file polling picked up the files they were all older than that.
What is the File Polling Interval you have set?
It will indeed pick up partial files–if the wait time (file age) isn’t long enough. How long that needs to be depends upon what is writing the file and how often/quickly the modified time on the file is updated.
The better approaches:
Rename files after writing
Use a lock file (source writes lock file, writes file, deletes lock file–polling waits until lock file is gone)
But since you indicated the source cannot be changed, you’re restricted to using file age. Custom code will not handle things any better than does the file polling port.
I tried it again. Now, The filepolling was enabled from the beginning. What I did was I have written a sample file 1. The fileage is 120 seconds and the polling interval is 15 secs.
This time around 100th second, I created the second file. The filepolling picked the first file at 120th sec but it also picked up the second file after 15 secs. It did not wait till 120 secs. Am I doing anything wrong.?
I have the same situation like you. Can you please tell what was your approach finally. Also, Can you please share the code which you’re checking for the filesize. Its very urgent for me.