I have a flow service which uses FTP Login, GET, PUT (in another FTP location), Delete.
I need to pick the files up from an FTP location (LINUX server) for every 5 sec (which cannot be done using a scheduler set up, as 1 min is the least option allowed).
As per my knowledge, file polling can be used only for Windows file share directories. So in my case, should i think of using scheduler with a minimum of 1 minute time interval???
Plz shed some light on this. Also let me know if there are any other approaches.
File polling can use any network sharing mechanism. In Unix, as long as it is a mounted volume and the proper privileges are set, IS can get at the directory.
Another approach to consider is using an external scheduler. Maybe cron or something else more sophisticated. It would invoke a service over http, and the service would then do the FTP work.
Does the 5 second interval mean every 5 seconds? Or just a wait of 5 seconds between runs? If the former, be careful about overrunning your server if the FTP processing can take a while.
If the latter, you can use the IS services to programmatically create one-time scheduled tasks–the last step in the service would be to define a one-time task to run itself 5 seconds from now.
Thanks for your reply, Rob. Yes, the 5 sec interval means, that my FS need to be invoked for every 5 secs. It’s a very simple service which can be finished within 5 secs lest it should fail due to an FTP connectivity issue.
Do you mean to say that the directory where the files are placed on LINUX servers should be a mounted volume (file share network drive which doesn’t require any logins as Windows file share directory) with proper privileges???
If true, then i can use file poller. If not, the only way left for me is to use WM complex repeating Scheduler task. Am i right???
If your IS is on Windows and the directory is on Linux, there are a couple of options:
Software on the Windows box to mount the Linux volume as a mapped drive. To Windows apps, it will appear as a local drive.
Software on the Linux box to expose the volume as a Windows share (something like Samba). The account under which IS runs as a service on Windows will need to have network privileges and be able to access that exposed UNC share–e.g. \someserver\volume
Thanks for your prompt response, Rob. I think, Option 1 can be used in our case. Option 2 cannot be used, as Linux is a 3rd party server in our case and we dont hold any rights on it.
Regarding option 1, can you tell me how to map Linux volume as a map drive on Windows. What should i tell the guys on Linux side to do to achieve this?
I’m sure this has been talked about or thought of, but what’s the reason for sticking with FTP instead of using a more “real-time” approach?
FTP works well in more “batch-oriented” scenarios, where the schedule is measured in several minutes or hours. IMHO, when you start talking about picking up files every 5 seconds, you may want to start looking for alternatives.
If getting away from FTP is not an option, but the sending system has the ability to change the location to which it FTP’s the file, then another option may even be for the sending system to FTP the file to the IS directly via the IS’s FTP port. Once the file is sent, the service to which the file was FTP’ed is invoked automatically so there’s no need for the Scheduler or File Polling. I’ve heard mixed feelings about using this feature of the IS, but it’s worked well for me in the past.
I agree with Percio that polling every 5 seconds seems odd.
For the option 1, the only thing the Linux admin would need to do is to make sure the right account has the needed privileges. To get Windows to recognize the Linux volume, you’ll need software on the Windows box. There are many, many options for this. Do a search of the web for software that provides facilities, such as NFS, to access Unix volumes from Windows.
I completely agree with you that it seems odd to pick up files every 5 seconds. But I was told that we will expect huge volume of data during the day time and we need to pick up all the data. Moreover, we were told that we MUST pick up the data FROM THEIR Linux servers. So i’m left with no other option of using a scheduler service to run my FS for every 1 minute lest they give us access to their exposed UNC share.
Rob,
Thanks for the info. I’ll search the web for exposing the directory as UNC share and let them know of the available options.
“I’ll search the web for exposing the directory as UNC share and let them know of the available options.”
You mentioned that the Linux is a 3rd party server and that you can’t make any changes to it. Thus, don’t look for exposing that as a UNC share since that will require changes on the Linux box. Instead, look for solutions where the Windows box is a client to the Linux box, using standard Unix protocols such as NFS. On the Windows side, such a solution will appear as a mapped drive.
Sorry for the delay in my response and Thanks for your suggestion, Rob. We’re still considering various options that can be implemented on Windows side.