Can’t believe I have to ask this after this long using the webMethods…but never had to do this before.
I’m trying to use the pub.client.sftp:ls command to filter on filename to exclude certain extensions.
Files always start with X12.123 however once the files are downloaded with “get”, the source system appends a .downloaded extension to the filename. An example is “X12.123.downloaded”
I need to know how to grab all files except the .downloaded files so we don’t processes the same files each time.
Is this possible?
Matt, per my knowledge, the ls wM service does not allow regEx, but can you try using regEx in the path parameter to check?
If it doesn’t allow it as I suspect, then you can -
Get a listing and filter the filenames (regEx) in your flow, before you invoke get to fetch the desired file in a loop
I think (theoretically) you can write something similar to executeOSCommand specifically for SFTP with relevant inputs (haven’t tried this for remote execution over SFTP, just brainstorming)
Change the design a bit, so that after you download a file, you move it to a “completed” folder on the remote server
These are some I can think of right now.
P.S - You can consider Active Transfer (managed file transfer) if you need an enterprise-grade solution, but it is overkill if you only have a small file integration footprint.
We created a wrapper for pub.client.sftp:ls to add regex support (and a couple of other features). The scheme used was:
Caller encloses the filename portion with [ ] to denote the presence of regex. This also distinguishes between regex and glob as the base ls service supports glob.
Wrapper checks the path to see regex specified. If so, splits off the regex, using the parent only in the call to the base ls.
Filters the list returned by the base ls to match the regex. For this, we created our own “public” package service that uses java.util.regex.Matcher and Pattern for matching. You could also potentially use java.nio.file.PathMatcher, which uses Matcher/Pattern underneath.
Obviously the description of the steps is a bit simplified, but works.