FTP sessions sharing data

, ,
1

Has anyone experienced cases where two different interfaces running at the same time will cause collisions with each other due to sharing of data??? Let me give an example:

  • Interface #1 runs at 6:00AM daily using id ID1 logging into DIRECTORY1 looking for files with FORMAT1

  • Interface #2 runs every 20 mins(0, 20, 40) using ID1 logging into DIRECTORY1 looking for files with FORMAT2

Files for Interface #1 exist at 6am but not for Interface #2 but the logs show that the ls command used the settings for Interface #2 and not for #1 causing the interface to skip over the files it was supposed to process.

Has anyone experienced these sort of collisions before between FTP sessions running at the same time???

2

Jim,

This problem has caused me some grief recently.
You MUST write your ftp service so that a logout gets executed whether the service is successful or not. If a logout is not executed the ftp session remains open until it times out. I could not figure out exactly what that time limit was. Secondly set the timeout value on login service.
My services are coded like this.
SEQUENCE: (exit on SUCCESS)
---- SEQUENCE: (exit on FAILURE)
---------ftp commands
---- SEQUENCE: (exit on DONE)
---------error routine

With the above sequence structure if the ftp command errors out in drops in to my error routine which forces a logout. If the commands complete successfully the last step is also a logout.

The main problem is that the first FTP session is not being closed. You can search this group for a number of threads on that topic.

Good Luck!

3

Sorry Chris that’s not the problem… When using the FTP command in the WMPublic package there is no explicit logout command possible. No session key is returned by that so I cannot logout from the service… I do not follow the login, command, logout structure, the FTP service is supposed to summarize that to save on coding…

Also this doesnt address the sharing of data between two sessions running at the same time… The first FTP session is being closed but when it closes before the 2nd one is done, I get Not logged in errors from the 2nd one due to the sharing of the FTP session…

4

Jim,

I also ran into the above issues using pub.client:ftp.
This is a flow that uses:
pub.client.ftp: login
pub.client.ftp: cd
pub.client.ftp: ls
pub.client.ftp: get
pub.client.ftp: put
pub.client.ftp: logout

If any of the invokes between the login and the logout fails you will have a hanging session because the logout is not executed.
I ended up writing my own service to call the pub.client.ftp services so that I could control the error checking.

I’m sure if you put some pub.flow:debugLog statements into pub.client:ftp you will probably see which of the invokes is failing. My guest would be that when you execute the pub.client.ftp: get and the file does not exist it will return an error.

HTH

5

The solution is as follows:

apply WmPublic fix 126 - this fixes cases where the cd command within the FTP command fails… now it will throw an exception… this wasnt happening before…

wrap individual services that could possibly collide with a java service that invokes a separate context for each invocation… schedule that instead of the service directly… this avoid the sharing of session id’s, data, etc…

i need to reiterate that it WAS NOT the lack of a logout command causing problems… it was that two sessions that run at the exact same time using the same ID will share data…

6

Jim,

Thanks for posting the solution. Out of curiosity what was the behaviour when the cd fails prior to installing the fix? What did it return?
Thanks.

7

Jim,

Where can I find WmPublic fix 126? We applied IS_4-6_SP2. Does WmPublic fix 126 already included in IS_4-6_SP2?

8

Hey Chris,
What we were experiencing was connection drops between two of our servers located in different countries. It appears that WM would fail right at the cd step, but not throw an exception and WM would login again and continue its operations. We noticed files that would wind up in the home directory of the user(eg /home/smxgnrl) instead of the directory we were explicitly passing to the service on put commands. Of course if no exceptions are thrown we think everything is fine until users start complaining that they are not receiving files.

On get commands we noticed errors saying file not found, when the file was clearly in the right directory… Well it wasnt in the home directory so this is why the service failed… Same would happen if the connection dropped before a delete command… File isnt in home directory so delete command fails…

This may have been a huge issue with us because there are network connection issues between some of our servers, so it wouldnt surprise me if most implementations havent experienced this problem…

Id still highly recommend installing the patch if you have lots of FTP interfaces because it is a bug in the FTP implementation…

Also, we turned our logs up to 9 and only selected the following facilities: 12,13, 14, 26, 51, 55, 64, 71, 90

This will give you tons of detail on your FTP services activity without bombing the logs with crap you dont care about…

9

Here is the java code to wrap scheduled services in a context wrapper… The only way to avoid FTP session collision:

inputs:

server: your servername:port
ifc: interface name ie folder name
service: actual service to be run

Schedule this instead of the service…

// ------------Begin code---------------

Context user = new Context();
String server = new String();
String ifc = new String();
String service = new String();
String id = new String();
String pass = new String();

// pipeline in
IDataCursor pipelineCursor = pipeline.getCursor();
if ( pipelineCursor.first( “server” ) ) server = (String) pipelineCursor.getValue();

if ( pipelineCursor.first( “interface” ) ) ifc = (String) pipelineCursor.getValue();

if ( pipelineCursor.first( “service” ) ) service = (String) pipelineCursor.getValue();

if ( pipelineCursor.first( “id” ) ) id = (String) pipelineCursor.getValue();

if ( pipelineCursor.first( “password” ) ) pass = (String) pipelineCursor.getValue();

try {
user.connect(server, id, pass);

// pipeline out
pipelineCursor.last();
IData out = IDataFactory.create();

out = user.invoke(ifc, service, pipeline);

user.disconnect();

pipelineCursor.insertAfter( “out”, out );
pipelineCursor.destroy();

} catch (Exception e) {
Service.throwError(e);
}

// -----------End code-------------------

10

Not sure Min, request it from wmsupport directly… We dont have SP2 installed yet so it may be in that patch… Confirm with wm support…

11

Jim,

Thanks for the Java Code post.
Interestingly our case is still open with wm support and no one mentioned WmPublic fix 126.
I’ll forward this thread to my support rep.

12

Has anyone tried Jim’s code above.
I’m assuming that the id and pass should also be inputs to the service.
When I compile I’m getting an error on Context not found. What Class do I need to import? I’ve tried java.lang.Object but the error persisted.
Any ideas?

Java is not my strong point.

13

Import com.wm.app.b2b.client.Context.

14

Rob,

Thanks. Clean compile.
Where did you find this? I was searching the docs in the API but didn’t know where to look.

15

It’s in the API docs. You can get to those on the menu in Developer at “Help | Java API Reference” or you can open index.html in
[developer dir]/doc/API/Java.

16

When using the Java Service above, how exactly is the setup done?
Do I register the Java Service and assign it to the queue? If so and I have multiple queues all the Task will have the same delivery method, essentially the name of the Java Service,how will it know which task belongs to which queues.

I need to understand the correct procedure for implementing this service on scheduled queues.
Thanks.

17

Just wanted to thanks Jim for a great post regarding using a context wrapper for scheduled ftp services. It has solved a major ftp hang problem for us. Thanks Jim!

18

Hello, just a small enhancement for the code:
in order to throw the exception from the client
session you should use the catch block.

…
} catch (Exception e) {
throw new ServiceException(e.getMessage());
}

instead of

…
} catch (Exception e) {
Service.throwError(e);
}

Regards

19

I currently run into the webMethods sharing FTP session problem. Thanks for a great post regarding using a context wrapper for scheduled ftp services. One question to ask, when I try to run this Java service to connect to FTP server. I got following exception. And the service I want to run was not invoked by the wrapper service. What I can do to solve this problem?
[30963]2003-11-04 15:45:17 PST [ISP.0090.0004E] FTPContext –
java.net.SocketException: Connection reset by peer: socket closed
[30962]2003-11-04 15:43:24 PST [ISP.0090.0004E] FTPContext –
java.net.SocketException: Connection reset by peer: socket closed
[30961]2003-11-04 15:40:09 PST [ISP.0090.0004E] FTPContext –
java.net.ConnectException: Connection refused: no further information
[30960]2003-11-04 15:34:44 PST [ISP.0090.0004E] FTPContext –
java.net.ConnectException: Connection refused: no further information
[30959]2003-11-04 15:31:00 PST [ISP.0090.0004E] FTPContext –
java.net.ConnectException: Connection refused: no further information

20

Update requested: Does anyone know if these ftp collision problems are still apparent in 6.1 or can we use the built-in ftp service? I need to use ftp services and want to make sure I won’t run into similar problems. Can we use wm.pub.client:ftp still or do we need to do each step individually with logging in and out explictly?