I have a small problem with sftp. We have couple of Integrations that do sftp to remote servers after processing the data. We are facing with the following error:
Error Message: I/O Exception running command: java.io.IOException: Cannot run program “sftp”: error=12, Not enough space
We have students requesting for Transcripts and when we process the request and use SFTP to drop the file on remote server. Once this error starts happening, all the Integrations that uses sftp are failing due to this.
SoftwareAG Resolution:
Problem:
I/O Exception running command: java.io.IOException: Cannot run program “sftp”: error=12, Not enough space
Resolution:
This particular error is an OS error. It usually involves swap space. The recommendation is to allocate at least swap equal to 4 x JVM_MAXIMUM_MEMORY or 2 x PHYSICAL RAM whichever is larger.
Increase the “swap space” should resolve the error.
NOTE: The above solution didn’t help us.
Part time Solution: We have to re-start the Integration Server whenever we have this kind of issue.
Did anybody has to face this kid of issue, if so please let us know how you were able to resolve the issue.
Also I would say please re-open the SAG ticket and troubleshoot more as it seems issue is back and not fully resolved may be involve your target OS team also in the call for the root cause analysis sake as it sounds to be purely an internal OS issue with the remote sftp server tself.
Let me break down the problem a bit and perhaps it will help you find your issue.
Your code is calling a native program, namely sftp. Unlike the execution of code within the JVM itself (e.g. running a simple Flow or Java service), the execution of a native program requires resources (including thread(s) and memory) in addition to those that are already allocated to the Integration Server’s JVM. This is one reason why it’s better to stick with pure Java solutions over solutions that rely on making native calls.
This error indicates that you’re running out of physical space, meaning: the space already allocated to applications like the IS + the space needed to make these sftp calls > the physical amount of memory of the server.
This could mean a few things. For example:
You need more physical memory (i.e. everything is configured correctly - your server is just not big enough)
You are allocating too much memory to your IS (and other applications on the server - if any), leaving very little physical memory for these other native calls to execute
You could have a memory leak which is gradually eating up your memory and eventually making you run out
You could have hung threads which are holding on to the memory
With this information in mind, you now need to do some investigation to understand exactly what is happening.
Start by looking at how much memory your box has and how much memory is allocated to your applications (e.g. IS)? If you sum up the maximum memory allocated for each application on the server, does it leave enough physical memory for these other native calls?
Start monitoring your server’s memory. Does it get consumed more and more over time? If so, who is consuming the most memory? Is it normal or is there a leak?
Start monitoring the processes/threads on your server. Do you see an accumulation of a certain type of process over time? Perhaps you start seeing many sftp processes executing simultaneously. If so, why? Are they hung? Perhaps a timeout was not set?
And by all means, as RMG recommended, go ahead and reopen the ticket with SAG.
To add to Percio’s point about “stick with pure java”, Integration Server does have built in SFTP services starting with 9.0. You should not have to call out to external services to get this capability.
Earlier release(WM 8.2) we were using pub.openssh:sftp, which I don’t like it. The above ones are really proper services to use. Session key, login, get/put do all the stuff and logout. I am planning on changing to the above services now as our upgrade progresses.
I didn’t give up yet, I still have the ticket open with SoftwareAG. I will keep you guys posted.
The mentioned “pub.client.sftp:*” services have been introduced with 9.5.
We switched our implementation from the WmSSH2 package (a SAG Consulting Package) to the build-in services during our migration from 7.1 to 9.5.
Except for the public key authentication we got them working. We keep investigating the public key authentication as one of our partner systems requests this.
Both variants (the WmSSH2-Package and the build-in serivces) are using Jcraft Secure Channel as library.
Benefit of using the new build-in services is that they are covered by the support team, where the WmSSH2-Package and the openSSH-Package were not.
