large file (>140MB) transfer fails between 2 Wm Servers

, ,
1

With the following error:
The problem is that whenever I try to send a 140meg file to CnnnB2B, I get
an immediate error message of the form:

$errorDump Server error occurred
$errorInfo $errorDump Server error occurred
$error OutOfMemoryError
$localizedError OutOfMemoryError
$errorType java.lang.reflect.InvocationTargetException
$service pub.file:getFile
$user Administrator
$time Wed Feb 20 10:55:16 PST 2002
$pipeline filename /home/wmethods/temp4
fileToSend /home/wmethods/temp4
fileToSendList /home/wmethods/temp4

remoteServerAlias CnnnB2B
remoteServerAliasList CnnnB2B

$callStack $service pub.file:getFile
$currentPath

$service CexpTrigoTestGD:SendGD
$currentPath /0

$error OutOfMemoryError
$errorType java.lang.reflect.InvocationTargetException

  1. What am I shipping

A 140meg file, a quasi-XML file (it’s actually a couple XML files patched
together to make a big file.)

  1. From where to where

corp data center to remote data center, smaller files upto 120MB file transfer goes thru fine between 2 wm servers

  1. Version of B2B

4.0.1, build 697

  1. Patch level

Dunno, not sure to find this out. We did just install 4 patches, though.

  1. Java version

1.2.2 (46.0)

  1. OS version

2.4.14 Debian Linux

2

I don’t have a solution for your problem but I need your advice on something else. I see that you are handling large documents. We also need to be able to handle documents as large as 200 MB and I need to estimate the hardware support that we will require for this. Could you please tell me about the hardware/operating system that you are using to support large documents on your end.
Thanks.

3

We have created a service that splits the large files into smaller files.

4

Hi There:
In reference to this topic we have large batch application files that we need to send outbound from TN and our IS server. Today we only do POR’s but have noticed that even a batch of 200 POR’s dropped at the door of the IS server can take up to an hour to process. This is a new feature (Large File) for us so anybody have any tips on how we can utilize this? We also want to do invoicing next which is 5 times as large. How do we accomodate this?
Thanks

5

We are having large file problems also.

Any chance we could look at/use the service you have created?

Regards, D.

6

I have a requirement to receive large flat files (>50M) via FTP, encrypt them, and pass them on to another webMethods system through our firewall.
I receive the file via FTP, and am able to encrypt it, but when I try to pass the encrypted data to the other system I am having problems.
I first attempted to read the data back into a string, but I often get an “out of memory” error. The error looks similar to the one described in this initial posting.

IfI manage to get the data into the pipeline, a remote:invoke or gd:send seems to work, but the target system receives truncated data.

Does anyone know what is happening here, or can suggest an alternative way to pass this data?

Any ideas would be greatly appreciated.

7

i would suggest writing the encrypted file back to the disk, then use a java service
that sends chuncks of the file to the other server,

for example the service reads 100 KB and sends it as the byte array into the remote invoke of the service on the remote servver.
this could be done using a guaranteed delivery mechanism, on the other server, another service is needed to recieve the byte array and append it the newly created file.

hope this helps.

8

actually to be more percise i bring part of the code we developed for a similar purpose,

consider that the service (srv) is a simple writeToFile jave service on the remote host,

public String transactU(String ifc, String svc, Values data, long TTL, int RETRY_NUMBER){
        try{
            String transactionID = tc.startTx(TTL, RETRY_NUMBER);
            System.out.println("Start: UTransaction ID received:"+transactionID);
            Values out = tc.invokeTx(transactionID, ifc, svc, data);
            System.out.println("Invoke: IData output received:"+out.toString());
            tc.endTx(transactionID);
            System.out.println("UTransaction Ended Successfully");
            return new String("OK");
        }catch(TXException e){
            return new String("Error # 002-UTransaction.TXException error: " + e.getMessage());
        }catch(DeliveryException e){
            return new String("Error # 003-UTransaction.DeliveryException error: " + e.getMessage());
        }catch(AccessException e){
            return new String("Error # 004-UTransaction.AccessException error: " + e.getMessage());
        }catch(ServiceException e){
            return new String("Error # 001-UTransaction.ServiceException: " + e.getMessage());
        }
    }