Reading binarytext files

, ,
1

Hi,

I am having this problem of “out of memory” error again.

There is a 5MB pdf file in the hard disk.
I read the file using pub.file:getFile service (as bytes),
after reading, I pass the bytes to pub.string:base64Encode service.
It takes lot of time hanging on the base64Encode service and throws “out of Memory” error.
Is there a beter way to handle this ?

Thanks and Regards
Jay

2

One way would be to process the file as a stream instead of loading it all in memory. The approach would be something like:

getFile as stream
open an output stream to file **
loop until eof
…read some number of bytes from the stream (1k, 5k, ??k) **
…base64Encode those bytes
…write the encoded bytes to the output stream **
end loop
flush output **
close **

Items marked with ** are things you’d need to write as Java services. Or you could do the whole operation in a single Java service, invoking the base64Encode using the wM API. You might want to write your own open/close services to replace pub.file:getFile, since we don’t know how it works internally and there is no close service to release the stream/underlying file handle.

Of course this doesn’t address transporting the encoded file to its destination but you can use a stream approach with the built-in services (i.e. pub.client:ftp accepts a filename for put which presumably does not load the entire file into memory).

HTH

3

Thanks Rob.
We are intending to use webMethods GD facility to send this encoded string to the customer (also uses webMethods B2B Server4.6).
Can we use this mechanism or we need to develop our own API’s to send this across to the other party?
Any ideas?

rgds
Jay

4

The GD facility will be fine. The key to remember on the GD services is that they don’t actually do any of the transport work themselves. They just make sure your desired service is run and returns success. In your case, you’ll probably want to have GD invoke a custom service that 1) loads a file as a stream; 2) invokes pub.client:http, passing the stream.

I vaguely remember someone posting a caution about the http service but I can’t seem to find it. The caution was that the service read the entire stream into memory so that it could set the content-length header. Can anyone shed light on this?