How to transfer large message from Frontend webMethods server to backend webMethods server

, ,
1

Hello experts

FE flow service → 1 MB file → BE flow service

From our frontend webMethods server I need to send a 1 MB file to our backend webMethods server, that is call a flow service on the backend service where the large file is the input to that backend flow service.

I have tried these approaches but in vain:

  1. using a node reference when calling the BE service
  2. convert the file to bytes and transfer the bytes
  3. transfer the file as stream using pub.io:stringToStream before sending to BE (I get “invalid stream type”)

Question:
What is the best way to transfer the 1 MB file from a FE flow service to the BE flow service?

Kind regards Mikael

2

1 MB is not very large now days. it can be easily held in memory in form of String or bytes. Are you using pub.remote:invoke? if not, try it, with the payload in String form first.

3

If you can send the file in an Async approach then think of implementing a Pub-Sub Integration between FE and BE Integration Servers.

  1. Write the file to the common mount where both FE and BE can access.
  2. Publish a Message with File Name with Path.
  3. Subscribing Service to read the file content from the shared path and process as per your business needs.
4

Another approach to consider. Keep the data as a stream all the way.

For either FE or BE server, never load the entire “file” into memory. This means no use of strings, nodes (in general, the way they are typically used) or anything else that loads an entire “object” into memory. When receiving the data from a caller (at the FE or the BE), configure things such that your service gets access to the stream – don’t have the “helpers” convert to an object for you. Handle the stream yourself.

Using a file as an example:

getFile as stream.

Use HTTP or whatever protocol you want to call the BE sever. Pass that stream to the service. For HTTP you’ll want to specify chunked transport so that it doesn’t try to load all the bytes into memory to determine the Content-Length

On the BE server, do not load the incoming stream to a string. If you use a node, use node iteration (which can be cumbersome). Or just write the stream to a local file.

1MB is indeed not very big – but when using the typical built-in services it will get duplicated early and often, which might become a memory issue. :slight_smile: