Publishable document File size getting increase 10 times when publishing the IS doc to UM

, ,
1

Hi,

Please help me to understand why the publishable document file size getting increase to 10times when publishing it to UM. I am publishing a file of size 20MB but it is getting published to UM more than size of 200MB (Around 10times of actual size).

Thanks,
Lohit

2

Hi Lohit,
How are you checking the size of the message/event in UM?

Regards,
Bhaskar

3

Hi Bhaskar,

After publishing the doc to UM , i am checking it in server logs.

Because the max publishble size is 150MB in UM and the document size is around 200MB so it fails with below error:

DEFAULT_IS_JMS_CONNECTION: com.wm.app.b2b.server.jms.ResourceUnavailableException: javax.jms.IllegalStateException: MessageProducerImpl.send failed: An event element exceeds the max buffer size:20971520

Thanks,
Lohit

4

Hi Lohit,
Max buffer size of 20971520 amounts to 20MB not 200MB.

Regards,
Bhaskar.

5

Hi Bhaskar,

Thanks for correcting me.

Sorry UM is set to receive the filesize of around 20MB and we were trying to publish the doc of size of 16713 KB and we were getting the below error:

DEFAULT_IS_JMS_CONNECTION: com.wm.app.b2b.server.jms.ResourceUnavailableException: javax.jms.IllegalStateException: MessageProducerImpl.send failed: An event element exceeds the max buffer size:20971520

Thanks,
Lohit

6

I am not sure which version of UM you are using. I could successfully publish a message with a file of 19MB size.

7

I could successfully publish a JMS message with a file of 19MB size in 10.3

8

Hello Lohit,

What do you mean by 16713 KB? Is this 16 KB or 16 MB.
How do you measure the size of your payload?

Thanks,
Stefan

9

Hi Stefan,

it should be 16 MB in this case.

Regards,
Holger

10

Lohit,

I would suggest you to re-think the design approach that you are currently following as it is not a good practice to publish a huge message to UM, you can try other integration patterns.

If you are sure to go with the current approach, make sure you also check other factors like network, CPU, memory, disk space (persist the message) etc.,

There is no hard limit on MaxBufferSize but the above factors will play a role. Make sure you do a benchmark testing in your lower environments like (pre-prod/QA) with the expected volumes of messages.

11

Hi Mahesh

Yes, i have changed the approach and now i am not publishing 16MB of file data to UM.

But my doubt is that publishable doc gets increase from the actual file size when we publish it to UM ??because we had a setup of 20MB in UM and the file size we published to UM was 16MB and it was failing to publish to UM.

Thanks,
Lohit

12

UM should not increase the IData size and I am not sure what could be the issue, try reproducing the issues again with higher max buffer size and make sure you restart UM before you run the test.

Let see what others say about this.

13

Thanks Holger,

I was wondering where is the 10-fold increase claim coming from then.
Anyway, UM is adding a few hundred bytes for headers, but nothing as drastic as what is reported here.
It is however an entirely different question of what is given to UM as a payload, since the data may be encoded differently by the publisher application, compared to the 16MB payload that Lohit is seeing, hence my question, how and at what point is the size of the payload measured.

Thanks,
Stefan

14

Hi Stefan,

We are measuring the payload by the size of the input document. This is a file polling service which reads the file content and publishes the content to UM. Lohit is mapping the xml data to IS publishable doctype and then publishing to UM. This is not happening for other interfaces. Wondering what is causing only this interface to overshoot the size of the file published.

Thanks,
Kavitha

15

Hi All,

Lohit was mapping payload data again in the publishable doc which was causing the size to increase

16

Hi All,

Thanks for addressing the issue. Lohit was mapping extra payload data to the publishable doc which was causing the size issue.

He removed the mapping and its working fine.

Thanks,
Kavitha

17

Glad to hear that the issue is resolved and ty for the update!

Cheers!
RMG