Usage of multiple transformers in a single map step - advice needed

, ,
1

Hi All,

My requirement is to trim all the fields in my document before sending that to the partner.
I have around 30 fields in my document.
Can I use a single map step and call 30 transformers (inbuilt trim svc)?
I doubt if this causes any performance issue.
I am thinking to do this in 4 to 5 map steps where I can call 6 to 7 transormers in single step.
Could you please let me know pros and cons of going with this approach?
Any other better suggestion is most welcome.

Thanks,
Venkat

2

Hi,

I would write a generic java service that can accept any document, recurse through it and trim all the fields. Is quicker and can be reused elsewhere. I did one such service and reused it, a few years back.

3

Using multiple transformers in a single map step is fine. The only real concern is maintenance–it can be a pain to review/edit each of the transformers.

The suggestion from haragopal is also a good approach. I too have done that in past projects.

4

In fact, there are a couple of services in PSUtilities that should do exactly that, ps.util.idata:trimFields and ps.util.idata:removeNullFields. Use these services as samples to help you create your own service.

5

Thanks all for sharing your expertise here.

I have a doubt here…

When Reamon says, “Using multiple transformers in a single map step is fine”

How many number of transformers can be used in a single map step?
What performance issues we have to consider here?
Is this purely based on the thread availability as transformers execute concurrently?

Thanks,
Venkat

6

There is no limit on the number of transformers.

There is no meaningful performance consideration.

Transformers are not executed concurrently. The documentation implies that they are but they are not. The section in the documentation is explaining how the order of transformer execution is not defined and to “assume that [IS] concurrently executes the transformers at run-time.” Thus, they imply that they are but they don’t explicitly state that they are. Experience of many is that they are not.

If one thinks about the implications of running each transformer in a thread one quickly concludes that doing so would be overhead intensive for little/to no gain. For example, a common use for transformers is to help translate data from one form to another. To be able to set the data into a target document, threads would need to be synchronized. The overhead for managing this would quickly outweigh any supposed performance gains.