Max Limit on number of mappings in a map step

, ,
1

Dear Members,

I solicit your opinion on the questions listed below:
Is there any limit on the number of mappings inside a map step?
Is there any limit on number of invocations of transformers like pub.string:trim or pub.string:replace in a map step? or in general what’s the max limit of threads of flow service that can be invoked.

Thanks,
Srivatsa

2

While these sound suspiciously like interview questions…

To my knowledge, there is no technical limit to the number of mappings in a map step nor to the number of transformers called.

The questions seem to imply a perception that mappings or transformer calls are each run in their own thread. They are not. Although the documentation says “treat transformers as though they run in their own thread” they do not do so. It is more of a caveat to emphasize that one cannot rely on the order in which the transformers will be called.

3

Ha ha no it’s not an interview question for sure :slight_smile: Thanks for the reply.

The questions were based on the problem I am facing in one of the interfaces.

The scenario goes like this:

  • We have a map step which has around 61 mappings and each link in the map step calls a transformer before mapping it to target side.
  • the transformer mentioned above is nothing but a service which does a String trim on the trailing side i.e a wrapper service which internally calls pub.string:replace which uses the regex “\s+$” to remove trailing white spaces
  • sometimes the input passed to this transformer could be null data, for which it returns a null output.
  • In our map step after the last mapping link, the map step throws a class cast exception.

the above being the scenario, I did a debug to see what was causing it to fail and hypothesized that transformer could be the problem as some inputs to it were null values.

  • When passed null value the transformer just returns back null as output(as expected) and not exception
  • Then I created a dummy service which used the same document on the target side as in the scenario just explained and passed null values to transformer and then mapped it to target document there too it didn’t throw an exception.
  • So I disabled the last few steps in actual scenario to see if a particular field was causing exception. But exception is thrown from what becomes last step in mapping after disabling few mappings.

So arose my question about threads.

Thanks,
Srivatsa

4

Gentle advice, from “How to ask questions…”

“It’s not useful to [say] what you think is causing your problem. …make sure you’re telling them the raw symptoms of what goes wrong, rather than your interpretations and theories.”

Following this principle, the content in your second post is what you probably should have started out with. The thread aspect was the wrong avenue to explore.

Keep in mind that the order of transformer execution and maps is undefined–they can be executed in any order. Have you stepped into the map step? You should be able to see which specific transformer is throwing the exception.

The class cast exception is probably the best thing to chase down. Is there a field or two, on either source or target side, that is not a string? Watch the pipeline carefully at runtime as it will show you the run-time type–which may differ from what design-time may show. E.g. a string var holding a date string may actually be a Date object at run-time (just an example–not sure what types you’re working with).

5

Thanks I will keep that in mind for future posts.

Yes I have stepped in to map step in debug mode. The target fields in the document are all String fields and "Allow Null: property of field is also set to “true”; Source fields are also String fields. The exception is thrown from “pub.string:replace”. When I checked in debug mode the exception is thrown after the last map step. At that point I found the source data was null, so to analyse the problem I created a map step with the same target document as in actual scenario and mapped from source different data like “null” and string data through the “pub.string:replace” service, there it didn’t throw an exception. String replace service just returns a null if passed null data and that gets mapped.

6

Apologies for updating this thread a bit late. We actually found out the field which was causing the problem. It turned out to be simple enough though it didn’t strike me in the first place.

it had to do with string list passed to pub.string:replace transformer hence the cast exception as reamon had suggested.

the intriguing part was that the source field though declared as just String variable was carrying StringList data and this exception was actually thrown within the first 10 mappings when viewed from Designer view, whereas when run in debug mode the exception is thrown after the last map step(again from designer point of view).

is it because map step waits for all the mappings inside it to finish before throwing the exception?