This there any benefit in using transformers to invoke a service as opposed to just entering the service as a flow step?
I am trying to speed up some code that is taking a record of 30 entries and formating the fields into a single string. This string is then added to a buffer list which will contain all my records as strings. The number of records is currently 2000. This process can be run for 5 threads at a time.
There are 2 services involved in this loop, one to create the concatenated string and the other to add the string to the buffer list. They are both invoked as flow services. Would putting them in as transformers speed up my processing.
I know the simplest thing is to try it out but we are doing some data loads at present and I am forbidden from running any big jobs.
Hi Sheila,
Yes. In a flow service, wherever you see 2 or more services invoked sequentially and if the later service don’t use the transformed data of the previous service, then you can add all these services in a transformer for the parallel execution. This will reduce the total run time.
In a side note, I never call the string concatenation service. Instead, I use the “Variable Substitution” option offered by Flow.
e.g. If I have string1=“Bhawesh”, string2=“Singh” and if I need to populate string3=“Bhawesh Singh” (string1+" " +string2); I will not call the concatenation service, instead I will set string3 = %string1% %string2%. This makes Flow execution faster.
On what are you basing this comment? It is my understanding that multiple transformers in a single map step are invoked serially, not in parallel. The order of the invocation is undefined/indeterminate.
On what are you basing this comment? I’d be quite surprised to see variable substitution be faster than concatenation. Have you done some benchmark testing?
I always prefer transformer as invoke service not inside a map step.Sometmes if you use transformer in a map step and if this inside a Loop step you will see warning in the logs saying transformers cannot loop ad reason is when you map transformer to a doclist fields with out specifying index this happens.
Unless you have different source to target fields one to one mapping in a map step like you want to use trim transformer functions for multiple fields then it make sense to have in one map step.But if you just one concatination then directly invoke the concat service instead of map step transformer.
The same discussion held many times in this forum,please do search you may find it useful.
A search for “transformer vs. invoke” turned up this thread. Be sure to read to the end of the thread. Some of the early posts in it refer to old behavior.
Possibly. How are you appending each string to the buffer list? appendToStringList becomes very slow at some point, depending on your system. Usually around 1000 entries. You may want to consider creating Java services to place objects in a java.util.LinkedList or other appropriate Collection and when the list is fully populated, convert it to a String list (an array of strings) in the pipeline for subsequent looping/processing.
Hi Rob,
You wrote:
1>
“It is my understanding that multiple transformers in a single map step are invoked serially, not in parallel.”
That prompted me to take a relook at the Developer’s guide. the webMethods Developer Users Guide 65.pdf, Page 222 says:
“When inserting transformers, assume that webMethods Integration
Server concurrently executes the transformers at run time.”
Shouldn’t I infer from this that if one runs 2 service as transformer in a map step both of them will run concurrently??
2> “On what are you basing this comment? I’d be quite surprised to see variable substitution be faster than concatenation. Have you done some benchmark testing?”
Yes, I did the benchmark testing in 2003 on IS4.6 and string concatenation service took 1.6 times the time taken by the varible substitution.
Since then (i.e. in past 3 years) I have not used this concatenation service at all. However, I have not performed such bench marking on IS6.x version.
Now that you have expressed your doubt about this, I will perform this benchmark on IS6.x platform and publish my result.
I think the performance difference between calling pub.string:concat (either as a transformer or using a stand-alone invoke) versus using variable substitution on a MAP step would come from not having the overhead of invoking the service. The underlying concatenate operation should take the same amount of time.
The other advantage of variable substitution is avoiding multiple calls to the concat service when more than two strings are involved.
I do not believe that anything magic happens in a Map step containing multiple transformers to execute more than one transformer at a time.
I think the statements in the documentation are there to attempt to convey that transformer execution order inside a Map step is indeterminate. If you need sequential execution of transformers you must use separate Map steps.
I guess webMethods meant to say that transformers are independent of each other and do not execute in a specific order, hence developer should “assume” that the server concurrently executes the transformers at run time.
It looks like the AppendToStringList is what is causing me the slow processing. I have not had any experience of linked lists in java. How does it interface with the webMethods calling service.
Having so many entries/record, I’d be inclined to use pub.report services to create your output. I think the overall code will be concise and easily maintainable. You probably want to use pub.report:runStringTemplate. Performance-wise I think it’d be quite fast too.
its sheilawoodward :lol: :lol: Unknown guest!