Ok prior to 4.7 Integration server we were using 4.0. appendToRecordList did not at this time copy by reference a record item to a record list. It copied by value. I assumed the documenation was just wrong because it worked the way I wanted it to and the way I think it should work now.
While upgrading this by reference thing actually started to work the way the documentation claimed. So if I’m appending a record item to a record list. The entire list always ends up being the value of the record on the last iteration. Thus all entries in the record list are maintaining pointers back to what the current value of the record is. To me this functionality is garbage and serves little or no purpose.
I have seen several posts on how to deal with this, but all of them aren’t clear and still don’t come to one common conclusion.
My question is how can i a safely add an individual element to recordlist without screwing up the other items in the list? Also creating a unique record name for each name is not reasonable if you have a list that’s not predetermined in size which is the case most of the time in any real world situations.
I found a solution far different but seemingly far easier than the posts I’ve read related to this. Most of the posts I see showing a solution has replies stating that the solution still isn’t working properly.
The way i did it… was manually add properties to the fromItem parameter of the appendToRecordList service. I then manually set those variables using variable substitution and the list now maintains all of its lists correctly without pointers.
However I still wonder if there is a better way. Let me know if someone has an easier solution.
Since the record list isn’t really a list, it is an array, each time an item is added to it and new array has to be allocated and the entries of the original array have to be copied into the new object. This is fine when it is called a couple times, but has deadly performance when called often.
Since 4.6 there have been examples in WmSamples of how to work with lists in a scalable way. See the complexMapping large doc example. There are services that:
create a real list object.
add to the list.
turn the list into a IData.
You can then clone or not clone based on your applications needs and have very little overhead in adding to the list.
Fred,
Thanks for the input, however we are migrating from 4.01 we have hundreds of places where we are using appendToRecordList to add line items to recordlists. Performance is fine and not an issue.
The real issue of my post is now in 4.6 the appendToRecordList appends by reference/pointer only. On dynamic lists this makes it nearly impossible to add record items to record lists, because the list maintains a list of points back to one variable.
Example:
1. record Item = 1
2. call appendToRecordList
append record Item to Record List
3. List now has a a Item with a value of 1
4. change the value of the record Item to 2
5. call appendToRecordList
You would expect appendToRecordList to have two records. One with a value of 1 and the other a value of 2.
This is not the case. The list has to entries both with the value of 2. So because the appendToRecordList is maintaining a pointer back to the value of your record item and not the value itself. Your entire list is equal to the current value of the last know record item.
This has been subject of many conversations without very good solutions as far as I can tell.
Plus I understand the need for Idata on large doc's however when doing routine flows with small or medium docs... Not being able to use the gui to manipulate records and recordlists and their property is a real efficiency loss.
I don’t see any change to those services (just name and parameter renaming) that would have changed the behavior of those services back to when they were moved to the current source code control system in 11/2000.
I originally wrote those services and used the same basic assumption of the Flow language, which is to map using references whenever possible.
There must have been some other change to the Flow invocation path that caused this behavior change as a side effect.
Somewhere during 4.x most of the built-in services where converted from the 3.0-style signatures (Values in and Values returned) to the >= 3.5-style signatures (just a IData pipeline in). In doing so, the invoke path was optimized to really have only one instance of a IData object used as the pipeline (unless there is an exception and the pipeline has to be rolled back). This reduced memory use, but may have caused this problem.
I haven’t heard of this problem before. Please contact webM Support and register this as a compatibility bug.
It would not be hard to create an appendToRecordList that does a clone of the incoming IData before adding it to the destination array and WmPublic could be patched with this new service. This is not pretty, so maybe someone in support can come up with a better solution.
Fred,
Thanks for the background info. This is purely a problem of by reference vs by value. The bad thing is by reference in the way webMethods works really serves no purpose there are tons of ways to copy variables to other variables, but on the other hand to add a record item to a record list in a loop is a must have thing.
I’ve seen several recent lenghthy discussions on this topic in this forum, however the solutions are debated and impose allot of quirky workaround logic.
I hadn’t noticed any change in the behavior of the service either. AFAIK, a record list has always been an array. A Java array has always been a by-reference thing, never a by-value thing (except for primitive types). This is not a wM issue per se. It’s a Java issue.
From Tim’s example steps:
Example:
record Item = 1
call appendToRecordList append record Item to Record List
List now has a a Item with a value of 1
change the value of the record Item to 2
call appendToRecordList
I’ve never seen the case where step 4 didn’t change the object for both entries in the list. To get two distinct entries one must allocate another record. The seemingly funky thing is to get FLOW to allocate another record for you. This is normally done by dropping the record in the loop and then assigning as needed. So Tim’s example would become:
Example:
record Item = 1 (this creates a new record)
call appendToRecordList append record Item to Record List
List now has a a Item with a value of 1
3.1 Drop record Item
set the value of the record Item to 2 (this creates a NEW record)
call appendToRecordList
This might seem funky but it isn’t really–to do the same thing in Jave requires 2 calls to new.
Rob,
Thanks so Much for the example, your posts are extremely informative. Actually in 4.0.1 I found a SAP related BC document stating that appendtorecord list didn’t always maintain pointers, and that this was corrected going forward. Which we are running the SAP BC which is 99.9% Integration Server 4.0.1
This is why all my flows worked the way that I expected even without dropping and creating the records, until we upgraded. Meaning that i could reuse the same record to append to the array and the array would still maintain unique values ( Not Pointers ). Without Dropping
What you said makes sense about the java arrays, except in WebMethods dropping the variable. This doesn't make sense if you deallocate the record variable how would the array still maintain a pointer back to that variable which contained the value it needed?
The appendToRecord list really needs a By Value/By Reference Parameter associated with the service.
If your method works, it might be cleaner and faster than mine. My method was to overload the appendToRecordList Input Record Variable with my own Record structure, and set the values using variable substitution. Doing so makes it impossible for the array to maintain anything but the value in the array. However variable substitution might be a little slower than your method, we haven't tested, but I'm sure deallocating and reallocating the variable has some signifigant overhead as well.
All,
I have the same problem using appendToRecordList.
Can you please explain more detailed what you mean with Step3 “1 Drop record Item”.
I’m looping over an IDoc with several segments. Some of them have the same name –> need the same mapping.
After checking the Segmentname I map the segment into a record.
This record I want to append to a recordList. And here I have the same Problem.
info1 in record
appentToRecordList
3 info2 in recordList
appentToRecordList
recordList with two segments with info2
Input Parameter:
toList: rec_COPR
fromItem: COPR (with the informations)