movingwindow on nested iterators

vijay_thota · July 8, 2015, 5:56pm

hi

when i am setting movingwindow flag on nested iterators the 2nd getNextIterator is throwing error saying it should be only true when iterating on nodes. :shock:

Ex: xml structure is like xmltag1->xmltag2. Passing tag1’s iterable node to tag2’s getNextIterator causes this error.

Does any one have faced this situation and resolved it? :roll:

thanks
vijay

Baharul · July 11, 2015, 2:10pm

Hi Vijay,
Are you talking about “pub.xml:getXMLNodeIterator” service.
If so,then I think you no need to pass movingWindow values.If you have more then one node then pass criteria.Then use “pub.xml:getNextXMLNode” in some repeated step to get the next node.

Please let us know if you face any difficulty in implementing the same or you can pass your exact xml structure.

Thanks
Baharul Islam

Mahesh_K_Sreenivas · July 11, 2015, 3:41pm

We use moving window input while handling large xml file.

Can you attach your xml, for your reference reference the link below

vijay_thota · July 11, 2015, 4:47pm

Hi Mahesh

The file is huge and cannnot attach, I can give a sample example as below

<rootnode>
	<nodeparent>
		<parentname>ParentOfNode_1_2</parentname>
		<childnode>
			<name>Node 1</name>
			<price>100</price>
		</childnode>
		<childnode>
			<name>Node 2</name>
			<price>23</price>
		</childnode>
	</nodeparent>
	<nodeparent>
		<parentname>ParentOfNode_3_4</parentname>
		<childnode>
			<name>Node 3</name>
			<price>100</price>
		</childnode>
		<childnode>
			<name>Node 4</name>
			<price>23</price>
		</childnode>
	</nodeparent>
	<childnode>
		<name>Node 5</name>
		<price>4353</price>
	</childnode>
	<childnode>
		<name>Node 6</name>
		<price>23.4</price>
	</childnode>
	<childnode>
		<name>Node 7</name>
		<price>12.4</price>
		<resources>
			<resource type="test">This will be in the output</resource>
			<resource type="test1">This won't be in the output</resource>
		</resources>
	</childnode>
</rootnode>

As can be seen above, the nodeparent tag is repeating and I want to iterate on this and subsequently get the iterator of childnode under it and process each childnode individually.

However for the service getXMLNodeIterator if I give criteria nodeparent and childnode then it loads all “childnode” under each “nodeparent” while iterating on nodeparent, which doesn’t make sense as I am telling to iterate on childnode also but it loads all childnode at once while iterating on nodeparent. So I have to explicitly get each iterator one by one by calling getXMLNodeIterator twice using parent child relationship.

During the first call of getXMLNodeIterator with criteria nodeparent and movingWindow to true, I get first node of this iterator and pass it to get the second iterator with criteria childnode and movingWindow set to true again then it throws an error saying that this flag can only be used once.

vijay_thota · July 11, 2015, 4:48pm

Hi Mahesh

The file is huge and cannnot attach, I can give a sample example as below

<rootnode>
	<nodeparent>
		<parentname>ParentOfNode_1_2</parentname>
		<childnode>
			<name>Node 1</name>
			<price>100</price>
		</childnode>
		<childnode>
			<name>Node 2</name>
			<price>23</price>
		</childnode>
	</nodeparent>
	<nodeparent>
		<parentname>ParentOfNode_3_4</parentname>
		<childnode>
			<name>Node 3</name>
			<price>100</price>
		</childnode>
		<childnode>
			<name>Node 4</name>
			<price>23</price>
		</childnode>
	</nodeparent>
	<childnode>
		<name>Node 5</name>
		<price>4353</price>
	</childnode>
	<childnode>
		<name>Node 6</name>
		<price>23.4</price>
	</childnode>
	<childnode>
		<name>Node 7</name>
		<price>12.4</price>
		<resources>
			<resource type="test">This will be in the output</resource>
			<resource type="test1">This won't be in the output</resource>
		</resources>
	</childnode>
</rootnode>

As can be seen above, the nodeparent tag is repeating and I want to iterate on this and subsequently get the iterator of childnode under it and process each childnode individually.

However for the service getXMLNodeIterator if I give criteria nodeparent and childnode then it loads all “childnode” under each “nodeparent” while iterating on nodeparent, which doesn’t make sense as I am telling to iterate on childnode also but it loads all childnode at once while iterating on nodeparent. So I have to explicitly get each iterator one by one by calling getXMLNodeIterator twice using parent child relationship.

During the first call of getXMLNodeIterator with criteria nodeparent and movingWindow to true, I get first node of this iterator and pass it to get the second iterator with criteria childnode and movingWindow set to true again then it throws an error saying that this flag can only be used once.

vijay_thota · July 11, 2015, 4:48pm

Hi Mahesh

The file is huge and cannnot attach, I can give a sample example as below

<rootnode>
	<nodeparent>
		<parentname>ParentOfNode_1_2</parentname>
		<childnode>
			<name>Node 1</name>
			<price>100</price>
		</childnode>
		<childnode>
			<name>Node 2</name>
			<price>23</price>
		</childnode>
	</nodeparent>
	<nodeparent>
		<parentname>ParentOfNode_3_4</parentname>
		<childnode>
			<name>Node 3</name>
			<price>100</price>
		</childnode>
		<childnode>
			<name>Node 4</name>
			<price>23</price>
		</childnode>
	</nodeparent>
	<childnode>
		<name>Node 5</name>
		<price>4353</price>
	</childnode>
	<childnode>
		<name>Node 6</name>
		<price>23.4</price>
	</childnode>
	<childnode>
		<name>Node 7</name>
		<price>12.4</price>
		<resources>
			<resource type="test">This will be in the output</resource>
			<resource type="test1">This won't be in the output</resource>
		</resources>
	</childnode>
</rootnode>

As can be seen above, the nodeparent tag is repeating and I want to iterate on this and subsequently get the iterator of childnode under it and process each childnode individually.

However for the service getXMLNodeIterator if I give criteria nodeparent and childnode then it loads all “childnode” under each “nodeparent” while iterating on nodeparent, which doesn’t make sense as I am telling to iterate on childnode also but it loads all childnode at once while iterating on nodeparent. So I have to explicitly get each iterator one by one by calling getXMLNodeIterator twice using parent child relationship.

During the first call of getXMLNodeIterator with criteria nodeparent and movingWindow to true, I get first node of this iterator and pass it to get the second iterator with criteria childnode and movingWindow set to true again then it throws an error saying that this flag can only be used once.

Mahesh_K_Sreenivas · July 11, 2015, 6:20pm

I see your sample xml is not correct. Just have a close look at line 24 it is starting with - Is it normal

Should it be like this: inside a root tag. If yes I can share you the code. Contact me on my email.

    <nodeparent>
	<parentname>ParentOfNode_1_2</parentname>
	<childnode>
		<name>Node 1</name>
		<price>100</price>
	</childnode>
	<childnode>
		<name>Node 2</name>
		<price>23</price>
	</childnode>
</nodeparent>
<nodeparent>
	<parentname>ParentOfNode_3_4</parentname>
	<childnode>
		<name>Node 3</name>
		<price>100</price>
	</childnode>
	<childnode>
		<name>Node 4</name>
		<price>23</price>
	</childnode>
</nodeparent>

Mahesh_K_Sreenivas · July 11, 2015, 6:21pm

I see your sample xml is not correct. Just have a close look at line 24 it is starting with - Is it normal

Should it be like this: inside a root tag. If yes I can share you the code. Contact me on my email.

    <nodeparent>
	<parentname>ParentOfNode_1_2</parentname>
	<childnode>
		<name>Node 1</name>
		<price>100</price>
	</childnode>
	<childnode>
		<name>Node 2</name>
		<price>23</price>
	</childnode>
</nodeparent>
<nodeparent>
	<parentname>ParentOfNode_3_4</parentname>
	<childnode>
		<name>Node 3</name>
		<price>100</price>
	</childnode>
	<childnode>
		<name>Node 4</name>
		<price>23</price>
	</childnode>
</nodeparent>

vijay_thota · July 11, 2015, 6:49pm

It’s just a sample file to explain abt the issue . Don’t worry too much about the semantics of the file. If you really could solve the issue email me the code to kalyan.cdev@gmail.com I will take a look at it.

Mahesh_K_Sreenivas · July 11, 2015, 7:03pm

Yes, I know it might be a typo or copy paste error I want you to confirm on the same and I am not worrying about it

Check the code sample I have attached it here.
X.zip (6.77 KB)

Mahesh_K_Sreenivas · July 11, 2015, 7:05pm

The code can look simpler if you use moving window=false. You can try it out. Ask me if any questions.

Mahesh_K_Sreenivas · July 11, 2015, 7:05pm

The code can look simpler if you use moving window=false. You can try it out. Ask me if any questions.

vijay_thota · July 11, 2015, 7:26pm

Thanks for sharing the code…the 3 steps just after the first branch will consume memory and will not work in my case. I have attached the scrshot for ur reference.

Mahesh_K_Sreenivas · July 11, 2015, 7:42pm

Yes, let me see if I find any alternative solution. What is your xml file size?

vijay_thota · July 11, 2015, 7:47pm

consider it as large xml ~100mb

vijay_thota · July 11, 2015, 7:47pm

consider it as large xml ~100mb

Mahesh_K_Sreenivas · July 11, 2015, 7:56pm

Good deal and interesting. I will keep you posted if I find any alternate solution. Let me know if you find something.

For handling large xml file I assume you should even take in to effect of your JVM MAX and MIN memory assigned to IS? Is it?

Mahesh_K_Sreenivas · July 11, 2015, 7:57pm

Good deal and interesting. I will keep you posted if I find any alternate solution. Let me know if you find something.

For handling large xml file I assume you should even take in to effect of your JVM MAX and MIN memory assigned to IS? Is it?

vijay_thota · July 11, 2015, 8:07pm

yep ofcourse its all about memory when handling large files…i will let u know if i find anything that works…appreciate ur efforts on this

thanks

vijay_thota · July 11, 2015, 8:08pm

yep ofcourse its all about memory when handling large files…i will let u know if i find anything that works…appreciate ur efforts on this

thanks