I have a huge pda 76 meg. The size refers to the nature of the data being returned, namely 1 dimensional arrays of size (1:999), and 2 dimensional arrays (1:99,1:10) etc. Although in reality the max size the pda would be populated to is approx 32k, in theory, a record may hold in its entirety 76 megs (when including all child records etc).
What happens, is that an external call is made from a client to the web services middleware and the middleware does a rpc to my natural 4 code via entire x. i populate the pda it goes back, entirex converts it to xml and sends it back as a web service. Problem is a query takes 60 seconds and this is with the pda being no where near full!
So how could i redesign this to overcome the huge size problem?
I was thinking of returning chunks/segments of the PDA, as in 0-10, 11-22, 22-33 etc to multiple calls from the middlware enitre x and then the middleware could amalgamate the segments into one. Would this make sense and speed things up?
Or if i used dynamic arrays? This would save me declaring huge arrays up front that may never get fully populated. But this still wont get over the problem of me reaching the full 76 meg in size and sending it across in one go.
76mb is going to take time no matter what you tune. Once EntireX adds the XML tags, it will be even larger.
It might help to estimate the “typical” patterns - eg. 75% of the time you need 32k, 20% up to 1mb, 5% up to 76mb, or what ever your application patterns are likely to do.
If there is a high likelihood of the small result, then use the small size first. You can then do additional calls for the remaining data. Dynamic arrays should help in most of the larger size calls.
Do you need to use XML? For all its benefits, there is overhead to using XML. If you can use one of the direct wrappers (Java, .Net), you will save on network traffic and XML parsing time.
Yes we need to use xml as we will be using web services and moving away from rmi via ejbs.
When you say do additional calls i assume you dont mean the end ‘client’ should do multiple calls but that we would from our middleware do multiple calls to my natural code to get the data back from the pda in segments/chunks? Correct?
How would you suggest this would be implemented? I guess i would receive range parameters into the pda and i would use to return the array range segment or something?
In the early days, before we had access to large PDAs and variables, these types of workarounds were common.
You can use conversations and have the server maintain “state” - the Natural code would keep track of where it was by using CONTEXT variables. I don’t usually recommend this approach, although it can work.
More often the preferred approach is to use something to keep track of where the call is - starting value starts at a low value, next starting value is returned by the RPC client (your middleware in this case), which might be the last value in the previous block or a separate “next” value previously sent by the server.
If you are using a non-unique descriptor/superdescriptor in the logical read statement, check out the STARTING WITH ISN clause. Send the last ISN read to the client, who returns it (along with the corresponding last value received that goes with that ISN). The READ/STARTING WITH ISN always returns the first record AFTER the starting with ISN.
On the other suggestion: Depending on where your web services need to be positioned, if your middleware is (for example) Java, you can use the Java Wrapper to make the direct RPC calls, then the middleware can assemble the XML document to present to the end client.