I have large amount of XML data: 300MB in total.
I have to divide them into small pieces. Which size do you suggest? Anybody has tested that before?
Thanks!
My experience is that the determining factor should be:
What granularity do you actually need ?
1 How big data-chunks do you present to the end-user(-program)
2 At which level do you need locking of documents
and then at the more technical side
A Does your application use DOM ? - because big DOM-trees take BIIIG amounts of memory and time in parsing
B Tamino validates before storing/updating using a DOM
So my rule of thumb is:
Make it as small as possible while still taking advantage of the hierarcic benefits of XML.
Finn
This is a very interesting and very important question!
Sometimes the right answer is that an XML document should be “all the information about one business object” (for example, a product or a customer).
But sometimes that gives you documents that are too big for comfort, or too small for comfort. Managing extremely large documents demands a lot of memory, and exchanging lots of small documents can lead to an increase in message traffic. A “comfortable” size from this point of view is, I would say, 5Kb - 50Kb.
If there is a unit of information that you will usually want to put on the user’s screen in one go, then that is probably your ideal document.
These are just some ideas. As often with database design, there is a need to balance what is convenient from the point of view of the application developer with what is most efficient in the database. The only firm Tamino rule is to avoid really large documents if you can.