loading large number of XML docs from a single file using ja

I have a file which is 850MB in size and contains a single document whose child elements have to be loaded into Tamino. (I use the Java loader without the -d option i.e. the child elements of the main document are loaded)

The problem is that if any of the documents is not valid, the loading process is terminated. Is there anyway I can force the loading to just ignore the invalid document and continue?

Since the file is so huge, it is not feasible to check each document in it for errors. So I just want to load all the valid documents using a single java loader process.

I need some help urgently. Any advice would be appreciated.

hi amvil,

i’d say the java loader will not work for you if not all instances within your document are ok.
you can specify
-m 1
which will make sure that at least all instances prior to any error are written and commited, but the first erroneous document will abort the java loader.

IMHO you will either need to check the documents with a different tool and insert only the valid documents. writing a java program that does that should not be too difficult.

OR unpack/repack your documents so that you can use the inoxmld mass loader - it has an option that allows it to simply report the rejects.


[presumably needles to say, but just for the sake of completeness:
have you considered making sure that the document is valid in all its branches?]

hope to have helped,

andreas f.