inoload - Memory Allocation Failed

, ,
1

Hello.

I’m trying to load on my database 900,000 records of approximately 10K each one, with the command inoload. I have divided the 900,000 files in 1000 folders in 900 files each one, and 900 sequentially execute commands inoload, one for each folder (one behind the other)
I have not yet to have a full load of 900,000 records and I always get the error MEMORY ALLOCATION FAILED (in uncertain times and for different load tests)
As a result, also the error:

Committing loaded documents - this may take a while Can not contact Tamino Server - HTTP status code 400 received

The machine for this purpose has 16 GB of memory.
The parameters of my database are:

BUFFER POOL SIZE: 1,000
DYNAMIC POOL SIZE: 20,000
Maximun TRANSACTION DURATION: 36,000
NON_ACTIVITY TIMEOUT: 3,600
REPLICATION DELAY: 20,000
FLUSH GROUP SIZE: 2,000
FLUSH LIMIT: 700
WORK XML Threads: 200
XML Maximun SESSIONS: 2,000

I looked in the Tamino log (AABinosrv.09-01-10.txt) and the load fails just at:
10-JUN-2009 12:52:51 INODCI6565: Data loading completed, number of documents processed 1000, loaded 1000, rejected 0
10-JUN-2009 12:52:53 INOLDI3002: Preparing documents for load
[XTS] 12:56:04.990 390be20 Trace: (Module: xtsthr Line: 605) Create Thread = DrvReceive755 failed rc = 12
[XTS] 12:56:04.990 390be20 Trace: (Module: Threads Line: 742) ReadThread creation failure Rc returning =- 61 …

Please I need help if someone finds the problem.
Thank you very much in advance.

2

Hi Ivan,

For this type of questions I recommend that you open a support request for further investigation in the Tamino support group and if needed in the development group.

You should at least include information about the used version of Tamino, the type of inoload you are execution (with concurrent write, without concurrent write) and may be you can also provide some example files, server log files, and a server request log (if you do not know what this is, colleagues from support will help you with information on how to create it).

Best Regards,

Michael

3

Thanks M.Gesmann.

Where is the Tamino support group / development group?

I am using Tamino 8 and each call to inoload is:

…\bin\inoload.exe -function load -server taminoserver:3200 -collection corpus/TEI.2 -input D:"folder""subfolder"*.xml -verbose -log log.xml

There are 870 calls to inoload, one for each subfolder.

I attached some xml examples, error_logs and server log files

Thank you very much.
example.xml (658 Bytes)
log.xml (973 Bytes)
AABinosrv.14-28-37.txt (17.7 KB)

4

Sorry, I think I was not clear enough.
After having had a look into the attached files I do not think this can be fixed on the fly. Somebody from Research&Development in Software AG needs to look closer into this.
When you are using Tamino 8 you are having a contract with Software AG which normally also covers maintenance agreements. I propose to open an issue following the maintenance processes.

Hope this helps,

Michael

5

I notice that you appear to have increased the number of XML work threads ?!
I found a support request for tamino 441 where this increase was exactly the reason behind the errormsg you have met ?
So perhaps this is worth a go ?

Another hint from my earlier experience with the massloader:
The more loadsessions the more overhead, because a lot of information from the already loaded documents are (re-)written to disk and merged and (re-)loaded.

So dividing into several loadjobs may be a good idea for other reasons, but 1000 loadjobs sounds a bit of an overkill to me :wink:
Finn