Adabas Capacity for concurrent unloads or decompresses

, , , ,
1

My client did an unload of an ADABAS system containing approximately 1,500,000,000 records. This took approximately 7 hours. using ADARUN with the ADAULD command. It then took an additional 20 hours to decompress this data using ADARUN with ADACMP DECOMPRESS

They are looking for ways to perform these tasks in less time - Could ADABAS sustain multiple concurrent unloads and decompresses?

I’ve also asked them to try using the INFILE command with DECOMPRESS rather than doing separate UNLOADs and DECOMPRESSes

2

Have you tried setting SORTSEQ=ISN, STARTISN and NUMREC options in parallel unloads? You might have to experiment with different number of parallel unloads (and the corresponding parallel decompresses) to see what works best in your environment (largely a function of the number of available processors and existing load).

Doing parallel unloads will be resource intensive, but should be able to reduce the total wall time needed to unload. Parallel decompresses can then be run against each unload file.

The exact STARTISN / NUMREC values to use will depend on the file being unloaded and how many parallel unloads you want to run. The values will need to be selected carefully to ensure neither dropping records nor including duplicates.

Let us know how it goes!

3

Thank you Douglas … I will have to schedule some weekend time for this. We will be
sure to let you know how it goes

4

Good afternoon,

I asked the client to try running 4 parallel unloads as follows:

ADAULD FILE=1,SORTSEQ=ISN,STARTISN=1,NUMREC=39348316
ADAULD FILE=1,SORTSEQ=ISN,STARTISN=39348317,NUMREC=39348316
ADAULD FILE=1,SORTSEQ=ISN,STARTISN=78696633,NUMREC=39348316
ADAULD FILE=1,SORTSEQ=ISN,STARTISN=118044949,NUMREC=39348316

and the first job worked okay while the remaining 3 received this error …

ERROR-072, Unexpected nucleus response 48
02- File currently in conflicting use

What am I missing? We do specify ADARUN MODE=MULTI

5

eww. Apparently ADAULD opens as UTYPE=EXU and only one user can have a file open for exclusive use (sorta what EXU says…). Not sure why EXU is needed for unload operation, but parallel unloads don’t appear to be a valid option presently. My apologies.

6

Well, the documentation reads

  • If EXU (the default) is specified, the file cannot be updated, but other users can read the file
  • If EXF is specified, only ADAULD can use the file; no other users can read or write the file.

According to this EXU should be ok, but … maybe “user” just means “user” as in “user” but not as in “utility” .

Maybe try using different ETID parameters for these parallel unloads (unless you need a specific one for a multiclient file) ?

7

Hey thanks Wolfgang … I will set up a quick test and see if this works. These parallel unloads do need to be done without any other “user” accessing the files, so hopefully this works

8

For parallelism - have you tried creating an offline save with ADASAV and then passing the backup dataset to multiple ADAULD SAVETAPE with SELCRIT and SELVAL?

And have you tried ADACMP DECOMPRESS with the INFILE option? That may save you enough time to not need parallelism…

9

Hi Bjarne,

The issue with ADAUNL suggestion above is the backup dataset is on about 15 tapes and can only be mounted to one job at a time.

The most serious issue is with the DECOMPRESS performance. We haven’t been able to find a way to improve performance except to run 4 decompresses concurrently. If we run these 4 jobs serially, it take 16 hours. If we run them concurrently it take about 7 hours.