Understnading event replicator

Thierry_Casier · May 7, 2008, 11:49am

100 recs/sec is also the number we achieve when starting an initial state.
We have found it very much depends on the hardware used.

We did our first initial state with the target adapter on a Sun Unix machine with sparc III type processors and we could replicate 20 recs/sec.

A second test on a plain PC with a 2.8GHz processor in our LAN (10Mb)environment resulted in 80 recs/sec.

A third test on a recent Linux dual core server with 2GHz processors in the backbone network (100Mb) resulted in 100 recs/sec.

We use MQSeries for transport but I don’t think this makes any difference.

We replicate agains Oracle and my Oracle admin said that there could be a performance gain on the target adapter if it would use ‘bindings’ but untill now I did not have the time to investigate this further.

Best regards,

Thierry

system · May 7, 2008, 12:17pm

I don’t think increasing messageprefetchcount would help. I think the next buffer will always be ready for ART.

I’ve no experience with VSAM pstore.

That the windows server does not utilize 100% I don’t think you can assume that the problem is then in Broker. What is the cpu usage for Broker? Please note that sql has to do many IOs which means he is very IO bound. So I don’t think you ever will be able to see this server go into 100% cpu.

Wayne_Campos · May 8, 2008, 1:17am

I needed time to run a test.

Source:
z/OS
ADABAS 7.4.4
Event Replicator 3.1.1
EntireX 7.3.3

Target:
Windows 2003 Server Standard Edition (VM)
SQL Server 2005
Tomcat 5.5
Event Replicator target Adapter 2.4.1

Initial State of 1,000,000 records.
Elapsed time: 8624672 ms
Rate: 116 records per second
No MU or PE.

    General Values                                TLOG Values

Subtasks … ___3 Maximum RPL Usage … _40 %
Max Output Size … _____32768 Restart RPL Usage … 30 %
EntireX Broker Stub Name … BKIMBTSO Input Queue Level … 0
Input Request Msg Limit … ________10 No Match Level … 0
Input Request Msg Interval ________60 Queue Completion Level … 0
Verify Mode … N Completion Level … 0
Format Buffer Validation … W Request Received Level … 0
Record PLOG information … Y Request Rejected Level … 0
Max Record Size … _____32760 Request Error Level … 3
Max Variable Record Size … _____32759 Status Request Level … 3
Retry Interval… ________30 I-State Start Request Level … 3
Retry Count… ________10 I-State Completion Level … 3
Retransmit Request Level … 3

Dest Name … DHVDEV87 EntireX Broker ID… EXB180:18020:TCP________________
Architecture … 2 Broker Service … ART-VDEV87_____________________
Threshold … 5 Service Name … TECH___________________
Destination Active … N Service Class … REPLICATE_______________________
Allow Logging … Y Retry Interval … GLOBAL____
Replicate Util Chgs … N Retry Count … GLOBAL____
Max Output Size … 32768
TLOG Parms
------------------------ Destination Class … SAGTARG
Assign Level … 0 Destination Class Parameter Data …
Completion Level … 0 NOSPRE______________________________
SLOG Write Level … 0 ________________________________________
SLOG Read Level … 0 ________________________________________

[07 May 2008 11:10:20] 9329 [Art-Engine] INFO ada.forcedInfoLog(Define.java:257)

Event Replicator Target Adapter Version 2.4.1.17 Starting
(Hot Fix #17), Build Number 243 at April 2 2008 1650

[07 May 2008 11:10:20] 9423 [Art-Engine] INFO ada.forcedInfoLog(Define.java:257)
Software AG License Check Successful!
[07 May 2008 11:10:20] 9454 [Art-Engine] INFO ada.forcedInfoLog(Define.java:257)

Source Type…: BROKER
Host:Port…: EXB180:18020
Receive Service…: REPLICATE/TECH/ART-VDEV87
Send Service…: INPUTQUEUE/TECH/ART-VDEV87
Trace Level…: 0
Verbose Level…: 0
Max. Receive Len…: 32768
Capture Mode…: False
Security Mode…: False
Convert to Underscore…: True
Convert to Lowercase…: False
Use Occurrence for MU Table…: True
ART 2.3 Table Layout…: False
Wait Time…: 1200S

jdbc database type…: Microsoft SQL Server
database user name…: sa
database host name…: reisdfw01vdev87
database port number…: 1433
database database name…: reptor
database retry count…: 5
database retry interval…: 30
database Max. batch stmts…: 0
database Error Continue…: False
database delete before insert.: False
database update to insert…: False

[07 May 2008 11:10:21] 10517 [Art-Engine] ERROR PersistentWebdavImpl.(PersistentWebdavImpl.java:76)
INFO: Webdav cache create size: 20
[07 May 2008 11:10:21] 10611 [Art-Engine] INFO ada.forcedInfoLog(Define.java:257)
INFO: Set verbose level: 0
[07 May 2008 11:10:21] 10673 [Art-Engine] INFO ada.forcedInfoLog(Define.java:257)
The Event Replicator Target Adapter is started and waiting replication data

[07 May 2008 11:10:54] 43928 [http-8085-Processor4] INFO ada.forcedInfoLog(Define.java:257)
INFO: InitialState Process Started…
[07 May 2008 13:34:39] 8668616 [http-8085-Processor4] INFO ada.forcedInfoLog(Define.java:257)
INFO: InitialState Process Completed
[07 May 2008 13:34:39] 8668632 [http-8085-Processor4] INFO ada.forcedInfoLog(Define.java:257)
INFO: InitialState Estimated Elapsed Time: 8624672 ms
[07 May 2008 14:13:01] 10970444 [main] INFO ada.forcedInfoLog(Define.java:257)
INFO: Undeploy Server Requested. Start Shutdown Process.

system · May 8, 2008, 12:51pm

We did also increase the blocksize for pstore to 6518 (BLKSIZE=6518 in DEFAULTS=BROKER section). Default is 2000. Adabas device type is 8390.

Wayne_Campos · May 8, 2008, 5:54pm

We are using BLKSIZE=4096.

However, I think that this parameter is from the DEFAULTS=ADABAS section not the DEFAULTS=BROKER section of the attribute file.

system · May 8, 2008, 7:12pm

You’re right (DEFAULTS=ADABAS section). Sorry.

vtisuela · May 8, 2008, 11:06pm

Thank you very much for sharing your configuration and test results. They really help a lot in validating our set up. Please continue to share your thoughts on performance improvements.

Does a bigger BLKSIZE for Pstore help in the preformance? Can you share your values in the Broker attribute file?

Thanks a lot.

vtisuela · May 9, 2008, 5:38am

Wayne, Mogens, Thierry,

Which transport do you use for Broker, TCP/IP or SVC? Would TCP/IP be faster than SVC?

Thanks.

system · May 9, 2008, 12:15pm

Reptor <-svc-> Broker <-tcp/ip-> ART. I don’t think anything is faster than svc communication between Reptor anf Broker. BTW you run Reptor and Broker on the same LPAR?

DEFAULTS = ADABAS
BLKSIZE = 6518 * 8390=6518 8393=27644 (max 20000)

BLKSIZE = 13800 * 8390=6518 8393=27644 (max 20000)
DBID = 92
FNR = 218
FORCE-COLD = Y
SVC = 247
------------- Adabas Event Replicator --------------------------

DEFAULTS = SERVICE
CONV-LIMIT = UNLIM
CONV-NONACT = 4M
LONG-BUFFER-LIMIT = UNLIM
NOTIFY-EOC = YES
SERVER-NONACT = 5M
SHORT-BUFFER-LIMIT = UNLIM
TRANSLATION = NO
MAX-UOWS = 25000
DEFERRED = YES
STORE = BROKER
CLASS=ARF, SERVER=ARFQ, SERVICE=OUT
CLASS=ARF, SERVER=ARFQ, SERVICE=IN, MAX-UOWS = 10
CLASS=ARF, SERVER=ARFQ2, SERVICE=OUT
CLASS=ARF, SERVER=ARFQ2, SERVICE=IN, MAX-UOWS = 10
*

Thierry_Casier · May 13, 2008, 12:10pm

We use MQSeries for transport, so no SVC used in this setup.

Best regards,

Thierry Casier

vtisuela · May 14, 2008, 4:28am

Thanks for the replies.

Our Broker and Reptor run on the same LPAR.

We changed our pstore blksize from 2000 to 4096 but the performance did not improve, although I noticed that fewer records were written and deleted from the pstore (about 50%).

We monotored the activities in the pstore, Target Adapter, and SQL Server. The Target Adapter processing takes the longest, even after we gave it more memory and another CPU. It runs on a virtual machine. The CPU usage of the box does not even reach 100%, according to the performance monitor.

Any idea how to tune the Target Adapter?

Thank you.