Flow service performance issue

Hi All,

   My service selects records from DB (in this case 50K) which are PGP encrypted. I then loop over each record, decrypt and append each record to a file. Once all records are appended ...the file is PGP encrypted.

This design is slowing down my process.

Alternatives considered:

Select all records and publish each to broker. Trigger invokes concurrent threads which decrypt and append each record to a file (not sure if it leads to file locking problems).

Any suggestions would be appreciated.

Regards
Dave

How are you performing the PGP decryption? In the past this needed a call to a command shell. Doing so for each record will be slow. Perhaps you could devise a means of decrypting batches of records to avoid the overhead of the command shell invocation.

Not sure which database you are using. Is there a way to use its stored procedures to perform the PGP decryption?

Mark

Hi Mark,

Thanks for your reply. I am new to PGP, please correct me if i am wrong.

(pub.openpgp:decrypt) service is being used and i did notice that the service is referring to command shell. We are using SQL DB and yes we have an option of Stored procedure encryption/decryption, but we want to see if we can improve wM performance by changing our design. Would you please brief me with the process of decrypting batches of records. Also as i mentioned earlier i have to append all decrypted records to a file and encrypt the whole fine again.

Do we have problem with decrypt service as i am looping for each record (status,error and output variables) of decrypt service keep adding up in the pipeline for each iteration. Should i be explicitly dropping these variables ?

I would think that decrypting a batch of records using a stored procedure would be much faster than iterating over the same number of records making a call to the pub.openpgp:decrypt service for each.

You would need to read up on the PGP command line interface to see how best to decrypt multiple strings in a single call.

Mark

Hi Mark,

    Thank's for the advise.

Regards
Dave

Save file to hard drive is very slow operation. Please consider using local variables to store decrypted records and save it to file on the end of service.

You can also consider to use java services with has better performance than webMethods.

What do you mean about that? Can’t you check how it is working with dropping and without? Then you will know what is better :wink:

Additional thing that can help you: you can log times (e.g. to server logs) after each step and then check which step takes the longest time

Regards,
Lukasz Konkol