Multi thread issue

,
1

hi Guys,

I have below requirement.

Everytime One flow servcie(file pollar) executes and writes the data one file and zip the file with file name “index.xml”. In this process the flow service will be multi theraded(Ex:since it is a file pollar if two files in a directory it will process 2 files at the same time). So both the threads will write try to zip with same name index.xml which is not accessible. Is there any way to sort out the processing of multi threads.

2

You can use parameters like File Polling Interval and Maximum number of Invocation threads in the file polling port configuration to control the execution of Multiple threads.

If this does not work I would suggest you to have a scheduler service that will pickup all the files in that folder and process then one at a time.

3

Hi Satish,
In any case, not just with 2 files, even if you have 10 files that comes an hour apart (let’s say), you might have the same problem while trying to zip the file with the same name (file already exists)… Either you should create zip files with datetime associated to the file name, which will avoid this overlapping by another thread, or you should have a separate service that takes all the files and zip into one with the nama ‘index.html’. I think if you have your problem statement clearly, you can arrive at how you want to handle it…

-Senthil

4

Hi ,

Thanks for your response. I should zip the file “index.xml” evrytime. There is no change in requirement. But if two threads are running (Thread1: processing 100 files and Thread2: processing 10 msgs), obviously thread2 will finish quickly before thread1 completes. but thread2 also try to zip the set of files with index.xml at the same thread1 invokes. This is the issue. How we can resolve this

5

Question:
do you expect 110 files to be zipped as one single zip file, or two zip files one with 100 and another with 10 files?

  1. If it should be two different files, provide two different file names
  2. If it should be one file, write java code and use synchronized block. Open the zip file using stream objects, add files to the zip, and close the zip stream. The other threads will wait till the time the zip stream is closed.

HTH
Senthil

6

Hi Senthil,

It should be two files (100 files in one index.xml-thread1 and another 10 files in index.xml-thread-2). Once we create the zipped file index.xml file we will be doing SFTP to other location from webMethods directory. So at any point of time each thread creates one file to wM dir and do SFTP and then next thread. This is the process. We can not club the two files here.

Regards,
Satish Devineni

7

Hi Sateesh,
If you want your service to zip all the files present at a moment and name it as “index.xml” then you do not need multithreading. As in multithreading is enabled and lets say it is set to 2 then first thread will try to zip all the files and the second thread will also try to do the same and which is not possible and does not make any sense to achieve that so in your case only one thread is sufficient.

In another scenario, lets say you have 100 files in the folder and you have set multithread to 100 then each thread will pick one file zip it and do the rest as per the code but in that case file name can not be the same.

So what i feel from your requirement is you want to zip all the files present at a moment and name should only be "index.xml"and send it via SFTP, in that case multithreading is not required or not possible.

8

Yes Vikas, I agree with you. Thank you for your reply.Thanks Guys for your suggetions. I can’t change the file name “index.xml” , i have kept Maximum Number of Invocation Threads to 1 in filepollar settings. So that i can avoid mutlithreading.

Thanks,
Satish Devineni