Appending to a single file using the FTP service

, ,
1

Dear Gurus,

We are having a requirement where I want the subscriber to FTP in such a way that it can append to the existing file. How can I achieve this, wat is the best method in wM.

Secondly, If a publisher has published N number of canonicals, each of this canonical is identified with different ID. Can a single subscriber read all these N number canonicals and create a single flat file?

Does a wM allows to schedule a subscriber instead of triggering mechanism.

Thanks
Khan

2

Can you elaborate on what you mean by “subscriber to FTP?” Do you mean an IS service invoked via FTP? If so, the service can look for and open an existing file and append to it.
[/color][/font]

Yes. You’ll need a way to determine when all published documents have been received. One approach is for the documents to have the documents track which number of how many they are. For example, if there are 10 documents, the first would indicate 1 of 10, the second 2 of 10 and so on. When all have received the file is complete for further processing.

A “last one” flag might work but that depends on document order and timing if multiple threads are possible. If everything is serialized throughout the process then this can work.

Lastly, a timing approach can work. Write all documents received to a file. At specific times of the day, or after a time has passed after the first document has been received, process the file assuming all that have been received is all that will be processed for now.

Yes, services can be scheduled instead of fired by a Broker/Local/JMS trigger.

3

if you publishing to broker… i m afraid your subscriber can’t be scheduled.

4

My post which you quoted pointed out that an integration could be scheduled instead of using pub/sub.

Here’s another way…

A subscriber can be scheduled to control when the documents are processed:

  1. Establish a trigger as usual. Suspend it.
  2. Create 2 services that will be scheduled.
  3. The first service enables the trigger when invoked by the scheduler.
  4. The trigger and trigger service will process all documents waiting on the Broker.
  5. The second service, scheduled 10, 15, 60, etc. minutes after the first, suspends the trigger.
5

A subscriber can be scheduled to control when the documents are processed:

  1. Establish a trigger as usual. Suspend it.
  2. Create 2 services that will be scheduled.
  3. The first service enables the trigger when invoked by the scheduler.
  4. The trigger and trigger service will process all documents waiting on the Broker.
  5. The second service, scheduled 10, 15, 60, etc. minutes after the first, suspends the trigger.

Request you to explain more on this, I could not understand on how we can achieve this.

Thanks
Khan

6

Hi Rob,

Just a quick scenario question: what will happen to the rest of the docs in the queue if by chance the second scheduler kicks in and disables the trigger when the actuall tirgger service is in middle of process? Will the document feed to the trigger service stop if this happens?

Is there any remote possibilty that the queued up docs get lost when the broker goes for a restart before the trigger was enabled?

7

Here is just one more way to do it:

Have 1 trigger service and 1 append service.

  1. Let the trigger service process the docs as they come in but write the data to temp files on local drive with a unique id(doc id/timestamp) in the file name.
  2. Have the append service scheduled either at less frequency or once a day at EOD which will pick up all the temp files append them into a single master file. If data continuity is important then sort on the unique id in the file name while picking up.

Hope this helps

8

Any threads in progress will complete. New threads will not be started.

Recall there are 2 parts of a trigger, document retrieval and document processing. The first will stop retrieval of docs from the broker. The second will stop the kicking off of processing of documents in the local queue. The common approach is to suspend retrieval and leave document processing enabled (allows the local queue to drain).

Yes.

Volatile documents will be lost on Broker restart. Guaranteed documents will not.

9

For which step would you like additional detail?