Queue corruption

,
1

Hi all,
Sorry to bother but I really tried my best to look up past postings and visited the advantage website (read the manual) but there doesn’t seem to be a satisfatory explanation regarding the workings of the guaranteed queue.
I am from support and I have these unanswered questions:

  1. guaranteed documents - are they really guaranteed against sudden network, disk, OS, and app failure? If yes, great but how does it generally work? what happens to data still in the queues? are they resent the moment the system is up? What happens to data in transit (from source to broker) are they rolled back as I understand it?

  2. Now the real biggie: I suspect we still lose data if the queue itself is corrupted right? There are no tools at the moment to recover corrupted queues as I understand it. All we can do is overwrite the .qs, .log and .stor files in one go. so does that mean we still have to perform manual resubmission of documents?

  3. Finally, we are currently planning for our BCP (business continuity program) exercise where we have a redundant setup in place in another location away from the data center. This DRC server is kept in sync with production via ‘online replication’ - everything from database to Filesystem are replicated. Has anyone a similar setup? How does this replication do on a dynamic broker queue? will it get replicated as other files? Or I have no choice but to backup only the queue files but not the data (this brings us back to : what happens to data inside the queues if the queues are corrupted)

Have a nice day
Alvin

2

Here’s my shot at answering your questions. (I assume you are referring to Enterprise.)

  1. In Enterprise Server, queues exist on the broker. Yes, guaranteed docs are really guaranteed assuming that the right storage type is used. An adapter/broker client uses the APIs to send the document to the broker. If a success code is returned, the document is guaranteed to be safe (they use a two-phase commit to write the doc to disk before returning success).

I’m not privvy to how things really work internally, but from observations, the broker doesn’t “send” anything anywhere. All documents are pulled from the queues by the adapters/clients. Once the adapter/client acknowledges a doc, the broker will remove it from the queue. Thus, a doc should not be acknowledged until it has been successfully processed or stored, guarding against network/disk/os failures. Adapters/clients can use publish sequence numbers to address the “got-the-doc-but-the-ack-was-lost” type errors (in both directions–source to broker and broker to target).

  1. Yes, losing the queues is a Bad Thing. Resubmission of docs from the source systems is usually the easiest and safest thing to do, rather than trying to recover docs in transit.

  2. Can’t offer any insight on this one.