The Broker does not have enough storage to process the request

Hi all,
We have started getting this error 3 or 4 days ago:
com.wm.app.b2b.server.dispatcher.exceptions.TransportException: [ISS.0098.9014] BrokerException: Broker Failure (100-2015): The operation failed because the Broker does not have enough storage to process the request. Alert your administrator.

From what I’ve understood this means that the BrokerData.qs.stor files are full.

What I don’t really get is:

  • Full of what?
  • Why aren’t these files ever purged?
  • Since this error arrived suddenly it is probably caused by one particular service we are using, is there a way to browse through the files to have an idea of what is overflowing it?

We already added new 512MB files, but they were saturated after only one day!

We’re using Broker and IS 6.5.

I know this question has been asked several times but I didn’t find an answer that solved this.

Thanks in advance!
Jide

  • Full of what?

Documents waiting to be delivered.

  • Why aren’t these files ever purged?

Documents are removed once delivered to the subscriber(s). If the subscriber never connects and asks for them, and the documents are guaranteed, they will be held forever.

  • Since this error arrived suddenly it is probably caused by one particular service we are using, is there a way to browse through the files to have an idea of what is overflowing it?

It is likely old clients that registered subscriptions but then were never used or were “retired” after some time. You’ll want to use the Broker administration tools to review the registered clients to see which of them are stale and can be deleted (and associated storage freed).

Thanks for your reply,
We use myWebMethods to monitor the brokers, but on the concerned broker, there is no client that contains pending documents…
Should I look somewhere else?

By the way, there are 2 brokers on the broker server, I guess they share the same storage files?

“there are 2 brokers on the broker server, I guess they share the same storage files?”

Correct. You’ll need to check all Brokers of the server.

Ok, after a lot of investigation:
It seems that the broker catches every document that has its document type defined on it, even if there is no client associated to it, and EVEN if another broker from the same territory has also caught it.

We cannot delete the concerned document type on the IS because it itself publishes such documents. The question is: how come the documents end up in the datastore, even though they weren’t published on THAT broker, and even though ANOTHER broker on another broker server caught it?

We have put two IS in cluster, and they also publish and receive such document types, but they are on another IS that publishes documents

  • on another broker
  • on the same broker server
  • on another territory.

Thank you again, I’m definitely lost in the way these files are managed :S

Are you looking only at document type definitions on the Broker? If so, what you describe is normal. Brokers in a territory share all document type definitions–that’s one of the features of a territory. Document types are not the issue here.

It is documents (published documents with data in them) that would cause storage issues. Check all client queues on all Brokers to see if any of them are holding old documents and/or have been inactive for some time.

Ok we finally found the problem (and, more importantly, the solution)!

In the display options of the client list in the myWebMethods interface, you can set a maximum number of clients to display.
Since we have more clients than the default value of this parameter, the client list was incomplete. The clients that were overcrowded were, of course, in the “not displayed” list, so we couldn’t see the thousands of documents stored in it.

Is there ANY reason to hide a part of the client list? I only see a big potential source of mistake here…