Queue Storage file problem

Enterprise Server 5.0.1
Logger Adapter 4.1.1 (SQL Server 2000)

I had got this error,

The broker import file “namespace.adl” could not get saved to the broker:
Broker Failure (100-2015): The operation failed because the broker does not have enough storage to process the request. Alert your administrator. (BrokerAdminClient.setEventTypesAndClientGroups-8831).

So i increased storage space using the below command,

server_config storage C:\webmethods5\data\awbrokers\default -qs_storage_file
C:\webmethods5\data\awbrokers\default\MoreStorage.qs.stor 1024M 128M

After creating the extra storage file, now the server performace is hardly hit. Since the transaction are more and also I am doing logging, logging speed is vvvvvv slow.The logging queue length will always be in MBs.

Currently i have these files in my default directory wrt logging & storage.

  1. Broker.qs.log (32 MB)
  2. Broker.qs.stor1(512 MB) usage space is 30 MB
  3. MoreStorage.qs.stor (1 GB) usage space is 50 MB

Is there anyway to delete the MoreStorage file / any other way to handle this problem.

Thanks for ur suggestions…

Hi,

Did you ever get any responses to this request? We’ve got a performance problem and we’re in a similar configuration with two QSs. Did you return to a single QS and if so, did the problem disappear?

Our awbroker is running at near 50 % CPU time when doiing a batch operation via IOFileAdapter and documents are read extremely slowly from the queue by another adapter on a remote server. What we observe when we do a truss on awbroker is that there are close to 90 % of calls to time(). This makes us think that wM is tracing everyhing, but maybe there are other possible explanations for this high number of calls to time()?

Are there any known patches to the Broker? We’re on 5.0.1 080802.

Best regards,

Frank Olsen

Frank,

There are service packs available, I believe that 501SP11 is the latest (the 080802 date looks like SP7 if I am not mistaken). You should obtain the latest Fix / Service Pack from webMethods Technical Support.

Since broker performance is typically bound by the ability of the adapters to process documents, the broker is almost never the bottleneck. It is more likely to be related to the network, the ability of the target resource to accept incoming documents / transactions, or the speed of processing that is occurring in the target adapter. The use of multiple storage files should not have a great impact on overall performance.

These are just some general thoughts, obviously your specifics could be wildly different.

Dave

Hi David,

Thanks a lot for your response. We’ll check out the SP.

I think we can actually exclude the double storage file as a possible cause for our problems: we are in fact in the lucky situation of having a broker installed on two different servers and with one of the two everything works fine! We first thought that there was a difference in the number of QSs on the two servers, but we finally found another one in a different filesystem.

So, what is different between the two servers? In principle the configuration is identical: same versions, same ADLs, same parameters. Concerning the QS there are two differences: (1) on the slow server the .log file is 10 times the size of the same log on the fast server (approx. 1 GB versus 100 MB); (2) the slow server as two QS of 1 GB and 500 MB whereas the fast one has two og 1 GB.

Of course, there is no way to infer that the slow server logs more just because it has a bigger log file :wink: But, one possible explanation for all the time() system calls when doing the truss on awbroker – it was doing around 80k system calls/second where the fast server doing the same thing did around 3k sc/s – would be that the broker is tracing it’s internal activity and writing time stamps for each log line. Then again, there may be other explanations for calling time() related to the internal working of the broker. Can’t tell.

What is clear is that the the “source” adapter (an IO adapter reading an XML file) and the QS/broker is OK. The QS fills up at the same rate on the two servers. The difference is between the QS/broker and the “target” adapter: in fact the network is different in the two cases. For the fast broker server the “target” is in the same rack separated by a simple switch. In the slow case the broker and “target” adapter are interconnected via a VLAN between two locations separated by approx. 1 km. The again the connection between the two locations is fiber at 1 GB. Any differences in throughput and latency should be minimal.

When running this batch on the two servers the observed difference on the system level is that the awbroker consumes a lot more CPU (in user mode): 45 % CPU versus 3 %. That seems normal given the 80k system calls/second that the slow server consumes.

I wish I could see a logical explanation for calling time() something like 70k times/second. In particular, how can it be related to the difference in network topology…?

Just to be clear, in the two cases the same “target” adapter on the same server is used.

Hopefully we’ll be able to do some controlled experiments in the next few days.

Best regards,

Frank Olsen