Broker server restart issue

, , ,
1

Dear All,

I’m getting below errors while restarting broker server on windows machine through services.

  1. Error: Broker “broker 1”: Could not create configuration storage for Event Type “canonical 1”, error 1302 occurred.

My comment: Where broker 1 is the name of the broker created earlier on broker server and canonical 1 is one of the canonical present on IS.
As per my understanding error related to this broker 1 shouldnt be coming because broker server containing this broker itself doesn’t exist anymore when I check from MWS.However please be noted that broker 1 was present earlier as part of this broker server.

  1. Error: Broker “broker 1”: Out of storage occurred while processing update from Remote Broker “broker 2”. [RESTARTING]

My comment: Again error message is related to broker 1 which shouldn’t have occurred. broker 1 and 2 are part of same territory.

  1. Unable to initialize basic authentication module. Verify the contents of basic authentication config file.

My comment: No changes were done in this file. It was working fine earlier.

Can you please share your thoughts.

Thanks,
Ashish

2

Hi Ashish,

please provide more details about your environment.

Which wM Version?
Which Broker Version?
Any fixes to the above?

Are both broker servers managed by the same broker monitor?

What is the Broker Memory/Storage Usage (can be checked via MWS Messaging UI)?

Regards,
Holger

3

Hi Holger,

Please find my comments below.

wM version: 9.8
Broker version: 9.6
Fixes: No

Each broker server have their own dedicated broker monitor.

Broker memory/storage usage in MWS I can check only for the broker server (broker 2) which is up. Broker server (broker 1) which is down is not showing anymore in MWS however we have already checked the DB space where broker environment is hosted and its normal.
Here is the memory usage for Broker server (broker 2).

Total: 14 GB
Used: 9 GB
Free: 5 GB

Thanks,
Ashish

4

Hi Ashish,

you should apply at least latest Fixes for the Broker (Broker Core, C API and Java API) to both installations.
Update MWS Messaging UI with latest Broker Portal Fix accordingly.

Is there anything in the broker server log which indicates why broker 1 is going down/restarting?

What do you mean by DB space?
Broker is not using any databases except its own storage files.

Regards,
Holger

5

Hi Holger,

Thanks for your quick response. Via update manager I could see these fixes are already applied. Below are the fixes already applied:

Broker command line 9.6.0 Fix (W64)
broker core 9.6.0 Fix 7 (W64)
Broker Java API 9.6.0 Fix4
Broker Java API shared bundles 9.6 Fix4
Broker SPM plugin 9.6.0

For MWS also, broker portal fix is already applied. In broker server log as well same error messages are coming which I have shared earlier. However I have noticed one thing today. When I brought down broker monitor and server of 2nd environment, I was able to successfully restart the broker server of first environment. As soon as I brought up the broker monitor of 2nd environment again, broker server of first environment went down.

Please be noted that both 1st and 2nd broker environments are running on seperate machine however they are connected via broker territory.

My bad. I was referring to MWS database.

Thanks,
Ashish

6

Hi Ashish,

latest Fixes are:
Broker Core Fix14
Broker ComnmandLine Fix2
Broker Java API Fix9
Broker C API Fix3
Broker Portal 7

When the two installations are on two different boxes then there should be no port conflict and therefore no reason why the first Broker server is going down when the second one is started.
Please provide the storage usage for both broker servers (just with one of them started if neccessary).
Eventually you will have to add additional storage files to the broker storages.
See Broker Admin Guide for instructions.

Regards,
Holger

1 Like
7

Hi Holger,

There is a good news. Broker server is up now after changing the client prefix for the broker connection alias on Integration server however broker is showing not connected (activeYes [Not Connected]) on integration servers.

I restarted the broker server but result is still the same.
Can you please suggest what can be done to fix the same.

Thanks,
Ashish

8

Hi Ashish,

did you restart the integrationServer after changing the preifx?

When changing the broker alias you must restart the IntegrationServer immediately as otherwise the changes are lost.
When the Broker does not reconnect after restart check the server log for related error messages.

Check the list of clients on the broker server afterwards and eventually remove the old queue clients (those with the old prefix).

Regards,
Holger

9

Hi Holger,

Sorry for the delay. Yes server was restarted. However I got to know from my colleague that they did some other change as well to bring up broker server.
They removed BrokerConfig.qs* files and then restarted the broker server which created these files again and server came up but this is not the correct solution since deleting these files will cause deletion of all existing broker, client group and document type details thus we reverted back the change.
Now we are in the same situation. Below is the current utilization of the broker server (broker2) which is up:

Resource name Total capacity Used Free
Virtual memory 15.6 GB 8.36 GB 7.24 GB

As these are storage issues, can it be because of the volume of updates broker 1 is sending at the time of broker server restart to broker 2 is more than the free space available.

Please share your thoughts.

Thanks,
Ashish

10

Hi Holger,

To provide you an update this issue is resolved now. This was due to BrokerConfig.qs* files got corrupted. We restored these files from the backup of previous date.

Cheers,
Ash