I’m trying to create a Universal Messaging client in Java. The client should create an asynchronous durable subscriber that would be notified when events arrive on the channel.
The subscriber should be notified not on all events but only on those that contain a certain value in one of the fields.
My code for creating the subscriber is as follows:
nSharedDurableAttributes attr = (nSharedDurableAttributes)
nDurableAttributes.create(nDurableAttributes.nDurableType.SharedQueued, namedObjectName);
attr.setPersistent(true);
attr.setClustered(false);
attr.setStartEID(0);
attr.setFilter("stringField = 'aaa'"); // Only events with this value in the field are of interest
durable = durableManager.add(attr);
channel.addSubscriber(new EventListener(), durable); // EventListener just prints a line on the console when notified
However, the listener gets notified even when an event is pubished with the field stringField having some other value than aaa.
In the EnterpriseManager, I see the durable, and in its properties the selector stringField = 'aaa'. But still I get notifications I would not expect. Sure, I can check the events on the client and ignore the ones I’m not interested in, but I’d expect the filtering to happen on the server.
The filter looks correct as does the rest of the logic.
What type of events are these, UM native, JMS or protobuf events?
Could you do a test on the events you are receiving in the java logic and see if the key is present and what values it is equal to, perhaps something is not correct in the event structure?
thank you for the quick reply. The messages are of protobuf type. They can be correctly consumed, i.e. I see the correct values on the client.
However, I decided to switch the implementaion so that I
Use durable of type Named (instead of SharedQueued)
Consume messages actively via an iterator (instead of asynchronously per callback).
With this, I hope to gain more control on the consumer side wrt. to when and in which chunks the messages are consumed. Also, there will be no need in a live connection and the communication “UM → consumer”, i.e. the calls will be from consumer to UM. Are these valid thoughts?
I.e. on the consumer side, I’d open a connection to UM, read events, and then close the connection. This would be repeated as a scheduled job.
The only important thing for me is that the messages are saved and held on UM if a sender puts them there but there is no consumer at the moment. Will the above parameters (esp. Named type) fulfill these requirements?
Still, I’d be interested in how to set up the durable subscription so that only certain messages (with a certain value in a field) get there and all the other messages get filtered on the server. So that no unneeded data are sent over to the client. Sure, the filtering can be made on the client side, but I’d prefer to do it on the server.
An iterator will work exactly as you describe as will a Named Durable (Exclusive Durable), the interesting aspect of a Named Durable is only a single consumer can read from it thus not named Shared however still not serial either.
It does not have out of order acknowledgment being its largest down side so you must ensure when acking that all messages prior are processed. I would highly recommend you use the Shared Durable not the Shared Queue as we have removed the Shared Queue due to complexity and instability.
The filtering should work based on what you have provided however I believe there maybe some issue with the protobuf or you are sending events with no data as UM has a default filter rule which stipulates if we cannot filter it we will just forward it(claiming the filter matches), this avoids events being stuck in UM and is the preferred behavior when using Integration Server as it makes the message visible.
Something to check, lower the server log and run a test with publishing to see if there is any issues being reported with the protobuf that is registered with UM server.
thus not named Shared
. . .
I would highly recommend you use the Shared Durable
I’m still on v. 9.12 and see only Named and SharedQueued as available type that are worth to consider. Do you mean I should use Named (this is my intention)?
Hey,
How did you create the channel? Is it an IS document type? For filtering to work fine on UM side, UM needs to have the correct protobuf definitions too, so it may be worth double checking that, although I’d expect the filter matcher to discard the message if it can’t decode it. Then again, the definitions and the filter need to be available in both the main channel and the shared queue, so there’s room for something to go wrong. SharedQueues were removed for a good reason as Josh mentioned.
Did you manage to get the filtering working when you switched to an iterator?
thank you for chiming in. Yes, the channel was created by creating a publishable document on IS (with protobuf option) and synchronizing it with UM. If I use an iterator with a selector, I indeed get only the events that match the filter.
But then, some events would stay forever in the durable subscriber which I do not want.
Until I get the filtering working as expected (by that I mean that it’s applied when the message is copied to the durable subscriber) I’ll stick to a subscriber without a filter and will do filtering on the client side. There will be more network traffic than necessary but at least the server will be kept clean.
It’s what UM is designed for: message exchange. In this case, between an IS and a custom client (not an IS). Somehow I wanted to try it with UM, although there are of course other options (like other java messaging systems or Kafka). But IS and UM go well together hence the idea.
The down side of UM is that its docs are … err… not very detailed, and there are also not much docs on the internet. Even the JavaDoc is in a somewhat weird format not very comfortable for use. BTW: Does someone know how the docs was produced? To my knowledge, the standard javadoc tool produces something different.
I guess my query was a bit ambiguous. It wasn’t about why you’re using UM. But why you’re creating custom Java code to create a UM client.
An approach you might consider is use IS as the UM client/subscriber–abstracts all the complexity of managing durable subscriber, provides another level of filtering, etc. Then perhaps use JSON over HTTP to interact with the custom client. Basically, isolate the UM stuff in IS, the client just gets data via its API. Of course that means you’d need to create an API, if this app is totally something new.
As with most things, it’s a trade-off. Do you want the app to directly have a messaging interface or just an API interface and use IS as the protocol translator? Pros and cons to both.
Indeed! I don’t know, of course, what some of the hurdles were but it took an iteration or 2 for SAG to get the IS-interaction with UM right and properly mimicking the same interaction IS had with Broker.
UM docs are made through Doxygen as they were designed to be similar across the various platforms we support.
UM 9.12 is rather old so I assume at some point you may want to upgrade to a generally supported release either way you can still use SharedQueue to achieve the filtering semantics you would like and when you move to 10.5+ you will migrate to the new Shared Durable.
UM has different samples available on code usage patterns in addition to the javadoc, see Developer Guide which is also available in the Document PDFs
The part of being a suite product is there is many attitudes in how to use the product, UM is perfectly suited to the IS use case as it is custom deployments which you have seen from in the other conversation it just depends what is needed.
