I have a flow service which loads the configuration file (it is an xml file). I have cached this service as the config files will not be updated frequently. Also, I have invoked this service as transformer within the main service. All these things works fine.
The problem is after updating the config file, the cached values are not getting updated automatically even by invoking the service after the cache expiration interval. Even I have tried using the Prefetch option. The same thing works great when I use the option “Reset Service Cache” in admin console.
Also, one more thing is I am not passing any inputs to this service. I have read in wM documentation that the cached results will get stored based on input to the service. Only if the inputs changes, the service get the new values otherwise gets the cached data.
Therefore, pls let me know if anybody has the idea of how to cache a service which loads the config file without providing any inputs to the service.
After making the changes in the config file, you have to use “reset cache”, to remove the already cached result. It will make the room for the service to run in the integration server and store the new cached result.
You have already used the cached service as transformer, so it will be faster as the explicit values are getting compared to get the cached result.
Apart from that, you can use the cached service as the startup service, so when ever the packege gets loaded, the service will be executed and result will be available for the subsequent entry.
I got your point. But, ideally once the cache expire time is reached it should clear all the cache data and should reload the new pipeline values into the cache when the service is invoked next time. But, currently it requires an additional task of reseting the cache from UI. In this case, what is the exact use of setting the cache expire time when we have to explicitely reset the service cache after or before cache expire time is reached. I expected this to happen automatically but it is not happening :sad:…
Automatic cache flushing is controlled by the IS server property watt.server.cache.flushMins. The default setting is every 10 minutes. So if you specify the cache of a service to expire in 5 minutes, it may take up to 10 minutes for it to be purged.
It’s been my experience that caching config files isn’t valuable. It normally isn’t a bottleneck and caching it doesn’t speed things up meaningfully. It introduces issues when the config file is changed but cache doesn’t flush quickly enough automatically. Generally, if caching is useful then manual reset is usually the path to take when changes occur.
We have seen issues in some of our implementations where caching of services have not made any significant contribution to the performance. Instead of using IS provided caching, building a custom cache mechanism is more efficient. Using system level hashtable is one of the ways to implement this.
As with any tuning efforts, be sure to measure. Often what is thought to be faster, more efficient, etc. is not. Or at least not significantly so. No sense introducing complexity (e.g. dealing with cache not being cleared when you need it to be) for marginal gain.
@ reamon: It’s been my experience that caching config files isn’t valuable. It normally isn’t a bottleneck and caching it doesn’t speed things up meaningfully.
In my project, I need the config file to load for every incoming message processing for comparing the message level data with the data from the config files. Therfore, there are production scenarios where approximely 1 million messages gets bridged in a day. Therefor it is tedious to read the config file every time for procesing the incoming message. Therefore I have cached the service level data which loads the config files and I am expecting better performance improvement by this implementation. pls let me know if you don’t find any usefulness by this implemenation
@dhimate: We have seen issues in some of our implementations where caching of services have not made any significant contribution to the performance. Instead of using IS provided caching, building a custom cache mechanism is more efficient. Using system level hashtable is one of the ways to implement this.
I am using config files which are in XML fomat. These are not property files (name-value pairs). Caching using Hashtable is easier if there are name-values pairs in a single heirarchy. But, I am using complex config file structure where it is difficult to cache the values using the hashtable.
if you know any other methods, please let me know.
We cached the entire file content in a system hashtable variable. We used name as ‘file name’ and entire content of the file as ‘value’ in hashtable. This gave us improvement of approximately 2 transactions per second compared to IS caching. I am not sure if this is the most efficient way of doing it, but you might want to try it out.
A million messages a day may indeed be a volume where caching the config file could increase the throughput.
My key point: measure to find out if 1) throughput increases; 2) the increase is meaningful. If performance improves only marginally, you may decide that the improvement does not offset the refresh issues with caching.
Can you share how much increase was gained by introducing caching? And additional volume information? That can help others guage whether or not caching of config files is worthwhile for their own scenario.
I am not sure if this is the most efficient way of doing it, but you might want to try it out.