What evidence do you have of a memory leak? Does Integration Server ever crash as a result of running out of memory? If Integration Server is running out of memory, then the JVM will throw OutOfMemoryError exceptions which would be logged in the error or server log - have you seen any of those?
If there is a memory leak, then it’s likely due to some kind of non-Java operating system resource being allocated and not released properly, such as file handles for files opened but never closed, or socket connections opened but never closed. Does this sound possible for your use of Integration Server?
But perhaps your issue is just a misunderstanding of how the Java Virtual Machine (JVM) manages memory?
The JVM uses garbage collection for reclaiming heap memory from objects that are no longer in use or referenced. Garbage collection is a very expensive operation, and can result in small pauses in a Java application because garbage collection is a stop-the-world event where all threads are paused until the garbage collection finishes.
Since Java 1.2, the JVM has used a generational garbage collection algorithm to improve the efficiency of garbage collections. Generational garbage collection is based on the observation that most objects are short-lived (high infant mortality rate). The heap is divided into a number of generations, and, for the sake of argument, let’s just call it 2 generations (it’s more complicated than this in reality): the young, and old generations.
Objects are created in the young generation on the heap. When the young generation fills up, a minor garbage collection runs. Minor garbage collections are pretty efficient because it only checks objects in the young generation and most of those objects are likely dead. Any objects that survive a minor garbage collection are eventually promoted to the old generation.
Eventually the old generation will fill up as well, and then a major garbage collection event is required. Major garbage collections are much slower because they involve all objects, not just the young ones, and so major garbage collection events are minimized as much as possible.
For long-running JVM processes like Integration Server, we can observe the following pattern with the heap memory usage: the used heap memory will continue to grow until no more objects can be allocated, and at that point the JVM will run a major garbage collection and reclaim a significant amount of memory. On the Integration Server administration web page, this pattern looks like Integration Server is slowly using more and more memory all the way up to its limit, which is exactly what you described. However, once the heap is finally full, you should see the memory usage will drop right down due to a major garbage collection event. This behaviour is perfectly normal, because the JVM does not run a garbage collection until it actually needs to.
Does this sound like an explanation for what you’ve observed with your Integration Server?
Refer to this tutorial to learn more about JVM garbage collection: http://www.oracle.com/webfolder/technetwork/tutorials/obe/java/gc01/index.html.