EPL Memory profiling is about investigating and analyzing an Apama application’s behavior to aid in optimizations for better memory usage and identifying any leaks. Apama’s capability to profile the EPL memory usage does provide a handy mechanism to identify any listener or object leaks used in the application within the correlator. Identifying such leaks could otherwise be a daunting task for any developer.
Note: Usage of the memory profiler is recommended during the development or debugging of an EPL application but, not when in production since it impacts the performance of the application.
The EPL memory profiler is invoked using the -r (or --dorequest ) option of the engine_management tool, followed by a request. Several requests are available for the EPL memory profiler, which are described below. The information that is returned for a request can be viewed directly (for example, in the Apama Command Prompt), or it can be written to a comma-separated values (CSV) file which can easily be viewed in tabular form using a tool such as Microsoft Excel.
COMMANDS
The commands applicable for the EPL memory profile are explained briefly below. Details of the profiler can be seen in the documentation (Using the EPL memory profiler).
eplMemoryProfileOverview – Returns information on all the monitors in the correlator. Example:
engine_management -r eplMemoryProfileOverview
eplMemoryProfileMonitorInstanceDetail – Returns information for all EPL types across the monitor instances of a specific monitor in the correlator. The monitor-name is expected to be the fully qualified name of the monitor. Example:
engine_management -r eplMemoryProfileMonitorInstanceDetail <monitor-name>
eplMemoryProfileMonitorDetail – Command is similar to eplMemoryProfileMonitorInstanceDetail , except that it aggregates the object count and size from each monitor instance, displaying data per monitor rather than per monitor instance. Example:
´´´´
engine_management -r eplMemoryProfileMonitorDetail
´´´´
EXAMPLE
We will use a simple EPL application which gets a Request event and processes it to return a response to demonstrate identifying leaks in an EPL application using the EPL memory profiler.
Below are the EPL files for the example.
Event Definitions :
DemoLeaksEventDefinitions.mon
package com.apamablog.leaks;
/**
* Dummy Event
*/
event DummyHolder {
sequence<integer> dummySeq;
}
/**
* Event for Request
*/
event Request {
string requestId;
DummyHolder value;
}
/**
* Event for any valid response for a Request
*/
event Response {
string requestId;
}
/**
* Event for any error during execution of Request
*/
event ResponseError {
string requestId;
string error;
}
Request Handler monitor
DemoLeakds.mon
package com.apamablog.leaks;
// Monitor
monitor DemoLeaksMonitor {
// Holder for understanding the pending requests to be processed.
dictionary<string, Request> pendingRequests := new dictionary<string, Request>;
action onload() {
on all Request() as req {
on Response(requestId = req.requestId) as response {
// Remove from the holder indicating the request is no longer pending.
if (pendingRequests.hasKey(response.requestId)) {
pendingRequests.remove(response.requestId);
log "Response received for for request " + response.requestId;
} else {
log "Error : Response received for for request which is not pending " + response.requestId;
}
}
pendingRequests.add(req.requestId, req);
}
}
}
Request and Response Generator which basically generates 100 Request events and each Request generates either a Response or ResponseError event and about 25% of the responses will be errors.
DemoGenerator.mon
package com.apamablog.leaks;
monitor DemoGeneratorMonitor {
action onload() {
// For every request randomly send a valid or invalid response.
on all Request() as req {
if float.rand(1.0) < 0.75 {
log "Response sent for for request " + req.requestId;
route Response(req.requestId);
} else {
log "ResponseError sent for for request " + req.requestId;
route ResponseError(req.requestId, "Error");
}
}
// Just send 100 requests for the simplicity to demonstrate the
// leaks
on wait (5.0) {
integer i := 0;
while (i < 100) {
i := i + 1;
DummyHolder value := new DummyHolder;
value.dummySeq := new sequence<integer>;
route Request(integer.getUnique().toString(), value);
}
}
}
}
Now Launch the Correlator and let us use the EPL Memory Profiler to understand the application memory profile. Once the Correlator is started and the EPL files injected open the “ Apama Command Prompt “.
From the above EPL application the expectation is that once all Request events are handled there should be no pending requests in the dictionary. Fire the below command in the Apama Command Prompt.
engine_management -r eplMemoryProfileOverview
| Monitor | Monitor instances | EPL objects | Listeners | Bytes | Overhead bytes |
|---|---|---|---|---|---|
| com.apamablog.leaks.DemoGeneratorMonitor | 1 | 1 | 1 | 321 | 760 |
| com.apamablog.leaks.DemoLeaksMonitor | 1 | 98 | 25 | 17840 | 15688 |
The above table is the output we get in the csv format. Notice that the DemoGeneratorMonitor has only one EPL object and one listener which is on expected lines and the memory usage is also acceptable. The DemoLeaksMonitor has 98 EPL objects and 25 active listeners with unexpectedly high memory consumption. So to understand the further details on the DemoLeaksMonitor we can use the other commands available i.e. “ eplMemoryProfileMonitorDetail ” or the “ eplMemoryProfileMonitorInstanceDetail “. As seen above the monitors have only one instance each so the instance detail would not be of great value in this case so we can probe further by using the “ eplMemoryProfileMonitorDetail ” command.
The “ eplMemoryProfileMonitorDetail ” command expects the fully qualified name of the monitor being probed so we can execute the below command in the command line.
engine_management -r eplMemoryProfileMonitorDetail com.apamablog.leaks.DemoLeaksMonitor
| Monitor | Persistent | EPL type | ELP Objects| Bytes|
|—|—|—|—|—|—|
| com.apamablog.leaks.DemoLeaksMonitor| false | com.apamablog.leaks.DemoLeaksMonitor.Global Variables | 1 | 144|
| com.apamablog.leaks.DemoLeaksMonitor| false | com.apamablog.leaks.DummyHolders | 24 | 3120|
| com.apamablog.leaks.DemoLeaksMonitor| false | com.apamablog.leaks.Request | 24 | 3232|
| com.apamablog.leaks.DemoLeaksMonitor| false | “dictionary<string,com.apamablog.leaks.Request>” | 1 | 1304|
| com.apamablog.leaks.DemoLeaksMonitor| false | Listener – C:\Users\pad\workspace912\Demo\monitors\DemoLeaks.mon:11 | 24 | 4292|
| com.apamablog.leaks.DemoLeaksMonitor| false | Listener – C:\Users\pad\workspace912\Demo\monitors\DemoLeaks.mon:9 | 1 | 200|
| com.apamablog.leaks.DemoLeaksMonitor| false | sequence | 24 | 3072|
| com.apamablog.leaks.DemoLeaksMonitor| false | string | 24 | 1776|
As seen in the above table there are unexpected 24 Request event objects which have led to a similar number of DummyHolder event, sequence , string objects since they are contained within the Request event, also there exists 24 Listener objects at line 11 i.e.
on Response(requestId = req.requestId) as response {
In this application we were expecting all the Request events to be handled and the holding dictionary of the pending request events to be empty which does not seem to be the case. We yet do have 24 Request objects being held in the monitor and as the only place we are storing the Request object is within the dictionary, so we can infer that the dictionary is not being cleaned up. As the Request event can have responses of Response and ResponseError we have not handled the ResponseError event thus leading the Request objects to be held up in the dictionary. The monitor generates for about 25% of the Request events a ResponseError event. To prevent this leak let us handle the ResponseError event as a result of a Request .
As seen below in the DemoLeaks.mon adds a listener for the ResponseError event and removes it from the dictionary. See listener at line 21 below.
DemoLeaks.mon
package com.apamablog.leaks;
//Monitor
monitor DemoLeaksMonitor {
// Holder for understanding the pending requests to be processed.
dictionary<string, Request> pendingRequests := new dictionary<string, Request>;
action onload() {
on all Request() as req {
on Response(requestId = req.requestId) as response {
// Remove from the holder indicating the request is no longer pending.
if (pendingRequests.hasKey(response.requestId)) {
pendingRequests.remove(response.requestId);
log "Response received for for request " + response.requestId;
} else {
log "Error : Response received for for request which is not pending " + response.requestId;
}
}
on ResponseError(requestId = req.requestId) as response {
// Remove from the holder indicating the request is no longer pending.
if (pendingRequests.hasKey(response.requestId)) {
pendingRequests.remove(response.requestId);
log "ResponseError received for for request " + response.requestId;
} else {
log "Error : ResponseError received for for request which is not pending " + response.requestId;
}
}
pendingRequests.add(req.requestId, req);
}
}
}
Restart the correlator inject the EPL files and run the eplMemoryProfileOverview command in the Apama Command Prompt.
engine_management -r eplMemoryProfileOverview
| Monitor | Monitor instances | EPL objects | Listeners | Bytes | Overhead bytes |
|---|---|---|---|---|---|
| com.apamablog.leaks.DemoGeneratorMonitor | 1 | 1 | 1 | 312 | 760 |
| com.apamablog.leaks.DemoLeaksMonitor | 1 | 2 | 101 | 22040 | 19072 |
As seen in the above table the DemoGeneratorMonitor is the same as expected since we made no changes to it but the DemoLeaksMonitor has 101 Listeners which is quite unexpected since we expect only one listener for the Request event and no other listeners. As explained above the “ eplMemoryProfileMonitorDetail ” command can be used to probe further.
engine_management -r eplMemoryProfileMonitorDetail com.apamablog.leaks.DemoLeaksMonitor
| Monitor | Persistent | EPL type | ELP Objects| Bytes|
|—|—|—|—|—|—|
| com.apamablog.leaks.DemoLeaksMonitor| false | com.apamablog.leaks.DemoLeaksMonitor.Global Variables | 1 | 144|
| com.apamablog.leaks.DemoLeaksMonitor| false | dictionary<string,com.apamablog.leaks.Request>” | 1| 344|
| com.apamablog.leaks.DemoLeaksMonitor| false | Listener – C:\Users\pad\workspace912\Demo\monitors\DemoLeaks.mon:11 | 31| 6448|
| com.apamablog.leaks.DemoLeaksMonitor| false | Listener – C:\Users\pad\workspace912\Demo\monitors\DemoLeaks.mon:21 | 69 | 14904|
| com.apamablog.leaks.DemoLeaksMonitor| false | Listener – C:\Users\pad\workspace912\Demo\monitors\DemoLeaks.mon:9 | 1| 200|
As seen in the above table the listeners on line 11 and 21 are unexpected while there are no more Request objects as we have cleared the holding dictionary correctly.
As seen in the EPL we have two listeners for every requestId but we get only one response i.e. either Response or ResponseError we need to terminate one listener when the other fires. We can fix this by using the ‘ and not ’ pattern – adding ‘ and not OtherEvent(…) ’ will terminate the listener without firing it if OtherEvent is received, thus fixing the listener leak.
DemoLeaks.mon
package com.apamablog.leaks;
//Monitor
monitor DemoLeaksMonitor {
// Holder for understanding the pending requests to be processed.
dictionary<string, Request> pendingRequests := new dictionary<string, Request>;
action onload() {
on all Request() as req {
on Response(requestId = req.requestId) as response and not ResponseError(requestId = req.requestId) {
// Remove from the holder indicating the request is no longer pending.
if (pendingRequests.hasKey(response.requestId)) {
pendingRequests.remove(response.requestId);
log "Response received for for request " + response.requestId;
} else {
log "Error : Response received for for request which is not pending " + response.requestId;
}
}
on ResponseError(requestId = req.requestId) as response and not Response(requestId = req.requestId) {
// Remove from the holder indicating the request is no longer pending.
if (pendingRequests.hasKey(response.requestId)) {
pendingRequests.remove(response.requestId);
log "ResponseError received for for request " + response.requestId;
} else {
log "Error : ResponseError received for for request which is not pending " + response.requestId;
}
}
pendingRequests.add(req.requestId, req);
}
}
}
Restart the correlator inject the EPL files and run the eplMemoryProfileOverview command in the Apama Command Prompt.
engine_management -r eplMemoryProfileOverview
| Monitor | Monitor instances | EPL objects | Listeners | Bytes | Overhead bytes |
|---|---|---|---|---|---|
| com.apamablog.leaks.DemoGeneratorMonitor | 1 | 1 | 1 | 312 | 760 |
| com.apamablog.leaks.DemoLeaksMonitor | 1 | 2 | 1 | 688 | 264 |
As seen in the above table now we do not have any unexpected EPL object or listener count and the memory for the monitor also seems acceptable so now there seems to be no leaks in the application.
Attached are the above artefacts for reference.
DemoLeak1.zip
DemoLeak2.zip
DemoLeak3.zip
Disclaimer:
Utilities and samples shown here are not official parts of the Software AG products. These utilities and samples are not eligible for technical support through Software AG Customer Care. Software AG makes no guarantees pertaining to the functionality, scalability, robustness, or degree of testing of these utilities and samples. Customers are strongly advised to consider these utilities and samples as “working examples” from which they should build and test their own solutions