Common mistakes in EPL

Although we’ve tried our hardest to make the Apama Event Processing Language (EPL) familiar and easy to get into, there will always be some room for error. In this blog, we will explore some of the common ‘gotchas’ that often vex EPL newcomers, hopefully making the introductory experience as pleasant as possible.

Listener Capture and Scope

With the following code, one might expect output of the form ‘1, 2, 3, 4, 5…’

monitor test {
    action onload() {
        integer i := 0;
        on all wait(1.0) {
            i := i + 1;
            log i.toString();
        }
    }
}

Compared to the actual output which looks more like ‘1, 1, 1, 1, 1…’. Why is this?

When listeners are created, they capture the value of any local variables from the action, unless the variables are reference types in which case they take a reference. In the above example, i is copied by value to a new i variable contained only with the listener. When the listener cycles (ie is completed by the condition and recreated by the all ), the contained variables are recreated with their initial values. It’s important to remember that the listener action is more like a definition of a callback rather than a block which executes in flow with the rest of the surrounding code.

To create a counter of this sort, we need to instead wrap our integer in some kind of reference type, such as an event, sequence or member on the monitor. This does mean that multiple listeners capturing the same reference are in fact reading/writing the same object.

Leaks

Whilst EPL is a memory-managed and garbage-collected environment, unfortunately memory leaks are still possible. These usually come in the form of listener leaks. First let us review under which conditions listeners are destroyed:

  • They complete, ie enter a state in which they permanently evaluate to true or false. For example, on all A() and not B() completes on a B event (permanently evaluating to false) whilst on A() completes on an A event (permanently evaluating to true)
  • They are manually terminated with the .quit() action

So we could create a kind of leak by creating a listener which in practice will never complete but really should. For example, a listener which should process all events of a certain type until a ‘stop’ condition, but the ‘stop’ condition is never resolved for whatever reason (bugs, poor design, etc.).

monitor leaky {
    action onload() {
        on all StockTick() and MarketClose() {
            // we forgot to add the `not` operator to the MarketClose event and now this listener will run forever
        }
    }
}

Similarly, we could create a leak by creating a listener which we intend to manually terminate, but lose a handle to before doing so, as in the following code:

monitor leaky {
    listener l;
    action onload() {
        l := on all A() {
            log "listener 1";
        }
 
        l := on all B() {
            log "listener 2";
        }
 
        // we have lost our handle to listener 1!
    }
}

In the same way, it is also possible to leak a stream. Creating a stream without attaching it to a query will cause the stream to remain in existence, keeping a monitor instance alive and consuming resources for its processing. Consider the following code:

action streamLeakExample1(string s) {
    stream<float> prices := from t in  all Tick(symbol=s) select t.price;
    // If the elided code does not use the stream
    // a leak occurs when the prices variable goes out of scope.  
}

To avoid this, always ensure to terminate streams once you are finished with them, and avoid creating streams you do not intend to use immediately.

Contained Types Syntax

Now for a simple but common one. Container types, such as sequence and dictionary require that you specify their contained type in angle brackets, for example sequence<integer> . A small quirk is when specifying contained types – you must leave a space between each closing angle bracket, for example sequence<sequence<integer> >.

Spinning

Doing large amounts of processing in short order, for example in a loop, is often a lot more expensive and intrusive than one might expect. Let’s take a look at how this might negatively affect us:

monitor busy {
    action onload() {
        on all MyEvent(){
            do_work();
        }
    }
 
    action do_work() {
        integer i := 0;
        while (i < 1000000000) {
            // do something
            i := i + 1;
        }
    }
}

So we perform some expensive task on every event. However, due to the do_work() action being called in the same context as our listener, it will block our listener from firing again until it has completed! Additionally, a garbage collection check is only triggered upon a context being de-scheduled (either completing it’s work or swapped out for another context to run). Thus, tight loops such as the above can quickly build up memory usage as they block garbage collection from occurring, which can lead to OOM errors in an otherwise stable application.

To avoid situations like this, always use contexts to parallelise expensive tasks. Contexts are very cheap and a typical application will employ thousands of live contexts of varying lifetime. Modifying the example above:

monitor busy {
    action onload() {
        on all MyEvent() {
            // spawn to an anonymous context, will no longer block the main context
            spawn do_work();
        }
    }
 
    action do_work() {
        integer i := 0;
        while (i < 1000000000) {
            // do something
            i := i + 1;
        }
    }
}

Conclusion

We hope that this resolves some of the more common stumbling points when first developing with EPL. If there’s anything you feel we have missed or have a burning EPL pain point, why not post to Stack Overflow using the new Apama tag?