Dot dash, dot dash dash, dot dash, dash dash, dot dash (=Apama)
Level: INTERMEDIATE
As a little bit of Christmas fun, we’ll make use of this LED Christmas tree for Raspberry Pi by The Pi Hut to display festive tweets in morse code . This project makes use of the Twitter API, the Apama HTTP client connectivity plugin, and a C++ based EPL Plugin for interfacing with RaspberryPi’s GPIO.
As you will see, there is no “analytics” today, it is just a bit of seasonal fun that demonstrates:
- Apama running on an IoT device (the Raspberry Pi)
- Use of EPL (Day #6)
- Use of a custom EPL Plugin (to control the LEDs)
- Use of a standard connectivity plugin (HTTP Client) to integrate to an external data-source (Day #8)
And who doesn’t like flashing LED’s anyway!
Using the Twitter API
Twitter provides an API endpoint for fetching tweets which we will use for this project. To implement this project yourself, you will need to sign up for a developer account with Twitter. Once you have an account, go to your dashboard and click on “New Project” to create a project. When prompted, give your project a name, description, and use case. Once your project has been created, go to your project from the dashboard and create a new App. When you create your app you will be supplied with keys and a bearer token which you can use to authenticate with the API. You will only be given these once! If you lose them, you will have to regenerate them.
To test that your token works, run the following curl command to look up the Software AG Twitter account. You can either set $BEARER_TOKEN in your environment or replace it with your bearer token directly in the command. Try editing the query to look up your own Twitter account too.
curl "https://api.twitter.com/2/users/by?usernames=SoftwareAG" -H "Authorization: Bearer $BEARER_TOKEN"
You should see the following output:
{"data":[{"id":"20679454","name":"Software AG","username":"SoftwareAG"}]}
You can find out more about using the Twitter APIs including all of the available filters in the documentation. An easy way to generate correct API requests is using the Twitter API tools explorer.
Add Apama to your Raspberry Pi
The Apama Core Community Edition is not only available for 64bit builds of Windows/Linux on Intel & AMD, but is also available for use in IoT scenarios with a 32bit Linux build for ARMv7hf. That means you can use it on Raspberry Pi v3, v4, and Zero2 devices (but not the older v1 or Zero that were using the less capable ARMv6 CPU architecture). We’re generally using Raspbian OS on the Pi.
You can download the Apama Core Community Edition for Raspberry Pi from the downloads page on apamacommunity.com or by fetching the zip with curl (or wget ).
curl -O https://downloads.apamacommunity.com/apama-core/10.11.1.1/apama_core_10.11.1.1_armv7_linux.tar.gz
You should also download the checksum file (available on the same page) so that you can verify the download.
curl -O https://downloads.apamacommunity.com/apama-core/10.11.1.1/apama_core_10.11.1.1.sha256sum.txt
Verify the download with (change version as appropriate):
sha256sum --ignore-missing -c apama_core_10.11.1.1.sha256sum.txt
Assuming those files are in your ~/Downloads folder, we’re going to unpack them into a new softwareag folder in your home.
cd ~
mkdir softwareag
cd softwareag
tar -xf ~/Downloads/apama_core_10.11.1.1_armv7_linux.tar.gz
We can quickly check the correlator runs by setting up the environment and running the correlator to get the version string:
cd Apama
. bin/apama_env
correlator --version
and you should see the following output:
correlator v10.11.1.1.397181
(build rel/10.11.1.x@397181 on armv7-debian10 using Software AG suite version 10.11)
Now we are (nearly) ready to add the Advent Christmas tree project!
WiringPi dependency
The GPIO plugin we’ll use was written when an open-source library called “WiringPi” was mainstream and part of the normal OS distribution for the Raspberry Pi.
If you have a very recent OS version you may find that it is no longer installed or even available in the repo’s to be able to apt-get install.
You can check if you have wiringpi installed by attempting to run the following command:
gpio -v
The following worked for us when testing on a freshly installed Raspberry Pi OS (bullseye) in Dec 2021 - it is using instructions from the original author of WiringPi. http://wiringpi.com/wiringpi-updated-to-2-52-for-the-raspberry-pi-4b/
cd /tmp
wget https://project-downloads.drogon.net/wiringpi-latest.deb
sudo dpkg -i wiringpi-latest.deb
When running the gpio -v command again, it reports version 2.52.
Great. Now we’re ready to install our pre-made Christmas tree project and run it.
Running the Project on Raspberry Pi
To run the Christmas Tree project, clone the Apama_RPi_Advent_Tree repository from GitHub into your home directory on the Raspberry Pi.
cd ~
git clone https://github.com/louiseam-sag/Apama_RPi_Advent_Tree
This contains a pre-compiled binary of the GPIO EPL plugin and a deployed Apama project that will flash the LEDs using Morse code whenever tweets are received via a Twitter stream (see configuration steps above).
You will need to supply your own Twitter bearer token to the TweetsToLEDs.mon file.
Add the plugin location to your $LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/Apama_RPi_Advent_Tree/lib
Start the project with a single apama_env command:
$HOME/softwareag/Apama/bin/apama_env correlator --config $HOME/Apama_RPi_Advent_Tree/project
Flashing lights! Hurray!
You should now see the correlator start, collect some tweets, and then begin to log lines similar to the following:
2021-12-24 16:58:47.880 INFO [1740605568] - com.apamax.twitter.TwitterStream [9] DASH
2021-12-24 16:58:47.903 INFO [1761585280] - com.apamax.twitter.TwitterStream [7] DOT
2021-12-24 16:58:48.372 INFO [1793054848] - com.apamax.twitter.TwitterStream [11] DASH
2021-12-24 16:58:48.873 INFO [1782564992] - com.apamax.twitter.TwitterStream [14] DOT
2021-12-24 16:58:48.873 INFO [1841268864] - com.apamax.twitter.TwitterStream [15] DASH
2021-12-24 16:58:48.881 INFO [1751095424] - com.apamax.twitter.TwitterStream [8] DASH
2021-12-24 16:58:48.904 INFO [1761585280] - com.apamax.twitter.TwitterStream [7] DOT
The separate LED’s are flashing content of different tweets in morse-code.
When you want to stop it, simply kill the correlator process.
If you want it flashing faster, change line 339 of GPIOPlugin.mon - the default value is 500ms.
Extension 1: Using the GPIO EPL Plug-in in different ways
This is not necessary if you want to just run the Christmas tree project as I’ve included a precompiled binary and EPL wrapper in the project however the plug-in repository contains additional goodies including a plug-in that connects to a RaspberryPi Sense Hat and an experimental SPI plugin that you might want to use in your own projects.
First, you will need to build the plugin on your Raspberry Pi. Clone the repository and run make on the gpio directory:
git clone https://github.com/louiseam-sag/apama_GPIO && cd apama_GPIO/gpio && make
The makefile will automatically add the library to your LD_LIBRARY_PATH variable.
Make sure to include the plugin wrapper in the initialization list for any project you decide to make. With this plugin, you can control the GPIO pins directly, there are wrappers for easy set-up of LEDs and buzzers, and a Morse code converter that encodes strings into Morse code and sends them to your LEDs or buzzers.
Extension 2: Integrate with thin-edge.io
Maybe you’d like to control when the Christmas tree lights are enabled by adding support for thin-edge.io to allow integration with Cumulocity IoT cloud.
We have some examples to help you get started in GitHub at: https://github.com/thin-edge/thin-edge.io_examples/tree/main/StreamingAnalytics
For example, you could also register your phone with Cumulocity IoT, set up a geofence, and only have the lights enabled when you are at home.
Or, by also monitoring real-time data from an air-quality sensor over MQTT on the Thin-edge bus, you could turn on, or speed up, the lights as the CO2 in your room increases.
Final comments
We hope you have enjoyed this series of articles, and that you’ve learned more about Apama Streaming Analytics, and Cumulocity IoT, along the way.
We normally have a range of job opportunities available in various locations and different kinds of roles. Apama R&D is located mostly in Cambridge, UK, and Hyderabad, India. Cumulocity IoT R&D is in various European locations, and then there are often further roles available in PreSales and Consulting services globally. Take a look at https://jobs.softwareag.com - come join us!
Finally, may we take this opportunity to wish you all seasonal greetings, and a Happy New Year.
Best wishes, from Team Apama.
Now sit back and enjoy some sparkling Christmas lights!
Today’s article was kindly provided by Louise Amas a new graduate engineer from the Apama Engineering team.
This is Day #24 (final day) of a short series of brief tips, tricks, hints, and reminders of information relating to the Apama Streaming Analytics platform, both from Software AG as well as from the community.