Passing Secrets to the Correlator With Docker and Kubernetes

Knowledge base
,
1

Docker and Kubernetes both provide a means for passing encrypted data into your application through secrets.

To demonstrate this, a new sample has been added to our GitHub repository.

This sample demonstrates how to set the variable CORRELATOR_NAME using a secret, which is then read by a configuration file which is loaded into the correlator.

These secrets could be used for securely passing credentials to HTTP/REST clients. An example of doing this to access the openweathermap.org API is shown below (all commands are for Linux). This demonstration is an adaptation of the samples\connectivity_plugin\application\weather sample shipped with Apama.

Creating a Docker Secret

  1. Create a docker secret using the docker secret tool, for example, weather_api_key.properties:

echo "APPID=key-11111111111" | docker secret create weather_api_key.properties -

  1. Build the sample

docker build -t weatherSample .

  1. Create the service in detached mode, passing in the secret

docker service create --name weatherSample --secret=weather_api_key.properties -d weatherSample

  1. Inspect the logs to ensure everything is working as expected

docker service logs weatherSample

Creating a Kubernetes Secret

  1. Create a Kubernetes secret with the data you want to pass in, for example, secret.yml:

apiVersion: v1
kind: Secret
metadata:
  name: weather_api_key
type: Opaque
data:
  secret.properties: QVBQSUQ9a2V5LTExMTExMTExMTEx
# The above is APPID=key-11111111111 encoded in base64. On Linux:
# echo -n 'APPID=key-11111111111' | base64
  1. Create your pod, referencing the secret you created, for example kubernetes.yml:

apiVersion: v1
kind: Pod
metadata:
  name: sample
spec:
  containers:
  - name: correlator
    image: correlator-image
    volumeMounts:
    - name: secrets
      mountPath: "/run/secrets"
      readOnly: false
  volumes:
  - name: secrets
    secret:
      secretName: weather_api_key

  1. Create your secret:
    “ kubectl create -f ./secret.yml"

  2. Build the image:
    “ docker build -t sample ."

  3. Tag the image:
    “ docker tag sample registry/org/repository:image"

  4. Push the image:
    “ docker push registry/org/repository:image"

  5. Start the sample:
    “ kubectl create -f kubernetes.yml"

  6. Inspect the logs:
    “ kubectl logs sample"

Using the Secret

Allow your correlator to see this secret in your Dockerfile, for example “ CMD ["correlator", "--config", "/run/secrets/"] ”

In your correlator configuration file, reference the variable set in your secret. In this case, we want to append “ APPID={value of weather_api_key} “, like so:


startChains:
  httpWeather:
    - apama.eventMap:
        subscribeChannels:
          - httpRequest
    - mapperCodec:
        # HTTPRequest are outbound only, set the method to GET and copy
        # the URL and ID to metadata
        apamax.weather.HTTPRequest:
          towardsTransport:
            mapFrom:
              - metadata.http.path: payload.path
              - metadata.requestId: payload.id
##################################################################
              - metadata.http.queryString.APPID: ${APPID}
##################################################################
            defaultValue:
              - metadata.http.method: GET
        # Responses are success replies, moving the ID
        # into a payload field and setting the correct channel
        apamax.weather.WeatherResponse:
          towardsHost:
            mapFrom:
              - payload.id: metadata.requestId
              - payload.temperature: payload.main
            defaultValue:
              - metadata.sag.channel: weather
    - httpClient:
        host: api.openweathermap.org

In your EPL, send a HTTP request (in this case for the weather in London), and have the APPID automatically appended onto the end of the request:


event HTTPRequest
{
    integer id;
    string path;
}
event WeatherResponse
{
    integer id;
    string name;
    sequence<dictionary<string, string> > weather;
    dictionary<string, float> temperature;
    dictionary<string, float> wind;
}
[...]
integer id := integer.getUnique();
on WeatherResponse(id=id) as resp {
    print "Weather for "+resp.name+" is:";
    print " * Temperature: " + (resp.temperature["temp"] - 273.15).formatFixed(3);
}
[...]
send HTTPRequest(id, "/data/2.5/weather?q=London") to "httpRequest";

Removing a Secret

Note that you must explicitly remove a secret. This is done in Docker with:

docker secret rm weather_api_key.properties

And in Kubernetes with:

kubectrl delete -f secret.yml