In Apama 10.3.1, we added support for HTTP redirects, cookie handling for requests, HTTP request decoding, and HTML form encoding to the HTTP client transport. We also added support for HTTP requests and HTML form decoding to the HTTP server transport.
HTTP redirect support in the HTTP client transport
The HTTP client transport now supports HTTP redirects transparently. To enable this, set the followRedirects configuration option to true . If the URL you requested returns an HTTP redirect response, your request is made automatically on the URL to which you were redirected. For GET requests, this will be another GET request. POST and PUT requests will be converted to GET requests instead (this is standard behavior and a lot of REST APIs rely on this behavior).
The following configuration example shows how to define followRedirects in your HTTP client chain:
- httpClient:
host: www.example.com
followRedirects: true
HTTP cookies support in the HTTP client transport
The HTTP client transport can now transparently handle HTTP cookies for requests so that you no longer need to handle them in EPL. To enable this, set the cookieJar configuration option to true. The HTTP client then stores in memory all cookies received from the server and adds them to subsequent outgoing requests. Expired cookies are automatically removed from the internal cache. The HTTP client also honors additional cookie attributes such as path , expiry and max-age.
The following configuration example shows how to define cookieJar in your HTTP client chain:
- httpClient:
host: www.example.com
cookieJar: true
Gzip/deflate decoding support in the HTTP client transport
The HTTP client transport can now decode HTTP responses that have been encoded with gzip or deflate compression format. It is able to accept compressed responses from servers and will automatically decompress the result. No configuration is needed to turn this on, the transport always permits the server to send encoded responses.
If the server chooses to compress the body of the response with gzip or deflate compression format, the HTTP client transport will decompress the body of the response. Warnings will be logged for all unsupported content encodings.
Gzip/deflate decoding support in the HTTP server transport
The HTTP server transport can now decode HTTP requests that have been encoded with gzip or deflate compression format. No configuration is needed to turn this on. The decoding is based on the Content-Encoding header. If the client sends an encoded request, the HTTP server automatically decodes it. Warnings will be logged for all unsupported content encodings.
You can demonstrate the decoding with an HTTP server configuration like the one below. This configuration assumes that the Request event type has fields directly matching those in the JSON payload of the request.
dynamicChains:
http:
- apama.eventMap:
defaultEventType: Request
defaultChannel: requests
- jsonCodec
- stringCodec
- httpServer:
automaticResponses: true
allowedMethods: [POST]
You can then use curl to send the gzip-compressed HTTP request to the HTTP server transport, with the body of the HTTP request in JSON format:
echo '{"dataStr": "testing http server decompression", "dataInt":"1234", "dataFloat":"123.456", "dataBool":"true"}' | gzip > body.gz
curl -v -i http://localhost:9090 -H'Content-Encoding: gzip' -H'Content-type: application/json' --data-binary @body.gz
The HTTP server transport will read the Content-Encoding header and automatically decompress the payload.
HTML form decoding support in the HTTP server transport
The HTTP server transport now supports HTML form decoding and can decode multipart/form-data or application/x-www-form-urlencoded media types to a dictionary payload.
EXAMPLE: URL ENCODING
If the Content-Type header field contains the application/x-www-form-urlencoded media type, the request payload is decoded to a dictionary payload with string keys and string values.
The following configuration example for the HTTP server transport shows how you can make use of URL encoding:
dynamicChains:
http:
- apama.eventMap:
defaultEventType: URLEncodedData
defaultChannel: foo
- httpServer:
automaticResponses: true
allowedMethods: [PUT,POST]
The body of the HTTP request is directly decoded into a dictionary and mapped to a URLEncodedData event type. We’re also assuming here that the fields in the URLEncodedData event type match the fields in the form and no mapping is needed between them. You can also see that we don’t need a String codec and a JSON codec, since the transport is producing a dictionary payload directly.
The following code block shows the EPL event types and monitor code required to listen for the events from the above chain and then log the result:
event URLEncodedData{
string foo;
string abc;
}
monitor serverMonitor{
action onload()
{
monitor.subscribe("foo");
on all URLEncodedData() as urlData
{
log "EPL Received Event : " + urlData.toString() at INFO;
}
}
}
You can then use curl to send HTTP request to the HTTP server transport, with the request body containing two form fields:
curl -v -i http://localhost:9090 --data foo=value1 --data abc=value2
EXAMPLE: MULTIPART/FORM-DATA
If the Content-Type header field contains the multipart/form-data media type, the request payload is decoded to a dictionary payload with string keys and either string or binary values. For the parts that have binary data, additional metadata is created. This metadata contains the contentType , charset or filename information for each binary part. You can get the metadata as follows:
metadata.http.form.name.contentTyp
metadata.http.form.name.charset
metadata.http.form.name.filename
where name corresponds to the data in payload.name .
The following configuration example for the HTTP server transport shows how you can use the Mapper codec to map the metadata of a binary form field.
dynamicChains:
http:
- apama.eventMap:
defaultEventType: FormData
defaultChannel: foo
- mapperCodec:
"*":
towardsHost:
mapFrom:
- payload.contentType: metadata.http.form.file.contentType
- payload.filename: metadata.http.form.file.filename
- httpServer:
automaticResponses: true
allowedMethods: [PUT,POST]
The following code block shows the EPL event types and monitor code required to listen for the events from the above chain and then log the result:
event FormData{
string foo;
string abc;
string contentType;
string filename;
}
monitor serverMonitor{
action onload()
{
monitor.subscribe("foo");
on all FormData() as formData
{
log "EPL Received Event : " + formData.toString() at INFO;
}
}
}
You can then use curl to send an HTTP POST request to the HTTP server transport, with the request payload containing three form fields:
curl -v -i http://localhost:2498 -F foo=value1 -F abc=value2 -F file=@./test.txt
Please note that the correlator can’t directly understand binary components, you will need to decode them with a codec like the String codec (provided with Apama) or a custom codec that you write yourself.
HTML form encoding support in the HTTP client transport
The HTTP client transport now supports HTML form encoding and can encode a dictionary payload to either multipart/form-data or application/x-www-form-urlencoded media types.
To encode the request as multipart/form-data , you must set metadata.contentType to multipart/form-data and then map the payload to a dictionary with string keys and either string or binary payloads. You must use this method to send non-ASCII text or binary data. The binary data form fields should have the following additional metadata: filename , contentType and charset . filename is a required parameter. You can put these metadata items in a form dictionary as follows:
metadata.http.form.name.contentType metadata.http.form.name.charset metadata.http.form.name.filename
where name corresponds to the data in payload.name .
You can also encode the request as application/x-www-form-urlencoded by setting metadata.contentType to application/x-www-form-urlencoded and mapping the payload to a dictionary with string keys and values.
The following configuration shows how you can use the Mapper codec to map the metadata and payload:
- mapperCodec:
HTTPRequestMultiPartForm:
towardsTransport:
mapFrom:
- metadata.requestId: payload.id
- metadata.http.method: payload.method
- metadata.http.path: payload.path
- metadata.contentType: payload.contentType
- metadata.http.form.binary.contentType: payload.formMetadata.binary.contentType
- metadata.http.form.binary.filename: payload.formMetadata.binary.filename
- metadata.http.form.binary.charset: payload.formMetadata.binary.charset
- payload: payload.data
HTTPRequestURLEncoding:
towardsTransport:
mapFrom:
- metadata.requestId: payload.id
- metadata.http.method: payload.method
- metadata.http.path: payload.path
- metadata.contentType: payload.contentType
- payload: payload.data
The following is an example of an EPL application that shows the event types and monitor code for sending data with application/x-www-from-urlencoded and multipart/form-data.
//HTTPClient sending a dictionary payload request body having both key and value strings using application/x-www-form-urlencoded method
event HTTPRequestURLEncoding {
integer id;
string method;
string path;
string contentType;
dictionary<string, string> data;
}
//HTTPClient sending a dictionary payload request body having string key and string value using multipart/form-data method.
//Using formMetadata to provide the metadata for binary form data.
event HTTPRequestMultiPartForm {
integer id;
string method;
string path;
string contentType;
dictionary<string, string> data;
dictionary<string, dictionary<string,string> > formMetadata;
}
monitor TestFormEncoding {
action onload() {
dictionary<string, string> dataURL := {"string":"Hello World", "foo":"bar"};
integer id := integer.incrementCounter("HTTPClient.requestId");
//Sending application/x-www-form-urlencoded data
send HTTPRequestURLEncoding(id, "POST", "/", "application/x-www-form-urlencoded", dataURL) to "http";
dictionary<string, string> dataMultiPart := new dictionary<string, string>;
dataMultiPart.add("string": "Hello World");
string binaryData := ...; // got from some source
dataMultiPart.add("binary": binaryData);
//Metadata for form binary data field
dictionary<string,dictionary<string,string> > formMetadata := {
"binary":{
"filename":"file.txt",
"charset":"utf-8",
"contentType":"text/plain"
}
};
id := integer.incrementCounter("HTTPClient.requestId");
//Sending multipart/form-data
send HTTPRequestMultiPartForm(id, "POST", "/", "multipart/form-data", dataMultiPart, formMetadata) to "http";
}
}
Summary
The HTTP client transport can automatically handle HTTP redirects and HTTP cookies, and can encode HTML forms. The HTTP server transport can decode HTML forms. Both transports (HTTP client and HTTP server) support decompression.
For more information, see the following sections in the online documentation for Apama 10.3.1: “The HTTP Server Transport Connectivity Plug-in” and “The HTTP Client Transport Connectivity Plug-in”.
Feel free to post on our our stack exchange site with any questions you may have about this, or any other, topic.