webMethods API Gateway tutorial
Author: Jagadish, Dinesh (External)
Supported Versions: 10.2, 10.3 & 10.4
Overview of the tutorial
This tutorial helps to understand how the EventDataStore (or a simple Elasticsearch instance) can be secured using Search Guard, an Elasticsearch plugin that offers encryption, authentication and authorization. In this context, this plugin helps to secure the EventDataStore by exposing it over HTTPS protocol and enabling a basic authentication by configuring users.
This tutorial covers the below steps in detail.
- Installation and initialization of Search Guard plugin
- Adding users for basic authentication
- Changing Kibana configurations to securely connect to Elasticsearch
- Configuring API Gateway to securely connect to Elasticsearch
- Configuring Search Guard with user generated certificates
Required knowledge
The reader has to have:
- a basic understanding of API Gateway and its communication with EventDataStore for storing data
- a basic understanding of Kibana and its communication with EventDataStore for rendering the dashboards in API Gateway
Why?
EventDataStore, a wrapper over Elasticsearch, being the back-end store for API Gateway, it is necessary to protect this store from attackers and other misuses which would eventually damage the entire functionality of the API Gateway in a way that it can not be recoverable. Hence it is almost important to protect the EventDataStore and this security aspect can be achieved by using Search Guard, a third party plugin, which can be plugged into the Elasticsearch server for providing security functionalities.
Prerequisite steps
Complete the below prerequisites steps before get into the details of securing EventDataStore of API Gateway using SearchGuard plugin.
- Install API Gateway advanced edition of version 10.2 and above
Details
The below sections describe the details of securing EventDataStore using Search Guard plugin by installing the plugin and adding the necessary configurations both in EventDataStore, Kibana and API Gateway.
Step 1: Installation and initialization of Search Guard plugin
This section explains how to install the Search Guard plugin and make the EventDataStore ready for enabling security.
Shutdown the API Gateway. Open <SAG_Root>/EventDataStore/bin/enable_ssl.bat and comment out the last line ..\plugins\search-guard-5\tools\sgadmin.bat and save.
Copy sagconfig from <SAG_Root>/IntegrationServer/instances/<Instance_Name>/packages/WmAPIGateway/config/resources/elasticsearch to <SAG_Root>/EventDataStore. Then copy the certificates node-0-keystore.jks and truststore.jks from <SAG_Root>/EventDataStore/sagconfig to <SAG_Root>/EventDataStore/config.
Open <SAG_Root>/EventDataStore/config/elasticsearch.yml file. Remove all the properties that start with "searchguard" if such present and add the Search Guard properties as given below and save the file.
searchguard.ssl.transport.keystore_type: JKS
searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks
searchguard.ssl.transport.keystore_alias: cn=node-0
searchguard.ssl.transport.keystore_password: a362fbcce236eb098973
searchguard.ssl.transport.truststore_type: JKS
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_alias: root-ca-chain
searchguard.ssl.transport.truststore_password: 2c0820e69e7dd5356576
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.transport.enable_openssl_if_available: true
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_type: JKS
searchguard.ssl.http.keystore_filepath: node-0-keystore.jks
searchguard.ssl.http.keystore_alias: cn=node-0
searchguard.ssl.http.keystore_password: a362fbcce236eb098973
searchguard.ssl.http.truststore_type: JKS
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.truststore_alias: root-ca-chain
searchguard.ssl.http.truststore_password: 2c0820e69e7dd5356576
searchguard.ssl.http.clientauth_mode: OPTIONAL
searchguard.authcz.admin_dn:
- "CN=sgadmin"
Search Guard properties
A brief description of each Search Guard property is given below.
| Item | Desc | Possible Values | Default |
|---|---|---|---|
| TRANSPORT ( 2-Way authentication is enabled by default) | |||
| searchguard.ssl.transport.keystore_type | Type of keystore | JKS, PKCS12 | JKS |
| searchguard.ssl.transport.keystore_filepath | Location where the keystore | ||
| searchguard.ssl.transport.keystore_alias | Keystore entry name if there are more than one entries. | ||
| searchguard.ssl.transport.keystore_password | Password to access keystore. | ||
| searchguard.ssl.transport.truststore_type | Type of truststore | JKS, PKCS12 | JKS |
| searchguard.ssl.transport.truststore_filepath | Location where the truststore | ||
| searchguard.ssl.transport.truststore_alias | Truststore entry name if there are more than one entries. | ||
| searchguard.ssl.transport.truststore_password | Password to access truststore. | ||
| searchguard.ssl.transport.enforce_hostname_verification |
If true, the hostname mentioned in certificate will be validated. We set this to false as ours is general purpose self signed certs. |
true / false | true |
| searchguard.ssl.transport.resolve_hostname |
Applicable only if above property is true. If true, the hostname will be resolved against the DNS server. We set this to false as ours is general purpose self signed certs. |
true / false | true |
| searchguard.ssl.transport.enable_openssl_if_available | Use if OpenSSL is available instead of JDK SSL. | true / false | true |
| HTTP | |||
| searchguard.ssl.http.enabled | To enable the SSL for REST interface ( HTTP). We set this to true. | true / false | false |
| searchguard.ssl.http.keystore_type | Type of keystore | JKS, PKCS12 | JKS |
| searchguard.ssl.http.keystore_filepath | Location where the keystore | ||
| searchguard.ssl.http.keystore_alias | Keystore entry name if there are more than one entries. | ||
| searchguard.ssl.http.keystore_password | Password to access keystore. | ||
| searchguard.ssl.http.truststore_type | Type of truststore | JKS, PKCS12 | JKS |
| searchguard.ssl.http.truststore_filepath | Location where the truststore | ||
|
searchguard.ssl.http.truststore_alias |
Truststore entry name if there are more than one entries | ||
| searchguard.ssl.http.truststore_password | Password to access truststore. | ||
| searchguard.ssl.http.clientauth_mode |
Option to enable 2-way authentication. REQUIRE : Client will be challenged for client certificate. OPTIONAL : Will be used if client certificate is available NONE : Ignore client certificate even if it's available. |
REQUIRE, OPTIONAL, NONE |
OPTIONAL |
| Search Guard Admin | |||
| searchguard.authcz.admin_dn |
Search Guard maintains all the data in a index called "searchguard". This is accessible to only users ( client cert which will be passed in sdadmin command) configured here. |
||
| Misc | |||
|
searchguard.cert.oid
|
All certificates used by the nodes on transport level need to have the Search Guard to identify if an incoming request comes from a trusted node in the cluster or not. In the former case, all actions are allowed, in the latter case, privilege checks apply. Plus, the oid is also checked whenever a node wants to join the cluster. |
'1.2.3.4.5.5' |
|
|
searchguard.config_index_name |
Index where all the security configuration is stored. It's not confgurable for now but in the future |
searchguard |
Now run <SAG_Root>/EventDataStore/bin/enable_ssl.bat file. This will Install Search Guard plugin and start the EventDataStore.
Now shutdown and restart the EventDataStore. Go to <SAG_Root>/EventDataStore/plugins/search-guard-2/tools and execute the below command.
sgadmin.bat -cd ..\..\..\sagconfig\ -ks ..\..\..\sagconfig\sgadmin-keystore.jks -kspass 49fc2492ebbcfa7cfc5e -ts ..\..\..\sagconfig\truststore.jks -tspass 2c0820e69e7dd5356576 -nhnv -p 9340 -cn SAG_EventDataStore
This will Initialize the EventDataStore. Refer the image given below.
Step 2: Adding users for basic authentication
In this section let us see how an user can be provisioned in the Search Guard plugin to provide a http basic authentication.
Go to <SAG_Root>\EventDataStore\sagconfig and open sg_roles_mapping.yml file. Add the username (e.g. TestUser) in the users list as given in the below image.
Before providing the password it must be converted into hash code. To generate the hash code for your password , execute <SAG_Root>\EventDataStore\plugins\search-guard-5\tools\hash.bat. Type the password, press enter and it will generate the hash code as given in below image.
Now open sg_internal_users.yml file from <SAG_Root>\EventDataStore\sagconfig and add the username and password as given in the below image.
After completing the configurations, initialize Search Guard plugin again by executing the sgadmin.bat command as we did before. After initialization shutdown and restart the EventDataStore. Now the EventDataStore is running securely by opening a HTTPS port and requesting basic authentication details.
Step 3: Changing Kibana configurations to connect to Elasticsearch
Now we need to modify the Kibana server configurations to connect to the Elasticsearch securely over HTTPS port using basic authentication details.
- Open kibana.yml file from the directory <SAG-Home>\profiles\IS_default\apigateway\dashboard\config
- Remove the comments and update the following properties given below
- elasticsearch.username: TestUser
- elasticsearch.password: TestUser@123
- elasticsearch.ssl.verificationMode: certificate
- elasticsearch.ssl.certificateAuthorities: <file path of your root-ca.pem certificate>
- elasticsearch.url: https://<hostname>:9240
- After changing these configurations open uiconfiguration.properties file at <SAG_Home>\profiles\IS_default\apigateway\config and set apigw.kibana.autostart to false
Sample kibana.yml file image is given below.
Step 4: Changing API Gateway configurations to connect to Elasticsearch
Finally change the API Gateway configurations to connect to EventDataStore securely. Follow the below steps.
- Go to <SAG_HOME>/IntegrationServer/instances/default/packages/WmAPIGateway/config/resources/elasticsearch and open config.properties file. Uncomment the following properties and provide the values for them as below
- pg.gateway.elasticsearch.http.username=TestUser
- pg.gateway.elasticsearch.http.password=TestUser@123
- pg.gateway.elasticsearch.https.truststore.filepath=C:/SoftwareAG-A-103/EventDataStore/sagconfig/truststore.jks
- pg.gateway.elasticsearch.https.truststore.password=2c0820e69e7dd5356576
- pg.gateway.elasticsearch.https.enabled=true
- After making all the configurations, start the EventDataStore manually
- When EventDataStore is up and running, start the Kibana server manually (To start kibana server manually run kibana.bat file at <SAG_HOME>/profiles\IS_default\apigateway\dashboard\bin)
- Now start the API Gateway and you would be able to login and create APIs. Analytics page should be accessible without any challenge window for user credentials
Step 5: Configuring Search Guard with user generated certificates (optional)
The API Provider can generate his own certificates to be used with Search Guard instead of the default certificates that are shipped with API Gateway. To achieve this, Search Guard provides an offline TLS tool, using which all the required certificates can be generated for running Search Guard in a production environment. Follow the below steps to generate certificates using Search Guard offline TLS tool.
Download the tool zip file from https://search.maven.org/#search%7Cga%7C1%7Ca%3A%22search-guard-tlstool%22 (the same can be found in the attachments) and unzip it to a directory. Create a yaml file at <OfflineTool Installation Directory>\config. On executing the TLS tool command, it will read the node and certificate configuration settings from this yaml file and outputs the generated files in a configured directory. The sample file is given below. Please find the attachments for the sample file, Certificates.yml.
After configuring the yaml file run the below command to generate the required certificates.
<OfflineTool Installation Directory>/tools/sgtlstool.bat -c ../config/Demo.yml -ca -crt
The generated certificates can be outputted at <OfflineTool Installation Directory>/tools/out
For further details on this refer the link provided in the References. It is mandatory to copy the certificates listed below from <OfflineTool Installation Directory>/tools/out to <SAG_Root>/EventDataStore/config folder.
- Dinesh-node-1.key
- Dinesh-node-1.pem
- Dinesh-node-1_http.pem
- Dinesh-node-1_http.key
- Dinesh-client.pem
- Dinesh-client.key
- root-ca.pem
- root-ca.key
Now configure the generated certificates in the EventDataStore elasticsearch.yml file.
After completing all the above steps start the EventDataStore manually. After EventDataStore is up, since the Search Guard is not initialized with the latest certificates, a log message would be shown saying that the Search Guard is not initialized. The following section provide the steps to initialize the Search Guard with the generated certificates.
- Open command prompt and change directory to <SAG-Home>\EventDataStore \plugins\search-guard-5\tools
- Execute the command sgadmin.sh -cd ..\sagconfig –nhnv –icl –cacert ..\..\..\config\root-ca.pem -cert ..\..\..\config\Dinesh-client.pem -key ..\..\..\config\Dinesh-client.key -keypass <your certificate password> -p 9340 . A log "Done with success" will show up
- Now shutdown and restart the EventDataStore. The EventDataStore will now use the generated certificates for SSL communication
Troubleshooting
| # | Problem | Solution |
| 1 |
Search Guard may not be initialized. Initialize Search Guard Using sgadmin tool |
Make sure that Search Guard is initialized properly using the steps given above |
| 2 |
Error in ElasticSearch: |
It means Kibana or API Gateway is still communicating with Elasticsearch in HTTP. |
References
- Refer https://docs.search-guard.com/latest/index.html for Search Guard documentation
- Refer https://docs.search-guard.com/latest/offline-tls-tool#tls-tool for detailed understanding of Search Guard offline TLS tool
- Refer https://github.com/floragunncom/search-guard/wiki for Searchh Guard and Elasticsearch support matrix
Downloadable artifacts
- Certificates.yml file for Search Guard certificates configuration
- search-guard-tlstool-1.5.zip, Search Guard offline TLS tool
Learn more
- Refer http://techcommunity.softwareag.com/pwiki/-/wiki/Main/Tips+and+Tricks+of+API+Gateway for a detail on setting up Elasticsearch cluster