webMethods API Gateway tutorial
Author: Rizwan, Mohammed (email@example.com)
Supported Versions: 10.5
Overview of the tutorial
This tutorial helps to understand how the InternalDataStore (or a simple Elasticsearch instance) can be secured using Search Guard, an Elasticsearch plugin that offers encryption, authentication and authorization. In this context, this plugin helps to secure the InternalDataStore by exposing it over HTTPS protocol and enabling a basic authentication by configuring users.
This tutorial covers the below steps in detail.
- Installation and initialization of Search Guard plugin
- Adding users for basic authentication
- Changing Kibana configurations to securely connect to Elasticsearch
- Configuring API Gateway to securely connect to Elasticsearch
- Configuring Search Guard with user generated certificates
The reader has to have:
- a basic understanding of API Gateway and its communication with InternalDataStore for storing data
- a basic understanding of Kibana and its communication with InternalDataStore for rendering the dashboards in API Gateway
InternalDataStore, a wrapper over Elasticsearch, being the back-end store for API Gateway, it is necessary to protect this store from attackers and other misuses which would eventually damage the entire functionality of the API Gateway in a way that it can not be recoverable. Hence it is almost important to protect the InternalDataStore and this security aspect can be achieved by using Search Guard, a third party plugin, which can be plugged into the Elasticsearch server for providing security functionalities.
Complete the below prerequisites steps before get into the details of securing InternalDataStore of API Gateway using SearchGuard plugin.
- Install API Gateway advanced edition of version 10.2 and above
The below sections describe the details of securing InternalDataStore using Search Guard plugin by installing the plugin and adding the necessary configurations both in InternalDataStore, Kibana and API Gateway.
Step 1: Installation and initialization of Search Guard plugin
This section explains how to install the Search Guard plugin and make the InternalDataStore ready for enabling security.
Shutdown the API Gateway. Open <SAG_Root>/InternalDataStore /bin/enable_ssl.bat and comment out the last line ..\plugins\search-guard-5\tools\sgadmin.bat and save.
Copy sagconfig from <SAG_Root>/IntegrationServer/instances/<Instance_Name>/packages/WmAPIGateway/config/resources/elasticsearch to <SAG_Root>/InternalDataStore. Then copy the certificates node-0-keystore.jks and truststore.jks from <SAG_Root>/InternalDataStore/sagconfig to <SAG_Root>/InternalDataStore/config.
Open <SAG_Root>/InternalDataStore/config/elasticsearch.yml file. Remove all the properties that start with "searchguard" if such present and add the Search Guard properties as given below and save the file.
Search Guard properties
A brief description of each Search Guard property is given below.
|TRANSPORT ( 2-Way authentication is enabled by default)|
|searchguard.ssl.transport.keystore_type||Type of keystore||JKS, PKCS12||JKS|
|searchguard.ssl.transport.keystore_filepath||Location where the keystore|
|searchguard.ssl.transport.keystore_alias||Keystore entry name if there are more than one entries.|
|searchguard.ssl.transport.keystore_password||Password to access keystore.|
|searchguard.ssl.transport.truststore_type||Type of truststore||JKS, PKCS12||JKS|
|searchguard.ssl.transport.truststore_filepath||Location where the truststore|
|searchguard.ssl.transport.truststore_alias||Truststore entry name if there are more than one entries.|
|searchguard.ssl.transport.truststore_password||Password to access truststore.|
If true, the hostname mentioned in certificate will be validated. We set this to false as ours is general purpose self signed certs.
|true / false||true|
Applicable only if above property is true. If true, the hostname will be resolved against the DNS server. We set this to false as ours is general purpose self signed certs.
|true / false||true|
|searchguard.ssl.transport.enable_openssl_if_available||Use if OpenSSL is available instead of JDK SSL.||true / false||true|
|searchguard.ssl.http.enabled||To enable the SSL for REST interface ( HTTP). We set this to true.||true / false||false|
|searchguard.ssl.http.keystore_type||Type of keystore||JKS, PKCS12||JKS|
|searchguard.ssl.http.keystore_filepath||Location where the keystore|
|searchguard.ssl.http.keystore_alias||Keystore entry name if there are more than one entries.|
|searchguard.ssl.http.keystore_password||Password to access keystore.|
|searchguard.ssl.http.truststore_type||Type of truststore||JKS, PKCS12||JKS|
|searchguard.ssl.http.truststore_filepath||Location where the truststore|
|Truststore entry name if there are more than one entries|
|searchguard.ssl.http.truststore_password||Password to access truststore.|
Option to enable 2-way authentication.
REQUIRE : Client will be challenged for client certificate.
OPTIONAL : Will be used if client certificate is available
NONE : Ignore client certificate even if it's available.
|Search Guard Admin|
Search Guard maintains all the data in a index called "searchguard". This is accessible to only users ( client cert which will be passed in sdadmin command) configured here.
All certificates used by the nodes on transport level need to have the
Search Guard to identify if an incoming request comes from a trusted node in the cluster or not. In the former case, all actions are allowed,
in the latter case, privilege checks apply. Plus, the oid is also checked whenever a node wants to join the cluster.
|Index where all the security configuration is stored. It's not confgurable for now but in the future||
|To perform snapshot and restore operations, a user needs to have special privileges assigned. These two lines enable these privileges|
|searchguard.restapi.roles_enabled||Tells Search Guard which Search Guard roles can access the REST Management API to perform changes to the configuration|
Now run <SAG_Root>/InternalDataStore/bin/enable_ssl.bat file. This will Install Search Guard plugin and start the InternalDataStore.
Now shutdown and restart the InternalDataStore. Go to <SAG_Root>/InternalDataStore/plugins/search-guard-7/tools and execute the below command.
sgadmin.bat -cd ..\..\..\sagconfig\ -ks ..\..\..\sagconfig\sgadmin-keystore.jks -kspass 49fc2492ebbcfa7cfc5e -ts ..\..\..\sagconfig\truststore.jks -tspass 2c0820e69e7dd5356576 -nhnv -p 9340 -cn SAG_EventDataStore
This will Initialize the InternalDataStore. Refer the image given below.
Step 2: Adding users for basic authentication
In this section let us see how an user can be provisioned in the Search Guard plugin to provide a http basic authentication.
Go to <SAG_Root>\InternalDataStore\sagconfig and open sg_roles_mapping.yml file. Add the username (e.g. TestUser) in the users list as given in the below image.
Before providing the password it must be converted into hash code. To generate the hash code for your password , execute <SAG_Root>\InternalDataStore\plugins\search-guard-7\tools\hash.bat. Type the password, press enter and it will generate the hash code as given in below image.
Now open sg_internal_users.yml file from <SAG_Root>\InternalDataStore\sagconfig and add the username and password as given in the below image.
After completing the configurations, initialize Search Guard plugin again by executing the sgadmin.bat command as we did before. After initialization shutdown and restart the InternalDataStore. Now the InternalDataStore is running securely by opening a HTTPS port and requesting basic authentication details.
Step 3: Changing Kibana configurations to connect to Elasticsearch
Now we need to modify the Kibana server configurations to connect to the Elasticsearch securely over HTTPS port using basic authentication details.
- Open kibana.yml file from the directory <SAG-Home>\profiles\IS_default\apigateway\dashboard\config
- Remove the comments and update the following properties given below
- elasticsearch.username: TestUser
- elasticsearch.password: TestUser@123
- elasticsearch.ssl.verificationMode: certificate
- elasticsearch.ssl.certificateAuthorities: <file path of your root-ca.pem certificate>
- elasticsearch.url: https://<hostname>:9240
- After changing these configurations open uiconfiguration.properties file at <SAG_Home>\profiles\IS_default\apigateway\config and set apigw.kibana.autostart to false
Sample kibana.yml file image is given below.
Step 4: Changing API Gateway configurations to connect to Elasticsearch
Finally change the API Gateway configurations to connect to InternalDataStore securely. Follow the below steps.
- Go to <SAG_HOME>/IntegrationServer/instances/default/packages/WmAPIGateway/config/resources/elasticsearch and open config.properties file. Uncomment the following properties and provide the values for them as below
- After making all the configurations, start the InternalDataStore manually
- When InternalDataStore is up and running, start the Kibana server manually (To start kibana server manually run kibana.bat file at <SAG_HOME>/profiles\IS_default\apigateway\dashboard\bin)
- Now start the API Gateway and you would be able to login and create APIs. Analytics page should be accessible without any challenge window for user credentials
Step 5: Configuring Search Guard with user generated certificates (optional)
The API Provider can generate his own certificates to be used with Search Guard instead of the default certificates that are shipped with API Gateway. To achieve this, Search Guard provides an offline TLS tool, using which all the required certificates can be generated for running Search Guard in a production environment. Follow the below steps to generate certificates using Search Guard offline TLS tool.
Download the tool zip file from https://search.maven.org/#search%7Cga%7C1%7Ca%3A%22search-guard-tlstool%22 (the same can be found in the attachments) and unzip it to a directory. Create a yaml file at <OfflineTool Installation Directory>\config. On executing the TLS tool command, it will read the node and certificate configuration settings from this yaml file and outputs the generated files in a configured directory. The sample file is given below. Please find the attachments for the sample file, Certificates.yml.
After configuring the yaml file run the below command to generate the required certificates.
<OfflineTool Installation Directory>/tools/sgtlstool.bat -c ../config/Demo.yml -ca -crt
The generated certificates can be outputted at <OfflineTool Installation Directory>/tools/out
For further details on this refer the link provided in the References. It is mandatory to copy the certificates listed below from <OfflineTool Installation Directory>/tools/out to <SAG_Root>/InternalDataStore/config folder.
Now configure the generated certificates in the InternalDataStore elasticsearch.yml file.
After completing all the above steps start the InternalDataStore manually. After InternalDataStore is up, since the Search Guard is not initialized with the latest certificates, a log message would be shown saying that the Search Guard is not initialized. The following section provide the steps to initialize the Search Guard with the generated certificates.
- Open command prompt and change directory to <SAG-Home>\InternalDataStore \plugins\search-guard-5\tools
- Execute the command sgadmin.sh -cd ..\sagconfig –nhnv –icl –cacert ..\..\..\config\root-ca.pem -cert ..\..\..\config\Dinesh-client.pem -key ..\..\..\config\Dinesh-client.key -keypass <your certificate password> -p 9340 . A log "Done with success" will show up
- Now shutdown and restart the InternalDataStore. The InternalDataStore will now use the generated certificates for SSL communication
Additional Steps while using user generated certificates
- Using a key tool import generated pem certificate "root-ca.pem" into truststore.
- Steps for generating a empty truststore:
- keytool -genkey -keyalg RSA -alias endeca -keystore truststore.jks
- keytool -delete -alias endeca -keystore truststore.jks
- Command for importing pem file into truststore
- keytool -import -v -trustcacerts -alias endeca-ca -file root-ca.pem -keystore truststore.jks
After which, use the generated truststore in step 3.
Search Guard may not be initialized. Initialize Search Guard Using sgadmin tool
|Make sure that Search Guard is initialized properly using the steps given above|
ES-Log contains entries like the following:
[2020-01-01T12:00:00,097] [INFO] [c.f.s.c.ConfigurationRepository] Search Guard License Type: TRIAL, valid
[2020-01-01T12:00:00,097] [WARN] [c.f.s.c.ConfigurationRepository] Your Search Guard license expires in 17 days.
The Community Edition does not require a license. To use it, just install the Enterprise Edition and disable all commercial features by adding the following line to
Setting this flag will disable any commercial module or custom authentication domain and will run only the free Community features of Search Guard.
Error in ElasticSearch:
It means Kibana or API Gateway is still communicating with Elasticsearch in HTTP.
Error while running sgtlstool.bat or enable_ssl.bat
The system cannot find the path specified.
It means the JAVA_HOME was not set.
configuring JAVA_HOME will resolve the issue.
- Refer https://docs.search-guard.com/latest/index.html for Search Guard documentation
- Refer https://docs.search-guard.com/latest/search-guard-community-edition for information on disabling all commercial features
- Refer https://docs.search-guard.com/latest/offline-tls-tool#tls-tool for detailed understanding of Search Guard offline TLS tool
- Refer https://github.com/floragunncom/search-guard/wiki for Searchh Guard and Elasticsearch support matrix
- Certificates.yml file for Search Guard certificates configuration
- search-guard-tlstool-1.5.zip, Search Guard offline TLS tool
- Refer http://techcommunity.softwareag.com/pwiki/-/wiki/Main/Tips+and+Tricks+of+API+Gateway for a detail on setting up Elasticsearch cluster