Extras Steps on Configuring API Gateway 10.2 and above on Linux RHEL for accessing an External Elasticsearch CLUSTER

Knowledge base
, , , ,
1

Author: Fabien Sanglier (@SoftwareAG)
Supported Versions: 10.2 & above

webMethods API Gateway tutorial

The rationale for connecting API Gateway to an external Elastic Search was well explained in Configuring External Elasticsearch for API Gateway 10.2

BUT as I'm setting up a Highly-Available API Gateway cluster consisting of 2 API gateway nodes connected to an external 3-node Elastic Search cluster (EventDataStore) on Linux RHEL, I had to figure out a couple of extra steps not quite outlined in the previous tutorial.

1. Changing API Gateway configurations to connect to CLUSTERED Elasticsearch

Edit file {{SAG_HOME}}/IntegrationServer/instances/{{APIGW_INSTANCE_NAME}}/packages/WmAPIGateway/config/resources/elasticsearch/config.properties

And update the following values with the following clustered elastic search format (instead of a single host)

pg.gateway.elasticsearch.autostart=false
pg.gateway.elasticsearch.hosts={{ES_CLUSTER_HOST1}}:9240,{{ES_CLUSTER_HOST2}}:9240,{{ES_CLUSTER_HOST3}}:9240

 

2. Kibana to connect to CLUSTERED Elasticsearch

Edit file {{SAG_HOME}}/profiles/IS_{{APIGW_INSTANCE_NAME}}/apigateway/dashboard/config/kibana.yml

And update the following values with the following clustered elastic search format (instead of a single host)

  elasticsearch.hosts: ["http://{{ES_CLUSTER_HOST1}}:9240" , "http://{{ES_CLUSTER_HOST2}}:9240" , "http://{{ES_CLUSTER_HOST3}}:9240"]

And as explained in the tutorial "Configuring External Elasticsearch for API Gateway 10.2", make sure that kibana "autostart" is set to false.

To do so, Edit file {{SAG_HOME}}/profiles/IS_{{APIGW_INSTANCE_NAME}}/apigateway/config/uiconfiguration.properties

And update the following value:

apigw.kibana.autostart=false

 

3. Kibana to auto-start as a service using Systemd

Since Kibana can't be started automatically by API gateway per above steps, the tutorial calls for starting it using the "kibana" executable in:

{{SAG_HOME}}/profiles/IS_{{APIGW_INSTANCE_NAME}}/apigateway/dashboard/bin/kibana

This is fine for testing, but in real operational scenario, we will need to make sure the Kibana service gets started automatically when the server starts. This can be achieved using a RHEL SYSTEMD service.

Here are the steps to create it:

  • Log in as root (or use SUDO for all the following tasks)

  • Create a new file in /usr/lib/systemd/system/sag1apigwkibana.service

  • Edit the file with the following (make sure to update the <> tags with your specific values)

[Unit]
Description=APIGW Kibana
After=multi-user.target

[Service]
Type=simple
User={{SAG_USER}}
Group={{SAG_USER}}
WorkingDirectory={{SAG_HOME}}/profiles/IS_{{APIGW_INSTANCE_NAME}}/apigateway/dashboard/bin/
PIDFile={{SAG_HOME}}/profiles/IS_{{APIGW_INSTANCE_NAME}}/apigateway/dashboard/bin/kibana.pid
ExecStart={{SAG_HOME}}/profiles/IS_{{APIGW_INSTANCE_NAME}}/apigateway/dashboard/bin/kibana
ExecReload=/bin/kill -1 -- $MAINPID
ExecStop=/bin/kill -- $MAINPID
TimeoutStopSec=5
KillMode=process

[Install]
WantedBy=multi-user.target
  • Reload the systemd service:
systemctl daemon-reload
  • Enable the service to auto-start
systemctl enable sag1apigwkibana
  • Start the services
systemctl start sag1apigwkibana
  • Verify that kibana process is running:
ps aux | grep /bin/node

should return the following line:

{{SAG_USER}} {{PROCESS_ID}} ... 02:54 0:08 {{SAG_HOME}}/profiles/IS_{{APIGW_INSTANCE_NAME}}/apigateway/dashboard/bin/../node/bin/node ...

And Kibana should work in the UI.

Finally, when rebooting the machine, the Kibana service should be started automatically upon restart.

 

With the above steps, you should have not only a functional API Gateway (clustered or not) connected to an external Elastic Search *CLUSTER* but also a functional Kibana service also connected to that same external Elastic Search *CLUSTER*.
1 Like