Avoiding single point of failure for Elasticsearch in Kibana

Knowledge base
, ,
1

Author: Vaidyanathan, Praveen
Supported Versions: 9.12 and above

Introduction

In a typical API Gateway deployment setup, Kibana can talk to only one Elasticsearch (IDS - Internal Data Store) at any point of time. Depending on your deployment topology, this could cause single point of failure. This document covers few variants which you can use depending on your tolerance levels. Note : Refer here ro run Kibana and IDS in dedicated machines.

Using IDS as co-ordination node

1046×467 11.5 KB

In this approach, instead of talking to Elasticsearch which serves data, kibana talks to a co-ordination node. Coordination node is an IDS instance that co-ordinates only the search request and load balances the requests. The co-ordination node will have the following attributes set to false in it's elasticsearch.yml

elasticsearch.yml

node.master: false
node.data: false
node.ingest: false

Typically co-originating IDS runs in the same node as Kibana. This setup is not totally avoiding the single point of failure through IDS alone crashing is rare. "apigw.es.url" should be changed in uiconfiguration.properties to point to coordinating IDS. Refer the next section to avoid the single point of failure completely.

Using IDS behind load balancer

1046×467 12.8 KB

In this approach, Kibana will send the requests to LB and the LB routes the requests to one of the IDS. This configuration provides cover from single point of failure for IDS. "apigw.es.url" should be changed in uiconfiguration.properties to point to LB. Refer the next section to avoid the single point of failure for both kibana and IDS completely.

Using IDS and Kibana behind load balancer

1046×467 15 KB

In this approach, API gateway sends the request to LB and LB routes the request to one of the available Kibanas. Kibana will send the requests to LB and the LB routes the requests to one of the IDS. This configuration provides cover from single point of failure for IDS and Kibana. "apigw.es.url" should be changed in <Root>\profiles\IS_<Instance_Name>\apigateway\config\uiconfiguration.properties to point to LB. "dashboardInstance" property in <Root>\IntegrationServer\instances\<Instance_Name>\packages\WmAPIGateway\config\resources\beans\gateway-core.xml should be changed to LB.