Cluster tool usage in tcdb

CLUSTER TOOL USAGE IN TCDB (SSL CONFIGURED):

The cluster tool is a command-line utility that allows administrators of the Terracotta

Server Array to perform a variety of cluster management tasks. You require this tool in order to install the license file on the server and make it usable, regardless of whether your server configuration is single-node or multi-node.

The cluster tool can be used to:

  • Configure OR reconfigure a Cluster
  • Get status of a running server
  • Dump the state of running server
  • Stop the running server
  • Take backups of running server

Location:

Cluster tool can be found at following path:

<InstallationDir>\TerracottaDB\tools\cluster-tool\bin - cluster-tool.bat

Configure:

The configure command creates a cluster from Terracotta stripes details provided within the tc-config.xml, taking license key as an input.

No functionality is available on the server until a valid license is installed

Following is the syntax for configure cluster using ClusterTool.

  • clustertool.bat configure –n <clusterName> -l <fully_qualified_license_file_path> <fully_qualified_tc_config_file_path>

         OR

  • clustertool.bat configure –n <clusterName> -l <fully_qualified_license_file_path> -s <tcdb_server_host>:<tcdb_server_port>

-n: specify the name of the cluster you want to configure.

-l: specify the license file path

-s: specify the server host:port details, default port is optional

ReConfigure:

Is explained below by an example of configuring SSL on non SSL configured host.

SSL/TLS Security Configuration in Terracotta:

1. Keystore/Truststore - To provide a safe communication across the cluster by providing two-way verification of the identities of the two hosts involved in the communication

2. IP whitelist file - It lets only the known clients to access the TSA.

Usage flow:

Create keystore and trust store files and copy to Secure root directory OR SRD.

SRD will have two folder:

  1. Identity: should have Keystore file with .jks extension
  2. Trusted-authority: should have truststore file with .jks extension

Note: There should not be any files with .cert extension to be copied to SRD.

Update the following snippet in tc-config with SRD path as follows:

<security:security>

        <security:security-root-directory>path/to/security-root-directory</security:security-root-directory>

        <security:ssl-tls/>

        <security:whitelist/>

      </security:security>

</service>

So after the SSL is configured and you want to update the cluster with this new configuration, without hampering the existing data. You should be following simple steps below to update the configuration:

1. Initially you need to run the Re-configure command, to let the cluster know about the new configuration that is added in tc-config.xml as follows:

  • clustertool.bat reconfigure –n <clusterName> -l <fully_qualified_license_file_path> <fully_qualified_tc_config_file_path>

         OR

  • clustertool.bat reconfigure –n <clusterName> -l <fully_qualified_license_file_path> -s <tcdb_server_host>:<tcdb_server_port>

-n: specify the name of the cluster you want to configure.

-l: specify the license file path

-s: specify the server host:port details, default port is optional

You should be seeing the following output from the command:

License not updated (Reason: Identical to previously installed license)

Configuration successful

2. Restart the server and server will start with SSL mode

3. Now run the cluster-tool with -srd option to let cluster configuration know that server is now configured with in SSL.

  • clustertool.bat –srd <full_qualified_SRD_file_path> configure –n <clusterName> -l <fully_qualified_license_file_path> <fully_qualified_tc_config_file_path>

         OR

  • clustertool.bat –srd <full_qualified_SRD_file_path> configure –n <clusterName> -l <fully_qualified_license_file_path> -s <tcdb_server_host>:<tcdb_server_port>

-srd: specified the secure root directory

-n: specify the name of the cluster you want to configure.

-l: specify the license file path

-s: specify the server host:port details, default port is optional

Backup:

This command will take backup of running server in TCDB cluster.

Note that,

  1. Basic installation will not have backup plugin configured in tc-config.
  2. First we should update the tc-config with backup plugin details.

Following is the plugin to update:

            <service>

            <backup-restore xmlns="http://www.terracottatech.com/config/backup-restore">

            <backup-location path="/path/to/backup/dir" />

            </backup-restore>
            </service>

  1. Update the cluster to know about the update in tc-config.xml using cluster tool reconfigure command.
  • clustertool.bat reconfigure –n <clusterName> -l <fully_qualified_license_file_path> <fully_qualified_tc_config_file_path>

         OR

  •  clustertool.bat reconfigure –n <clusterName> -l <fully_qualified_license_file_path> -s <tcdb_server_host>:<tcdb_server_port>

-n: specify the name of the cluster you want to configure.

-l: specify the license file path

-s: specify the server host:port details, default port is optional

License not updated (Reason: Identical to previously installed license)

Configuration successful

  1. Restart the TCDB server to let cluster know about backup details updated.
  • clustertool.bat backup –n <clusterName> -l <fully_qualified_license_file_path> <fully_qualified_tc_config_file_path>

         OR

  • clustertool.bat backup –n <clusterName> -l <fully_qualified_license_file_path> -s <tcdb_server_host>:<tcdb_server_port>

-n: specify the name of the cluster you want to configure.

-l: specify the license file path

-s: specify the server host:port details, default port is optional

Status:

This command will provide you the status of running TCDB server OR servers in a single stripe OR multiple stripes.

Following is the command to check the status

  • clustertool.bat status –n <clusterName> > -s <tcdb_server_host>:<tcdb_server_port>  -s <tcdb_server_host1>:<tcdb_server_port1> ….

         OR

  • clustertool.bat status –n <clusterName> -s <tcdb_server_host>:<tcdb_server_port>

-n: specify the name of the cluster you want to configure.

-s: specify the server host:port details, default port is optional

Dump:

This command will dump the status of running TCDB server. Dump can be seen in the logs.

Following is the command to check the status

  • clustertool.bat dump –n <clusterName> > -s <tcdb_server_host>:<tcdb_server_port>  -s <tcdb_server_host1>:<tcdb_server_port1> ….

        OR

  • clustertool.bat dump –n <clusterName> -s <tcdb_server_host>:<tcdb_server_port>

-n: specify the name of the cluster you want to configure.

-s: specify the server host:port details, default port is optional

Stop:

The stop command stops the cluster, or particular server or servers in the same or different

clusters.

  • clustertool.bat stop –n <clusterName> > -s <tcdb_server_host>:<tcdb_server_port> -s <tcdb_server_host1>:<tcdb_server_port1> ….

         OR

  • clustertool.bat stop –n <clusterName> -s <tcdb_server_host>:<tcdb_server_port>

-n: specify the name of the cluster you want to configure.

-s: specify the server host:port details, default port is optional

TroubleShooting guide:

==> com.terracottatech.security.exception.SecurityConfigurationException: Did not expect <fileName>.cer in security root directory <securityRootDirectory>

Diagnosis:

10.3 onwards there are more validations added to TCDB and .cer file is not supported in SRD path, as per the documentation only .jks file is supported

Remove .cer file and keep only keystore and trust store alias with .jks file and run the cluster-tool again.

==> Error (FAILURE): org.terracotta.connection.ConnectionException: com.terracottatech.connection.ProbableSecurityConfigurationException: Handshake with server failed when this client tried to initiate a secure connection. Possible reasons: client security configuration is not valid, or server is not running with security enabled. Check client/server logs for more information.

Diagnosis:

1. Either your client security configurations are not valid

2. or Server is not running with security enabled.

3. You are running cluster tool without reconfiguring the cluster, you need to run the reconfigure first to let cluster gets updated with new changes made in tc-config file.

Later restart the server to start with security enabled and run the cluster-tool again to let cluster know about it.

==> java.lang.RuntimeException: com.terracottatech.security.exception.SecurityConfigurationException: Expected whitelist.txt file in the security root directory

Diagnosis:

1. updated tc-config with SRD service, ran the cluster tool

2. In tc-config along with srd service there will be an option of <security:whitelist/>.

3. you have whitelist.txt (file name must be whitelist.txt) in srd, in order to let server know which clients can access the TSA

4. Then restart server

==> com.terracottatech.security.exception.SecurityConfigurationException: Did not expect whitelist.txt file in the security root directory

Diagnosis:

1. When Server and client is in different machines and Server is runnig with SSL

3. You might copied the entire SRD into client machine as well.

4. But client does not need whitelist.txt. This file is only for Server access to identify the trusted cliens, hence the error is thrown.

5. Remove whitelist.tx from srd of a client machine and try connecting to server.