Running webmethods products on kubernetes cluster (aws ec2 cloud)

manoj.jannu · June 14, 2018, 7:33am

TABLE OF CONTENTS

1 Overview
2 Prerequisites
3 Create an AWS account
4 Create a user with Administrator policy
5 Installing requirements

5.1 Install kops for AWS
5.2 Install Kubectl
5.3 Install aws CLI

6 Configuring AWS Account in Client Machine
7 Running a IS/UM container in a kubernetes cluster

7.1 Using Private DNS
7.2 Using Public DNS

Overview

This article describes detail steps for setting up a Kubernetes cluster and running webMethods Integration Server/Universal Messaging as a container on AWS cloud

Prerequisites

Minimum Knowledge of Amazon web services (AWS).
A new Linux machine. Preferably CentOS 7.
Hands-on experience on Kubernetes using minikube.
A Docker image for webMthods product IS/UM.

Create an AWS account

Create an aws account with all details like credit /debit card details which will give you free tier access. It will take 24 hours to activate your account. Initially, it will deduct some amount of money and refund it to your account on verification.

Once the account is successfully verified you should be able to see services provided for free tier access.

AWS URL: https://console.aws.amazon.com

To verify the free tier access checks the details from “My billing Dashboard” which will give a summary of free tier usage

During this guide we will use only free tier access but free tier access will not allow us to launch more than one t2.micro instance so we need to request AWS billing team to increase this limitation by raising a support ticket. Please find the response from billing team which normally takes 48Hour to resolve your query.

So we are set to proceed with the next step.

Create a user with Administrator policy

Go to “IAM” page from services and create a user with the following details

Username: kops

Access type: Programmatic access

Attach existing Policies: AdministratorAccess

Keep a note of Access key ID and Secret access key which we will use later to connect to aws account from our local machine as a client.

Installing Requirements (kops, kubectl and aws command line utility)

Login to our client machine in our case a RHEL VM from where we will run the following commands to install kubectl, kops and aws command-line utility

Install kops for AWS

Kops is an official Kubernetes project for managing production-grade Kubernetes clusters. Kops is currently the best tool to deploy Kubernetes clusters to Amazon Web Services.

Download latest Kops

$ curl -LO https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-linux-amd64

Change the permission and copy the binary to /usr/local/bin/kops

$ sudo chmod +x kops-linux-amd64

$ sudo mv kops-linux-amd64 /usr/local/bin/kops

Verify the installation: kops version command will give the version details as below

$kops version

Install Kubectl

Use the Kubernetes command-line tool kubectl, to deploy and manage applications on Kubernetes. Using kubectl, you can inspect cluster resources; create, delete, and update components.

Download the latest release with the command:

$curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

Change the permission and copy the binary to /usr/local/bin/kubectl

$ sudo chmod +x ./kubectl

$ sudo mv ./kubectl /usr/local/bin/kubectl

Verify the installation: following command will give you the kubectl version

$ kubectl version

Install aws CLI

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services.

Install aws command line utility by installing its requirement.

$sudo yum install epel-release -y

$sudo yum install python-pip-y

$pip install awscli --upgrade –-user

$export PATH=~/.local/bin:$PATH

Verify aws command line utility by executing aws command. It should give you a helpful guide

Configuring AWS Account in Client Machine

Let’s create an ssh key which will be used to login to cluster

$mkdir .ssh

$ssh-keygen -f .ssh/id_rsa

You can provide any passphrase but need not to remember it.

Login to AWS from command-line interface using Access key ID and Secret access key generated while creating a user in aws. Provide AWS region that you want the Kubernetes cluster to be installed. We will keep the default values for region and output format

$aws configure

Now we are ready to execute AWS commands from command line interface

Running a IS /UM container in a Kubernetes cluster

We need following requirements to run a cluster on aws

IS/UM image in the repository: As an example we will use IS/UM image in this article. I have placed UM/IS Docker image in Docker hub as a public image.

You can follow the guide Installing Docker and running Integration Server as a container

And Building a Docker image for webMethods products to build a Docker image for webMethods products

S3 bucket: kops will store the cluster state in this persistent location. You can compare this as a google drive storage area.

Let’s create an s3 bucket in aws in us-east-1 region.

$aws s3api create-bucket --bucket kubernetes-kops-state-store --region us-east-1

Verify the bucket from aws UI. You can see a s3 bucket has been created with the name ‘kubernetes-kops-state-store’

Enable versioning for s3 bucket.

$aws s3api put-bucket-versioning --bucket kubernetes-kops-state-store --versioning-configuration Status=Enabled

Export the environement variable for s3 bucket in .bash_profile file

$export KOPS_STATE_STORE=s3:// kubernetes-kops-state-store

If you have a public host purchased, you can setup and run a cluster using public DNS which will resolve and redirect requests from host to aws cluster using Route53 DNS service. Otherwise for now you can set up and run a cluster using private DNS. Let’s explore both options.

Using Private DNS.

Create a Kubernetes cluster definition using kops by providing the required node count, node size, and AWS zone. Since we are using private DNS, we will use cluster name as ‘tutorial.k8s.local’

$export KOPS_CLUSTER_NAME=tutorial.k8s.local

$kops create cluster --node-count=2 --node-size=t2.micro --zones us-east-1a --name ${KOPS_CLUSTER_NAME}

Create a cluster using the above definition

$kops update cluster --name tutorial.k8s.local --yes

You need to wait for the cluster to be up and running.

Verify it from the AWS UI. You can notice that nodes are in running state also status check should be green.

You can also verify it from the console. It should give a result as ready state

$kops validate cluster

Run a Docker image in the AWS private cloud by pulling it from Docker hub

$kubectl run um --image=sprodocker/umimage:v10.1 --replicas=2 --port=9000

Verify that the deployment is successful and UM container is running in a Kubernetes cluster

You can create a service with the kubectl expose command and its --type=LoadBalancer flag.it will create ELB as an endpoint. This service listens on the port 9000 and redirects it to the target port 9000 (the port on which containers listens).

Create a service for a deployment, which serves on port 9000

$kubectl expose deployment um --port=9000 --type=LoadBalancer

Get the service details with the below command:

$kubectl describe services um

Get the LoadBalancerURL and port and verify it from UI

At the end lets cleanup our cluster and deployment

Verify that our cluster has been deleted .there should note be any instance in running state

Using Public DNS

If you have a public host purchased, you can setup and run a cluster using public DNS. It will resolve and redirect requests from host to aws cluster using Route53 DNS service.

In this tutorial I will use my free domain awstestdomainforfree.tk. You can buy one from any hosting provider or you can register one from Amazon RouteS3 service.

Let’s create a Hosted Zone from AWS UI. Open route53 home page and create a Hosted Zone. Please note I have provided Domain Name which is awstestdomainforfree.tk and type as Public hosted Zone.

Once you have create a hosted zone it will automatically create a collection of record sets for a domain. This will tell DNS how to route traffic on the internet for the domain.

Copy the record sets and update it in nameserver of domain.

Login to your host provider and update the nameserver details. Normally it exist inside ‘Manage Domain >Nameserver details’ page.

Note: In this tutorial I am using my purchased domain so nameservers listed here will not be the same for you.

Now, since we are using public DNS, we will use cluster name as domain name ' awstestdomainforfree.tk ' to create a cluster. Run following sequence of command to create a cluster

$export KOPS_CLUSTER_NAME=awstestdomainforfree.tk

$export KOPS_STATE_STORE=s3:// kubernetes-kops-state-store

$kops create cluster --cloud=aws --zones us-east-1a --state=s3:// kubernetes-kops-state-store --dns-zone=awstestdomainforfree.tk --node-size="t2.micro" --node-count="2" --dns public --name ${KOPS_CLUSTER_NAME}

Create a cluster using the above definition

$kops update cluster --name ${KOPS_CLUSTER_NAME} --yes

It will take some time to bring the cluster in to up and running state.

Run following command to validate the cluster. If you find any error as an output you should try to run the same command after sometime.

$ kops validate cluster

Run a Docker image in the AWS kubernetes cluster by pulling it from Docker hub

$kubectl run um --image=sprodocker/umimage:v10.1 --replicas=2 --port=9000

Verify that the deployment is successful and UM container is running in a Kubernetes cluster.

$kubectl get pods

Create a service using the kubectl expose command .

$ kubectl expose deployment um --type=NodePort

Once this service is up it will be assigned to a port. Get the service details with the below command:

$kubectl describe services um

Notice that service is assigned with a port 31707. To access this service directly without using LoadBalancer we need to open this port in the firewall.

Go to AWS UI and click on VPC>Security> Security Groups> open Inbound Rules of master node.

And add a new inbound rule to open 31707 port in the firewall

Now verify that UM is accessible with Public DNS http://api.awstestdomainforfree.tk:31707