products versions - {webMethods Integration 11.0 or above}
Introduction
Develop Anywhere, Deploy Anywhere introduces a new unified integration platform for webMethods and builds on the strengths of our existing products.
In this article, I will explain how you can build a readymade image for running your edge runtime container. This avoids the requirement for the edge runtime to synchronize each time it starts up and is our recommended practice for production environments.
Pre-requisites
- webMethods.io tenant that has these new features enabled,
- wpm.zip cli tools to allow you to generate the required Dockerfiles(s).
- GitHub account to allow you to pull packages without rate limiting.
- Empower account that allows you to download Software AG licensed packages.
Steps to follow
As part of the series of articles on Develop anywhere, Deploy anywhere I previously wrote about how you can build an integration based on webMethods Packages (Develop anywhere - A practical guide to using packages with webMethods.io). Now we will look at how you can deploy this integration into your production environment, without the container having to sync the packages at startup.
Syncing is a powerful feature to speed development and testing but is not necessarily a good fit in a production environment where you want to take advantage of rapid auto-scaling. Generally, it is considered good practice for containers to be based on pre-baked images, thus ensuring consistent behavior when scaling containers or restarting them.
In the following sections, we will outline how you can create an image for your edge runtime using the wam/wpm cli tools provided by Software AG.
Configure wam/wpm for local usage
Download wam/wpm from here - wpm.zip, and unzip to your local file system. Once done edit the PATH attribute in your environment to include the bin folder so that you can run the scripts from wherever you require.
$ setenv PATH=$PATH/<WPM_HOME>/bin
Check that either wam or wpm is executable e.g.
$ wam -h
webMethods Application Manager (wam)
version 0.0.3
Use this tool to generate a Dockerfile to build a webMethods runtime image from a project scaffold(s) or edge manifest file
usage: wam
-b,--base-image <arg> Base image, defaults to sagcr.azurecr.io/webmethods-edge-runtime:latest
-d,--dockerfile-name <arg> Name of Dockerfile to generate, defaults to Dockerfile
-e,--edge-manifest <arg> Edge manifest file to read
-h,--help Help information
-m,--edge-manifest-name <arg> Write an intermediate edge manifest with the given name instead of a Dockerfile
-p,--projects <arg> Project scaffold file or directory containing project scaffolds
-s,--service <arg> The project service to reference, if omitted all services will be included
-t,--target-dir <arg> Target directory for installation, defaults to /opt/softwareag/IntegrationServer
-v,--version Get the version number of this cli tool
You may need to set the execute permission if the above command fails or check the JAVA_HOME directory in the setenv.sh script, found in the …/bin folder.
Download the project scaffold for your webMethods.io project
In the article above, Develop anywhere - A practical guide to using packages with webMethods.io we explained how you can externalize the source code for your webMethods.io project, this also includes the scaffold file. This file describes all of the assets and configuration required for your project to function and it is the basis for generating the necessary edge runtime images.
You will need to use an external git repo and clone it to your local computer so that you can extract this file e.g.
$ git clone https://github.com/johnpcarter/EdgeTechCommunityDemo.git
Once cloned you can find the file in the ../config/scaffolding folder e.g.
Getting the required git/wpm tokens
In order to build an image the generated docker file will reference the wpm cli that is baked into our base image, which in turn will require the necessary tokens to be able to either fetch packages via the indicated package registry or git server.
Scroll down to the section packages and replace the gitUsername and gitToken entries with your own.
...
packages:
- name: EdgeJcHelloWorld
gitServerName: johnpcarter
gitUrl: https://github.com/johnpcarter/EdgeJcHelloWorld
gitUsername: <YOUR GITHUB ID>
gitToken: <YOUR GITHUB DEVELOPER ACCESS TOKEN>
gitBranch: main
- name: WmJDBCAdapter
wpmServer: https://packages.softwareag.com
wpmRegistry: softwareag
wpmToken: edge.scaffolding.WmJDBCAdapter.https://packages.softwareag.com.wpmToken
- name: JcPublicTools
gitServerName: johnpcarter
gitUrl: https://github.com/johnpcarter
gitUsername: <YOUR GITHUB ID>
gitToken: <YOUR GITHUB DEVELOPER ACCESS TOKEN>
gitTag: v2.1.0
You can generate a developer access token for git via your GitHub settings after logging in. wpm will require that you login into https://packages.softwareag.com with your empower credentials, and then generate an access token via the settings menu item.
Generate an image for the entire project
Now we can generate a Dockerfile to build our image, simply cd into the directory and run the following command
$ cd EdgeJcHelloWorld/config/scaffolding
$ wam -p EdgeJcHelloWorld.yml
wam -p EdgeJcHelloWorld.yml
webMethods Application Manager (wam)
version 0.0.3
extracting all services assets from project scaffold /Users/jcart/Documents/Work/wpm/EdgeJcHelloWorld/config/scaffolding/EdgeJcHelloWorld.yml
Generating docker file Dockerfile
Generating application.properties file
As you can see it has generated a Dockerfile and an application.properties file. If you open the Dockerfile you will see something like the following (albeit I have removed the tokens that you will have included above).
FROM sagcr.azurecr.io/webmethods-edge-runtime:latest
WORKDIR /opt/softwareag/wpm
RUN /opt/softwareag/wpm/bin/wpm.sh install -ws https://packages.softwareag.com -wr softwareag -j eyJhb... -d /opt/softwareag/IntegrationServer WmJDBCAdapter
RUN /opt/softwareag/wpm/bin/wpm.sh install -u johnpcarter -p ghp_... -r https://github.com/johnpcarter -d /opt/softwareag/IntegrationServer EdgeJcHelloWorld
RUN /opt/softwareag/wpm/bin/wpm.sh install -u johnpcarter -p ghp_... -r https://github.com/johnpcarter -d /opt/softwareag/IntegrationServer JcPublicTools
WORKDIR /
COPY application.properties /opt/softwareag/IntegrationServer/application.properties
You can see that it is leveraging the baked-in wpm command to ensure that packages can be easily pulled into the image.
The baked in version of wpm has a problem loading protected packages and so we will add the version that you downloaded as part of this article. Edit the generated Dockerfile and add the following lines just after the FROM command. Make sure that you have a copy of the wpm folder in the same folder as your Dockerfile, otherwise the ADD will fail. This addition will not be required once the 11.0.1 is released.
ADD --chown=sagadmin:sagadmin wpm /opt/softwareag/wpm
ENV PATH=/opt/softwareag/wpm/bin:$PATH
Using e2e tracing
If you want to trace the execution of services running in your edge runtime in our end 2 end monitoring solution then add the following lines to your application.properties file before building.
settings.watt.server.audit.service.include=$toplevel,edge.*,project.*
settings.watt.server.audit.service.exclude=$wmservices
Build your image
All that is left is to run the docker build command e.g.
$ docker build -t my-runtime-helloworld:latest .
[+] Building 0.4s (11/11) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.32kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for sagcr.azurecr.io/webmethods-edge-runtime:10.16.5 0.0s
=> [1/7] FROM sagcr.azurecr.io/webmethods-edge-runtime:10.16.5 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 3.70kB 0.0s
=> CACHED [2/7] WORKDIR /opt/softwareag/wpm 0.0s
=> CACHED [3/7] RUN /opt/softwareag/wpm/bin/wpm.sh install -u johnpcarter -p ghp_... -r https://github.com/johnpcarter 0.0s
=> CACHED [4/7] RUN /opt/softwareag/wpm/bin/wpm.sh install -ws https://packages.softwareag.com -wr softwareag -j eyJhb... 0.0s
=> CACHED [5/7] RUN /opt/softwareag/wpm/bin/wpm.sh install -u johnpcarter -p ghp_... -r https://github.com/johnpcarter 0.0s
=> [6/7] COPY application.properties /opt/softwareag/IntegrationServer/application.properties 0.2s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:d1046c96c7c818764dc25fbf846b4ec52d014d41087aae3899201044ac91036d 0.0s
=> => naming to docker.io/library/my-runtime-helloworld:latest
Spinning up the container
Starting the container is identical to the pairing process as as shown when registering or starting the container from the control plane in webMethods.io, you only need to replace the image with that of your own above, and with one other small change.
docker run -p 5555:5555 -d -e SAG_IS_CLOUD_ALLOWSYNC=false -e SAG_IS_CLOUD_REGISTER_URL=https://wmintanywhere.int-aw-au.webmethods.io -e SAG_IS_EDGE_CLOUD_ALIAS=EdgeRuntime_JC_MAC -e SAG_IS_CLOUD_REGISTER_TOKEN=... --name=JC_MAC_1bd35694 my-runtime-helloworld:latest
You will perhaps notice that I have also added an extra environment variable
SAG_IS_CLOUD_ALLOWSYNC=false, this is to ensure that the container does attempt or accept any updates to its packages. The whole point of basing our runtime on a prebuilt image is to avoid the container getting updated so this is an important change to make.
Slicing up your project into separate edge runtimes and merging projects.
In some circumstances you may only want to partially deploy your project to a runtime or even deploy several project fragments to the same runtime, wam allows you to do both;
To generate a fragment simply specify the name of the project service that you plan to deploy to the runtime from the project using the -s --service switch. In which case it will only deploy the dependent packages as required by the service along with the project package. It will also ensure that only connections that are required by that service or dependent services will be activated, all other connections will be disabled by default.
You can also combine projects by simply running the map command several times, whilst using the option -m --edge-manifest-name to generate an edge manifest file rather than directly a Dockerfile. In this case, it will update the existing file if it already exists with the new assets. The command will generate warnings if it detects conflicts when merging projects such as conflicting package versions.
Next steps
The above startup is yet again a very manual step and would not be a valid scenario for spinning up your container in a production environment. In my next article, I will show you how you now take advantage of your pre-built image to configure a Kubernetes deployment to auto-deploy and scale a number of pods for your project.
Useful links | Relevant resources
Develop anywhere - A practical guide to using packages with webMethods.io
Building an integration microservice from A to Z with webMethods and Kubernetes