I want to have community experts advice on deployment of flowService using CICD pipeline for continues integration.
Current Setup Details:
We have licensed of version API Gateway and IS 10.11 installed. We don’t have wm deployer or ABE license. Our Test environment is single node and pre-prod environment is clustered(3 Nodes).
Question:
Can we deploy flow service or package from test to pre-prod environment using jenkin or any other automation tool set. We don’t have license for wm deployer or ABE. What are our option for automated deployment of flowService or IS package from test to pre-prod environment.
You can create Flow services using the built in pub.replicator:* and pub.packages:* services in WmPublic and trigger them from Jenkins or other automation tools.
On the source server:
pub.replicator:packageCreation - Creates the zip file
pub.replicator:addReleaseRegistryEntry - Adds an entry to the Package Release Registry
pub.replicator:distributeViaSvcPush - Requires remote server aliases for the target servers
or
pub.replicator:distributeViaFTP - Requires host, port & credentials as inputs
On the target servers:
pub.packages:installPackage
Your CICD script would need to move the updated packages to the autodeploy folder. IS scans the folder every few minutes and loads any new/updated packages
Both this option and Theo’s only cover packages, IS deployments often include some configuration settings that are not included in packages
Configuration is the main issue. What resources do you need to connect to (databases, messaging brokers, …)?
This package created by @jahntech.cj could help for ART resources: https://tech.forums.softwareag.com/t/new-tool-automate-update-of-art-adapter-connections-on-integration-server/294569
Alternatively, you could decide to centralize the configuration in a dedicated package that is managed manually outside the CI/CD process.
Regarding the deployment of packages, the webMethods package manager may work on wM 10.11, it would allow you to deploy your packages from a git remote.
Before diving into an alternative solution, I’d like to point out that neither ABE nor the Deployer are licensed products. Knowing this, do you still want to explore a different solution?
You don’t need Deployer or ABE for CI/CD with pure Integration Server. In fact, in more than 15 years I have never used them for such a scenario.
Can you elaborate on the nature of your cluster? In particular, does it use Terracotta?
The approach I have always followed is conceptually the one that is recommded in the “Continuous Delivery” book and often mentioned in Dave Farley’s videos on the subject.
In a nutshell:
On CI env: Check out from VCS, build, run tests; on success create ZIP, and upload to binary repository (e.g. Artifactory, Nexus)
On TEST env: Download from binary repo, install, run tests. On success, promote in binary repo
On Pre-PROD env: triggered by promotion above, download from binary repo, install, run tests. On success promote in binary repo
In fact I am currently working on an online course for exactly this topic. Your question is a welcomed reason (so thanks! ) to write a small blog post on this.
Let me do the blog post and come back with a link later today.
No, I have always used standard Ant for this. It makes it a lot easier to handle the classpath. The latter is handled such in jcode that it works for standard things. But I often include additional libraries from ./common/lib/** and here Ant is much better. Plus it allows me transparently deal with different files for the same library across versions (think javax to glassfish change).
As to byte code in the VCS, I have always fought this. Up to the point that I would often exclude the java.frag files, since those are derived from the actual Java code.
Since this will be the foundation course, I will cover containers in general, but not K8S/OpenShift.
But I am highly interested to understand what you would like to see covered. We should probably not use this thread for this, though. Let’s take this offline or into a separate thread, whatever you prefer.
Others may have a different view, but you should never deploy at the IS service level (FLOW, Java, adapter service, etc.). That is too fine-grained. Doing so is the rough equivalent to deploying a single method or a class. The smallest deployment “bundle” should be a package. And IMO, should always be a full package, never a patch.
API Gateway is very different from stand-alone IS so the processes will differ in detail.
As a data point to join @jahntech.cj note about using Ant – we use Deployer exclusively for Integration Server (API Gateway is managed manually).
I support this strongly. It is not unheard of that people want to deploy discrete services. However, that is not something that really gains you anything. Yes, it reduces the file size. Nut other than that the increase in complexity is huge and certainly not worth it in my opinion.
With this rather broad and general statetment out of the way, it would be interesting to hear from @mhaseeb.tariq what the rational is behind the wish to deploy discrete Flow services. I am pretty certain we can find a good solution for whatever the requirement is.
Thanks everyone for the response. @jahntech.cj Our requirement is not to deploy single Flow service, that will cause issues. We want to deploy at package level. A complete package from Test to pre-prod.
For test environment we have 1 node and for pre-prod we have 3 node clustered environment we are using Terracotta obviously. For nodes backend you know more then there is elastic and other stuff.
In the admin UI we have the deployer we can deploy packages from there also.
But I want something automated like CI/CD pipeline.
) to write a small blog post on this.