API Governance for REST Services

1 Introduction

In the beginning of the wide usage of Web Services, the most used technique was SOAP. SOAP services were described by WSDLs then. A more recent means is REST ( = Representational State Transfer). REST is an architectural style following the HTTP protocol in that it regards services as requests towards resources using the well-known methods GET, POST, PUT and DELETE upon these resources.
In the following example we assume that the access on a storage of tools is modelled using a REST service. For the description of the service we use Swagger/Json. We will utilize CentraSite and API Gateway, both in version 10.5 as released in October, 2019.



The service for the storage consumer offers two methods, namely one for asking, which tools are available and another one that takes out a tool. In our example, the latter method only returns a message saying that a tool has been taken and how many tools of this kind are still left. In reality, using means of IoT (Internet of Things) this could just as well open a locker or do something similar to actually make the tool available.
Looking at the Swagger file describing the service's interface we see that the first method that informs about the tools currently available needs no input whereas the second method wants to know which tool to hand over.
The actual Storage service was created as a Dynamic Web Project in Eclipse and was deployed under Tomcat. Thus it is already callable, e.g. using Postman.

2 CentraSite

Allthough the service, as said previously, is already there and usable, companies typically do not offer their native services as such, but use some kind of portal to bundle their combined service's at one place for better control and observation. Thus we'll publish the Storage service on a hypothetical company's API Gateway and later on demonstrate how this enables to get a clear picture of the service's functioning.
As a first step the service needs to be introduced to CentraSite. On this behalf, open the BusinessUI and select 'Create Asset' from the main menu. Choose 'REST Service' as the type for the asset to-be. This changes the input screen in a way that allows the input of a file describing the new REST service. The default format is 'Swagger-2.0' which is what we want. The radio button needs to be 'File' and the 'Browse' button enables to navigate to our prepared Swagger file. Click 'Next' and subsequently 'Save' to create the asset. If not otherwise specified the name of the new asset is taken from the Swagger file.



To enable the service's deployment CentraSite needs to be introduced to a suitable API Gateway installation. Assuming an API Gateway is installed somewhere, making CentraSite recognize it is done via the BusinessUI's main menu entry 'Governance Rules'. In the 'Governance Rules' panel click the 'Add Gateway' Icon. Provide a name for the new CentraSite asset representing the API Gateway installation and in the gateway selection box choose 'API Gateway' (default is 'API Portal'), and provide the necessary fields. Make sure that the 'CentraSite Endpoint' field contains an address that later allows API Gateway to send runtime data to CentraSite. This means it should not use 'localhost' unless API Gateway is installed on the same machine. Since this field is read-only it might require a restart of the BusinessUI. Provide credentials that API Gateway will use for accessing CentraSite and - below - the URL of the API Gateway installation in the form 'http://<host>:<port>' wherein <port> is the port of the Integration Server (default here is 5555). Further down, provide credentials that CentraSite will use to access API Gateway. 'API Gateway WebApp URL' and 'Sandbox' are not required. Press 'Publish'.



Having successfully published the API Gateway from CentraSite enables a service to be deployed there. Before doing so the service must be virtualized. This creates another asset (of Type 'Virtual REST Service') that represents the stub that then resides on API Gateway. To virtualize a service go to the service's detail view and press the 'Virtualize' icon. This, among others, lists the service's methods that will also be present in the stub. We choose both methods (the default). The name of the virtual service is that of the native service if not otherwise specified.
The virtualized service can now be published to the API Gateway by pressing the 'Publish' button in the virtual service's detail view.

3 API Gateway

Having published the service to API Gateway makes it visible in the API Gateway's 'Manage APIs' screen. When opening the API Gateway's welcome page go there pressing 'APIs' in the main menu. Pressing the service's name next to the cloud icon opens the detail view for the service.


The service's detail view offers many aspects to look at the service, 'API details', 'Scopes', etc. Choose 'Policies'. This shows a screen exhibiting a nice picture in the middle illustrating the process when the service as deployed on API Gateway is called. On the bottom of the picture (below the Routing box) arrows indicate where the native service (as deployed on Tomcat) is referred to. Per default, the 'Transport' policy's tab is selected left of the picture and the 'Enable HTTP / HTTPS' Transport policy is active.



Click 'Edit' in the right upper corner to change something about the policies. Open the 'Traffic Monitoring' selection on the left and select 'Log Invocation'. This includes a respective icon in the picture and also opens the default settings w.r.t. this policy right of the picture. We stick to the Log Generation Frequency 'Always' as suggested, but add CentraSite to the Destinations. The default here is API Gateway only. We keep that also.
Having changed the policy settings as such, we activate these by clicking 'Save' in the right upper corner.
For sending events to CentraSite, API Gateway utilizes SNMP (Simple Network Management Protocol). If CentraSite and API Gateway are running on different machines, make sure that the firewall allows this, it is per default set to port 8181.


4 Calling the Service

We will now call the service as deployed on API Gateway, i.e. Postman can be used as before but with a different host (the machine the Integration Server runs on) and port (the Integration Server port, most likely 5555).
The result of a service call is no different from a call to the native service, but the call should be listed in API Gateway's storage (i.e. Elasticsearch) as well as in CentraSite.

In CentraSite the acquired runtime data can be made visible by different means. The easiest option is to select the 'Runtime Events' tab in the Virtual REST Service's detail view.

Also, runtime data can be displayed by some of CentraSite's predefined reports. Select 'Reports' in the BusinessUI main menu and View, for example, the report 'Run-time Asset Usage'. The displayed report shows that the service has been called three times yet by unknown consumers.


 

5 Introducing Consumers

In addition to noting that a certain service was called, it can also be listed by whom the call was issued. To this respect, assets of type 'Application' must be created and listed as consumers of a service. Furthermore one needs to specify, how the identification of the respective consumer is achieved.
As potential service consumers (rather tool consumers) we create six assets of type 'Application' namely 'Carpentry', 'Locksmithery', 'Mechanical Engineering', 'Tailoring', 'Weaving Mill' and 'Spinning Mill'.
Three steps are necessary to be able to assign a certain call to a consumer:
First, we need to tell API Gateway how to identify the caller. The method we'll try for this respect is identification by the caller's IP-address. To add the necessary information to the Application asset open the detail view and click on the pencil icon to edit. On the left choose 'Identification' and enter the IP-address of the machine you will call the service proxies from for both 'From IP-V4 Address' and 'To IP-V4 Address', or for the IP-V6 fields depending on the machine that calls. Issue 'ipconfig' in a Command Prompt for information on machine data.
Second, the Applications must be registered as service consumers. Go to the virtual service's detail view and click on the 'Consume' icon (the basket). Add the Applications one by one. Both, registering a consumer or changes to already registered consumers require a service republish to become effective.
In the end the API Gateway must define a policy to be on the lookout for a service's caller. In API Gateway go to the detail view of the service and there select the 'Policies' tab. Again, click 'Edit' to add a policy and then select 'Identify & Authorize Application' below 'Identify & Access' on the left. For the policy properties as displayed on the right, check the 'Allow anonymous' box and select 'IP Address Range' as Identification Type. Select 'Global applications' as Application Lookup condition.
Now, calling the service as deployed on API Gateway from machines corresponding to the Application assets, IP-V4 addresses no longer have the consumer 'Unknown', but a more meaningful entry, check with the report 'Run-time Asset Usage'.

6 Using Taxonomies

As we have seen previously, CentraSite can note, and later on inform, who has called which service. But CentraSite offers further means to classify and prepare the information thus obtained. To illustrate that we will use a Taxonomy to divide the potential consumers into categories and use CentraSite's ability to build custom reports to produce a listing of service calls per category.

The six consumers defined so far are now separated into the different crafts 'hardware' and 'textile'. To create a respective taxonomy we go to 'Taxonomies' in the CentraSite BusinessUI main menu and click the 'Add Taxonomy' icon. Enter the taxonomy's name 'craft', press 'Save' and subsequently add the two categories.


To classify applications with one of these categories we need to add a classification-typed attribute to the asset type 'Application'. Go to 'Asset Types' in the main menu and select 'Application'. Click the 'Add Attribute' icon and add the attribute 'craft'. To make the new attribute visible in the detail views of this type, also create a new profile named 'Classification'. Use drag & drop to add the attribute to the profile. Save the changes to the asset type.
Now, when an asset of type Application is displayed a third profile (besides 'Basic Information' and 'Identification') is offered. click 'Edit' to select a category.Having thus classified the consumers, we can now write an XQuery to get all crafts, for each craft get all consumers classified with this craft, and for every consumer, the number of calls issued to the storage service.

7 A Custom Report


The following XQuery will be the basis for the custom report. The Designer's reporting perspective we will later use to build the report is not a suitable place for developing queries. This is best done using specific XQuery interfaces among others available with webMethods Tamino XTools.

declare namespace cs = "http://namespaces.CentraSite.com/Schema/jaxr";
declare namespace sc = 'http://namespaces.CentraSite.com/Schema'
declare namespace re = 'http://namespaces.CentraSite.com/Schema/SOALink'
let $craftKey := collection('CentraSite')/cs:classificationScheme[cs:name/cs:localString="craft"]/cs:key
let $concepts := collection('CentraSite')/cs:concept[cs:parent=$craftKey]
let $consumerAttKey := "uddi_a7476ff0-a108-11dd-9c38-d8fd010529cc"
for $concept in $concepts
return <concept>
         <name>{string($concept/cs:name/cs:localString)}</name>
         <consumers> {
           let $conceptKey := $concept/cs:key
           let $consumers := collection('CentraSite')/sc:ConsumerApplication
                              [cs:classifications/cs:classification/cs:concept = $conceptKey]
           for $consumer in $consumers
           let $name := string($consumer/cs:name/cs:localString)
           return <consumer>
                    <name>{$name}</name>
                    <calls>{count(for $event in collection('RuntimeEvents')/re:events
                                  where $event/re:operationName="/fetchTool"
                                  return $event/re:eventDetails[re:attributeKey = $consumerAttKey  
                                                                    and re:attributeValue = $name])}</calls>
                  </consumer> }
         </consumers>
       </concept>

The query finds all craft categories which are those assets of type 'Concept' that are direct children of the 'craft' taxonomy. For each craft it creates an XML element having as first child the craft's name and as second child the list of consumers which are classified with this craft. For each consumer an XML element is created that contains the consumer's name and the number of calls by this consumer. We only count the calls towards the fetch method.

The attributes describing a call are organized in key-value-pairs. We use the pair denoting the consumer name here. Other details could be obtained, for example, the call's parameters, i.e. the actual tool fetched.
How to create custom reports is described elsewhere thus we'll just shortly list the steps to be undertaken to this respect:

  1. Install the Designer plus the CentraSite Reporting plug-in.
  2. Start the Designer and connect to CentraSite.
  3. Create a BIRT Reporting Project and go to the Report Design perspective.
  4. Create a report.
  5. Create a Data Source of type 'CentraSite XQuery'.
  6. Create a Data Set with the above query, table element = consumer, for the columns see the preview.
  1. Create a table in the report and fill with data from the Data Set.

The final picture shows a possible report using BIRT's aggregation mechanics, among others.


 

image.png

image.png

image.png

image.png

image.png