How to get on-prem IS to upload a file into Azure 'Blob' storage

[Nagendra_Prasad]

Personal opinion :
While the growth of dependencies looks ominous , it is perhaps Separation of Concerns ( Separation of concerns - Wikipedia ) in action , but then it works perfectly only if all the moving parts work as expected.
-NP

[Suresh_Ganta]

Hi Sonam, Nagendra,

Did you check and try the webMethods CloudStreams connector for Azure storage? provided with webMethods CloudStreams product @ Microsoft Azure Storage

Is there any particular reason of not trying this approach?

regards,
Suresh P.N.V.S Ganta
Software AG Product Management

[Nagendra_Prasad]

From the first post in this topic, I think perhaps it was a licensing thing?

@Sonam_Chauhan

I heard of some ‘Azure adapter’ for IS, but my site’s not licensed for it (and it may not be available for 10.1)

[Sonam_Chauhan]

@Nagendra_Prasad

Personal opinion :
While the growth of dependencies looks ominous , it is perhaps Separation of Concerns ( Separation of concerns - Wikipedia ) in action , but then it works perfectly only if all the moving parts work as expected.
-NP

Nagendra - that’s a great point to consider.

But the graph actually shows increase dependency “bloat” – a term with a narrow meaning the researchers defined to mean dependencies the calling code can actually never use (as analysed by their special tool)

I suspect (but cannot prove) my Java code above has contributed to this effect: 40 JARs to upload a file is just

[Sonam_Chauhan]

@Nagendra_Prasad

From the first post in this topic, I think perhaps it was a licensing thing?

@Sonam_Chauhan

I heard of some ‘Azure adapter’ for IS, but my site’s not licensed for it (and it may not be available for 10.1)

@Nagendra_Prasad - You are correct. I ran Installer against Empower, and didn’t see any “Azure” adapters available. So I figured I’d write something basic. I didn’t figure on the rabbit hole this would take me down last week. This info from Suresh is news to me.

@Suresh_Ganta - Thank you - I didn’t know about this. To someone like me with no knowledge of Cloudstreams, what does the ‘CloudStreams Provider for Microsoft Azure Storage’ package do ? As far as I can make out from the zip file and documentation, the package provides two flow services and dozens of data structures.

Is the functionality it provides something akin to method signatures and connection pools? Would the code by @Rakesh_Kumar3 in post #7 use it?

Also, what is the licensing model for WmMicrosoftAzureStorageProvider.zip?

[Suresh_Ganta]

Hi @Sonam_Chauhan

webMethods CloudStreams - is a patented cloud connectivity framework that allows developers to build connetors by configuration (instead of coding). Offers following capabilities:

    • Connection Mgmt./Pooling
    
    • Multi Auth & Session Management
    
    • Runtime Governance
    
    • Streaming Listeners & Replay Event
    
    • Security
    
    • Multiple Content Types Support
    
    • MIME/Multi-part
    
    • MTOM
    
    • Send/Receive Messages as form data

For a complete list of capabilities of the CloudStreams please refer to the post Why do I need to go for CloudStreams - #2 by Suresh_Ganta .

You would need a license of webMethods CloudStreams and more than 80+ connectors are offered for FREE for the most widely used SaaS applications in the market. For a complete list of connectors please refer to webMethods CloudStreams Connectors - Software AG Tech Community & Forums

The Microsoft Azure Storage Connector package provides OOB services for the Microsoft Azure Storage Services REST APIs exposed Microsoft and the CloudStreams framework provides the capabilities like Connection Pooling, OOB support for the most common authentication and authorization mechanism.

We can have a session planned, if your customer is interested to know more about this product and its capabilities to see if it meets their business requirements

regards,

Suresh P.N.V.S Ganta

Software AG Product Management

[Sonam_Chauhan]

Thanks for the good explain Suresh.

I am a coder for the customer. They are licensed for on-prem but not CloudStreams (as far as I know). So going this road won’t help (at least not for my little project). But a capability to keep an eye on.

[Holger_von_Thomsen]

Hi Sonam,

the jar dependencies of the product adapters like SAP, JDBC, WebSphere MQ et al. are mainly inferred by the corresponding systems and should not be considered as bloat in this case as these jars just provide the neccessary classes to connect to the respective servers.
These APIs/libs are usually provided by the target system vendors like SAP, IBM or the DB system and usually do not have external dependencies despite native libraries delivered together with the provided jars.

Regards,
Holger

[Rakesh_Kumar3 (Rakesh Kumar)]

@Sonam_Chauhan @Suresh_Ganta : We do not have license for webMethods CloudStreams and had to write our own custom framework for Authentication and connecting to different Azure services .

It was not a major effort for us to build the services to connect to Azure via custom service .

(Just a Thought) CloudStreams connectors should be core part of the Software AG Offering , instead of it being a separate licensed product . As more and more enterprise products are SAAS based and if it’s made part of core integration Product it will be more beneficial more Software AG and their clients (win win situation ) .

[Sonam_Chauhan]

@Holger_von_Thomsen

Hi Sonam,

the jar dependencies of the product adapters like SAP, JDBC, WebSphere MQ et al. are mainly inferred by the corresponding systems and should not be considered as bloat in this case …

Oh, I absolutely agree Holger. For example, the SAP adapter uses just 3 JARs, downloaded from sdn.sap.com (so one vendor: SAP). I suspect their JARs are tightly optimised and there is little bloat there.

In contrast, the Azure Java API code I just created referenced a Microsoft JAR, which pulled 39 more JARs (various vendors) into code/jars. All this just to upload a file into Azure Blob Storage. So I suspect bloat in my solution.

[Suresh_Ganta]

Hi @Rakesh_Kumar3

Major key points to consider when it comes to writing custom solutions vs webMethods CloudStreams are as below.

We offer the below key capabilities out of the box thus taking away the need to develop, maintain, support custom solutions to connect/integrate with various SaaS Apps as this requires a lot of time, efforts which in turn is cost, thus enabling you to focus on the business applications building for the customers rather than spending time and efforts on building connectivity to SaaS Apps.

Hence webMethods CloudStreams is offered as a separate licensed product and the connectors which we have today as well as the ones we are going to create in future are offered for FREE.

• The frequency of SaaS Apps updates with new features and enhancements and in turn the API updates is quite frequent. The SaaS vendors release frequency varies from weeks, to monthly, bi-monthly, quarterly etc…

• Connection Management, Authentication & Authorization Mechanisms support – As the technology is evolving new standards of Authentication & Authorization are adopted by the SaaS vendors. Writing custom code to adopt these per SaaS Apps and do connection and connection pool management requires lot of efforts.

• Support of Different Connectivity Patterns by configuration driven approach without coding which reduces connector development efforts considerably be it be for Software AG for the Out of the Box connectors, as well as for anyone who develops custom connectors using webMethods CloudStreams

  • REST & OData Connectivity Patterns

    • Metadata Lookup
    • Connector Creation from Open API & Swagger
    • OData v2.0 and v4.0 connectors to connect to any OData/REST based integration SaaS Vendors

  • Event Based Patterns

    • Streaming – CometD based, HTTP Streaming etc.

  • SOAP Connectivity Patterns

    • Everything in the WSDL
    • SOAP Metadata Lookup
    • WSDL with Schemas Included
    • Multiple WSDL by functional Area

• Ability to leverage the 80+ connectors for most widely used SaaS Apps as well as the ability to create custom connectors with configuration driven approach rather than coding with all the above key features support

Regards,

Suresh P N V S Ganta
Software AG Product Management

[Sonam_Chauhan]

@toni.petrov

Yes, @Nagendra_Prasad 's suggestion was to recreate post #12

@Nagendra_Prasad , @toni.petrov - Just did that below. Thanks again.

Using Azure Java API to upload data to Microsoft Azure Blob Storage

Intent Develop a simple, standalone webMethods Integration Server (IS) service that uses the Azure Java API to upload data to Microsoft Azure Blob Storage. This should work for IS version 10.1 and above. Overview See discussion here, where this approach was formulated. Alternatives such as an Azure REST-based solution (simpler, architecturally cleaner) and webMethods Cloudstreams were also discussed. The alternatives may be better. But this approach has these key features: Simple to use Us…