Tundra and TundraTN - useful packages for webMethods Integration Server and Trading Networks

Hi,

For the last year I’ve been developing a couple of open-source utility packages for webMethods Integration Server and Trading Networks in my spare time, and I’d like to share them with the community:

Tundra - [url]https://github.com/Permafrost/Tundra[/url]
TundraTN - [url]https://github.com/Permafrost/TundraTN[/url]

There are some cool services in these packages, which can seriously reduce the amount of boilerplate code that is sometimes required when writing services for IS or TN.

For example, Tundra/tundra.content:parse parses arbitrary content (specified as a string, byte array, or input stream) into an IData document in a uniform way for both XML and flat file formats. Or you can do the reverse with the complementary service Tundra/tundra.content:emit, converting an IData document to either an XML or flat file string, byte array or input stream in one line of code.

Building on those two services, Tundra/tundra.content:translate parses arbitrary content, calls the mapping service you specify with the parsed IData document as input, and then emits the resulting mapped IData document returned by your mapping service as either a string, byte array, or input stream. TundraTN/tundra.tn:translate then takes that concept and applies it to Trading Networks processing services. Called from a TN processing rule, you specify the mapping service to use and it does the rest: parses the bizdoc content (whether large or small), calls the specified mapping service, converts the resulting mapped IData document back to an input stream, and then routes the resulting content back to TN as a new bizdoc. As all of this boilerplate code is taken care of for you, all you have to supply is the mapping service that converts one IData document to another, which is as it should be: this is the code that changes from one project to the next, not the boilerplate required for dealing with TN or parsing content etc.

There’s Tundra/tundra.content:deliver, which delivers arbitrary content (string, byte array, or input stream) to a given URL in a uniform way. It currently supports delivering to file:, http: and https: URLs, but I plan to add support for other schemes like mailto: and sftp: in the future. TundraTN/tundra.tn:deliver then applies this concept to Trading Networks processing services, letting you deliver a bizdoc to a URL you specify directly from a TN processing rule.

There’s some services for messing with the pipeline at runtime too. For example, Tundra/tundra.pipeline:copy lets you copy pipeline values at runtime, with fully-qualified keys such as a/b/c[0]/d supported. You can even do variable substitution on the pipeline at runtime using Tundra/tundra.pipeline:substitute. And what about a try/catch/finally pattern for flow services with Tundra/tundra.service:ensure?

Anyway, that’s just a small preview of the goodies in these packages. Please check out the READMEs for the full list of services. As I’m still working on them, both packages are considered pre-release at the moment, and are liable to change - I will however endeavour to minimise backwards-incompatible changes going forward.

Feedback and contributions would both be very appreciated!

Regards,
Lachlan

2 Likes

Thanks for your effort…and great app dev looks like…

May be you / moderator can move this KB thread to shareware or FAQ section of this site

1 Like

Do you know how I would do this, as I’m not sure where the shareware thread is on this forum?

1 Like

If you don’t find that please reach out to - Desislava.Petrova@softwareag.com for the forum assistance.

You can do it in the Code Samples section. [url]http://techcommunity.softwareag.com/ecosystem/communities/codesamples/webmethods/esb/webmethods-esb-codesamples.html[/url] Click on Post new Code sample

1 Like

good idea:

1 Like

Exellent Code. Very very useful.

1 Like

Hi All,

I am interested to use the following services in wM 9.6. The code is not getting compiled.

I would like to keep java code to minimum. Is it possible to delete all other java services which are not required and use only these 2 java services?

tundra.pipeline:emit
tundra.pipeline:parse

Could you please help me with the package which provides minimum functionality instead of importing everything?

Thanks,
Shashank

What is the compiler error and also the chances are JS code may be not be up to jvm1.7…?

HTH,
RMG

Compiler doesn’t find many of the method calls used in webMethods java services. I am figuring out a way to list class and method dependencies and re-compile the necessary code on my own.

However, the developer has used so many classes and methods - it’s not easy to determine dependencies !!

Class files within Tundra pkg works fine. However, I can’t import/install a 3rd party code into my client’s machine. And also, I may not need all functionality developed in Tundra package.

I am looking for a way to

  1. extract Pipeline into Stream
  2. Convert Stream into Pipeline and String.

OK Let’s see if the original author replies here:

What are you trying to do pipeline introspection?

HTH,
RMG

Yes. I am trying to store Pipeline in the form of Stream into a database.

Yes, the problem you have is that the Tundra/tundra.pipeline:emit and Tundra/tundra.pipeline:parse services make use of other Java methods in the Tundra package for its implementation, so you can’t just copy those two services into another package without all the other dependencies and have them compile.

I’ve created standalone versions of just those two services for you into the attached package Pipeline-v0.0.1.zip. You should be able to copy/paste/move/rename these services and have it compile because all the dependent code is in the Java shared source for the services.

I created this package on Integration Server 8.2.2 and compiled the services for Java 1.6. Let me know if you have any issues running or recompiling these services on Integration Server 9.x and Java 1.7.

And just for completeness, here is the pipeline.java source file from the attached Pipeline-v0.0.1.zip package for the two services pipeline:emit and pipeline:parse inline for your perusal:



// -----( IS Java Code Template v1.2
// -----( CREATED: 2014-08-28 07:37:31 EST
// -----( ON-HOST: 172.16.189.132

import com.wm.data.*;
import com.wm.util.Values;
import com.wm.app.b2b.server.Service;
import com.wm.app.b2b.server.ServiceException;
// --- <<IS-START-IMPORTS>> ---
// --- <<IS-END-IMPORTS>> ---

public final class pipeline

{
	// ---( internal utility methods )---

	final static pipeline _instance = new pipeline();

	static pipeline _newInstance() { return new pipeline(); }

	static pipeline _cast(Object o) { return (pipeline)o; }

	// ---( server methods )---




	public static final void emit (IData pipeline)
        throws ServiceException
	{
		// --- <<IS-START(emit)>> ---
		// @subtype unknown
		// @sigtype java 3.5
		// [i] field:0:optional $encoding
		// [i] field:0:optional $mode {&quot;stream&quot;,&quot;bytes&quot;,&quot;string&quot;}
		// [o] object:0:optional $content
		IDataCursor cursor = pipeline.getCursor();
		
		try {
		  String encoding = IDataUtil.getString(cursor, "$encoding");
		  String mode = IDataUtil.getString(cursor, "$mode");
		  IDataUtil.remove(cursor, "$encoding");
		  IDataUtil.remove(cursor, "$mode");
		
		  IDataUtil.put(cursor, "$content", tundra.document.emit(pipeline, encoding, mode));
		} catch(java.io.IOException ex) {
		  tundra.exception.raise(ex);
		} finally {
		  cursor.destroy();
		}
		// --- <<IS-END>> ---

                
	}



	public static final void parse (IData pipeline)
        throws ServiceException
	{
		// --- <<IS-START(parse)>> ---
		// @subtype unknown
		// @sigtype java 3.5
		// [i] object:0:optional $content
		// [i] field:0:optional $encoding
		IDataCursor cursor = pipeline.getCursor();
		
		try {
		  Object content = IDataUtil.get(cursor, "$content");
		  String encoding = IDataUtil.getString(cursor, "$encoding");
		
		  IDataUtil.merge(tundra.document.parse(content, encoding), pipeline);
		} catch(java.io.IOException ex) {
		  tundra.exception.raise(ex);
		} finally {
		  cursor.destroy();
		}
		// --- <<IS-END>> ---

                
	}

	// --- <<IS-START-SHARED>> ---
	public static class tundra {
	  public static class document {
	    // parses an IData XML input stream to an IData object
	    public static IData parse(java.io.InputStream in) throws java.io.IOException {
	      com.wm.util.coder.XMLCoderWrapper codec = new com.wm.util.coder.XMLCoderWrapper();
	      return codec.decode(in);
	    }
	
	    // parses an IData XML string, byte array or input stream to an IData object
	    public static IData parse(Object content, String encoding) throws java.io.IOException {
	      return parse((java.io.InputStream)tundra.object.convert(content, encoding, "stream"));
	    }
	
	    // emits an IData object as a an IData XML string, byte array or stream
	    public static Object emit(IData document, String encoding, String mode) throws java.io.IOException {
	      com.wm.util.coder.IDataXMLCoder codec = null;
	      if (encoding == null) {
	        codec = new com.wm.util.coder.IDataXMLCoder();
	      } else {
	        codec = new com.wm.util.coder.IDataXMLCoder(encoding);
	      }
	
	      java.io.ByteArrayOutputStream out = new java.io.ByteArrayOutputStream();
	      codec.encode(out, document);
	
	      return tundra.object.convert(out.toByteArray(), encoding, mode);
	    }
	  }
	
	  public static class object {
	    // converts a string, byte array or stream to a string, byte array or stream
	    public static Object convert(Object object, String encoding, String mode) throws java.io.IOException {
	      if (mode ==  null) mode = "stream";
	
	      if (mode.equals("bytes")) {
	        object = tundra.bytes.normalize(object, encoding);
	      } else if (mode.equals("string")) {
	        object = tundra.string.normalize(object, encoding);
	      } else {
	        object = tundra.stream.normalize(object, encoding);
	      }
	
	      return object;
	    }
	  }
	
	  public static class bytes {
	    // converts a string, bytes or input stream to bytes
	    public static byte[] normalize(Object object, String encoding) throws java.io.IOException {
	      if (encoding == null) encoding = tundra.support.constant.DEFAULT_CHARACTER_ENCODING;
	      
	      byte[] bytes = null;
	      
	      if (object != null) {
	        if (object instanceof byte[]) {
	          bytes = (byte[])object;
	        } else if (object instanceof String) {
	          bytes = ((String)object).getBytes(encoding);
	        } else if (object instanceof java.io.InputStream) {
	          java.io.ByteArrayOutputStream out = new java.io.ByteArrayOutputStream();
	          tundra.stream.copy((java.io.InputStream)object, out);
	          bytes = out.toByteArray();
	        } else {
	          throw new IllegalArgumentException("object must be a string, byte[] or java.io.InputStream: " + object.getClass().getName());
	        }
	      }
	
	      return bytes;
	    }
	  }
	
	  public static class exception {
	    // throws a new ServiceException with the class and message from the given Throwables, which
	    // is useful because java services are hard-wired to only throw ServiceExceptions
	    public static void raise(Throwable[] exceptions) throws ServiceException {
	      if (exceptions != null) raise(message(exceptions));
	    }
	
	    // throws a new ServiceException with the class and message from the given Throwables, which
	    // is useful because java services are hard-wired to only throw ServiceExceptions
	    public static void raise(java.util.Collection<Throwable> exceptions) throws ServiceException {
	      if (exceptions != null) raise(message((Throwable[])exceptions.toArray(new Throwable[0])));
	    }
	
	    // throws a new ServiceException with the class and message from the given Throwable, which
	    // is useful because java services are hard-wired to only throw ServiceExceptions
	    public static void raise(Throwable exception) throws ServiceException {
	      if (exception != null) {
	        if (exception instanceof ServiceException) {
	          throw (ServiceException)exception;
	        } else {
	          raise(message(exception));
	        }
	      }
	    }
	
	    // throws a new ServiceException with the given message
	    public static void raise(String message) throws ServiceException {
	      throw new ServiceException(message == null ? "" : message);
	    }
	
	    // returns an exception as a string
	    public static String message(Throwable exception) {
	      String message = "";
	
	      if (exception != null) {
	        if (exception instanceof ServiceException) {
	          message = exception.getMessage();
	        } else {
	          message = exception.getClass().getName() + ": " + exception.getMessage();
	        }
	      }
	
	      return message;
	    }
	
	    // returns a list of exceptions as a string
	    public static String message(java.util.Collection<Throwable> exceptions) {
	      return message((Throwable[])exceptions.toArray(new Throwable[0]));
	    }
	
	    // returns a list of exceptions as a string
	    public static String message(Throwable[] exceptions) {
	      StringBuilder msg = new StringBuilder();
	      if (exceptions != null) {
	        if (exceptions.length == 1 && exceptions[0] != null) {
	          msg.append(message(exceptions[0]));
	        } else {
	          for (int i = 0; i < exceptions.length; i++) {
	            if (exceptions[i] != null) {
	              msg.append("[").append(i).append("]: ").append(message(exceptions[i]));
	              if (i < exceptions.length - 1) msg.append("\n");
	            }
	          }
	        }
	      }
	      return msg.toString();
	    }
	
	    // returns the call stack associated with the given exception
	    public static IData[] stack(Throwable exception) {
	      IData[] output = null;
	
	      if (exception != null) {
	        StackTraceElement[] stack = exception.getStackTrace();
	        output = new IData[stack.length];
	
	        for (int i = 0; i < stack.length; i++) {
	          output[i] = IDataFactory.create();
	          IDataCursor cursor = output[i].getCursor();
	          IDataUtil.put(cursor, "description", stack[i].toString());
	          IDataUtil.put(cursor, "file", stack[i].getFileName());
	          IDataUtil.put(cursor, "class", stack[i].getClassName());
	          IDataUtil.put(cursor, "method", stack[i].getMethodName());
	          IDataUtil.put(cursor, "line", "" + stack[i].getLineNumber());
	          IDataUtil.put(cursor, "native?", "" + stack[i].isNativeMethod());
	          cursor.destroy();
	        }
	      }
	
	      return output;
	    }
	  }
	
	  public static class stream {
	    // converts the given object, which must be either an input stream, byte array, or string, to an input stream, using the 
	    // given encoding if the object was a string
	    public static java.io.InputStream normalize(Object object, String encoding) throws java.io.UnsupportedEncodingException {
	      if (encoding == null) encoding = tundra.support.constant.DEFAULT_CHARACTER_ENCODING;
	      
	      java.io.InputStream stream = null;
	      
	      if (object != null) {
	        if (object instanceof java.io.BufferedInputStream) {
	          stream = (java.io.BufferedInputStream)object;
	        } else if (object instanceof java.io.InputStream) {
	          stream = new java.io.BufferedInputStream((java.io.InputStream)object, tundra.support.constant.DEFAULT_BUFFER_SIZE);
	        } else if (object instanceof byte[]) {
	          stream = new java.io.ByteArrayInputStream((byte[])object);
	        } else if (object instanceof String) {
	          stream = new java.io.ByteArrayInputStream(((String)object).getBytes(encoding));
	        } else {
	          throw new IllegalArgumentException("object must be a string, byte[] or java.io.InputStream: " + object.getClass().getName());
	        }
	      }
	
	      return stream;
	    }
	
	    // copies all data from the input stream to the output stream, then closes both streams
	    public static void copy(java.io.InputStream in, java.io.OutputStream out) throws java.io.IOException {  
	      copy(in, out, true);
	    }
	
	    // copies all data from the input stream to the output stream, and optionally closes both streams
	    public static void copy(java.io.InputStream in, java.io.OutputStream out, boolean close) throws java.io.IOException {  
	      try {
	        if (in == null || out == null) return;
	      
	        byte[] buffer = new byte[tundra.support.constant.DEFAULT_BUFFER_SIZE];
	        int length = 0;
	        
	        if (!(in instanceof java.io.BufferedInputStream)) in = new java.io.BufferedInputStream(in, tundra.support.constant.DEFAULT_BUFFER_SIZE);
	        if (!(out instanceof java.io.BufferedOutputStream)) out = new java.io.BufferedOutputStream(out, tundra.support.constant.DEFAULT_BUFFER_SIZE);
	
	        while((length = in.read(buffer)) > 0) {
	          out.write(buffer, 0, length);
	        }
	      } finally {
	        if (close) close(in, out);
	      }
	    }
	
	    // copies all the data from the reader to the writer, then closes both
	    public static void copy(java.io.Reader reader, java.io.Writer writer) throws java.io.IOException {  
	      try {
	        if (reader == null || writer == null) return;
	        
	        char[] buffer = new char[tundra.support.constant.DEFAULT_BUFFER_SIZE];
	        int length = 0;
	        
	        if (!(reader instanceof java.io.BufferedReader)) reader = new java.io.BufferedReader(reader, tundra.support.constant.DEFAULT_BUFFER_SIZE);
	        if (!(writer instanceof java.io.BufferedWriter)) writer = new java.io.BufferedWriter(writer, tundra.support.constant.DEFAULT_BUFFER_SIZE);
	      
	        while((length = reader.read(buffer)) > 0) {
	          writer.write(buffer, 0, length);
	        }
	      } finally {
	        close(reader, writer);
	      }
	    }
	
	    // closes the given list of objects, which must be one of the following types: java.io.InputStream, 
	    // java.io.OutputStream, java.io.Reader, or java.io.Writer
	    public static void close(Object ... streams) {
	      if (streams == null) return;
	      
	      for (int i = 0; i < streams.length; i++) {
	        Object stream = streams[i];
	        try {
	          if (stream != null) {
	            if (stream instanceof java.io.InputStream) {
	              ((java.io.InputStream)stream).close();
	            } else if (stream instanceof java.io.OutputStream) {
	              ((java.io.OutputStream)stream).close();
	            } else if (stream instanceof java.io.Reader) {
	              ((java.io.Reader)stream).close();
	            } else  if (stream instanceof java.io.Writer) {
	              ((java.io.Writer)stream).close();
	            }
	          }
	        } catch (java.io.IOException ex) {}
	      }
	    }
	  }
	
	  public static class string {
	    // converts a byte array, input stream or string to a string
	    public static String normalize(Object object, String encoding) throws java.io.IOException {
	      if (encoding == null) encoding = tundra.support.constant.DEFAULT_CHARACTER_ENCODING;
	      
	      String string = null;
	      
	      if (object != null) {
	        if (object instanceof String) {
	          string = (String)object;
	        } else if (object instanceof byte[]) {
	          string = new String((byte[])object, encoding);
	        } else if (object instanceof java.io.InputStream) {      
	          java.io.Writer writer = new java.io.StringWriter();
	          tundra.stream.copy(new java.io.InputStreamReader((java.io.InputStream)object, encoding), writer);
	          string = writer.toString();
	        } else {
	          throw new IllegalArgumentException("object must be a string, byte[] or java.io.InputStream: " + object.getClass().getName());
	        }
	      }
	
	      return string;
	    }
	  }
	
	  public static class support {
	    public static class constant {
	      public static final String DEFAULT_CHARACTER_ENCODING = java.nio.charset.Charset.defaultCharset().name();
	      public static final int    DEFAULT_BUFFER_SIZE        = 4096;
	    }
	  }
	}
	// --- <<IS-END-SHARED>> ---
}

Pipeline-v0.0.1.zip (23.9 KB)

OK great…I know you will see it and reply :smiley:

Perfect. Thanks a lot for your efforts.

Excellent code :slight_smile:

Are you using GIT hub for Version Control on your local VCS? Do you have any documentation on how to configure wM to use GIT hub as VC Tool?

Thanks,
Shashank

Sounds a good deal the code works :slight_smile:

Yeah, there are some really good utilities in Tundra.

Yep have a good day!

Hi Shank,

Yes, I use Git for version control with webMethods. I run Git from the command line, so the way I use it there is no integration with Developer or Designer.

I create one Git repository per package, and I set up a .gitignore file to ignore all *.bak files, but I store all other files in the Git repository (ie. all changes to all the flow.xml, node.idf, node.ndf, flow.xml, java.frag, *.java, *.jar, *.class files get tracked in the git repository).

I don’t use Git branches, because there’s no good way to merge flow.xml files, so I just commit all changes to the master branch and push those changes to a remote repository stored elsewhere to the one on the development Integration Server.

If you want to learn Git, there’s a good free book Pro Git on the Git web site.

I hope this helps…

Regards,
Lachlan

Tundra v0.0.15

Attached is a pre-built binary of the latest release of my Tundra package, v0.0.15.

This should make it easier for forum members to try out the package (rather than having to mess around with git or github.com to get a copy).

Feedback appreciated!
Tundra-v0.0.15.zip (3.41 MB)