Integration Server and Java

I disagree completely. FLOW is programming. That particular business person may also have enough technical capability to write FLOW is fine, but let’s not get into the silly notion that writing FLOW services doesn’t require programming. It does.

  • each step is in memory of IS another objects and any flow is a huge tree of objects - there isn’t any compilation to bytecode, all flow processing is executing on declarative stucture of step objects - slow and memory consuming,
  • if You have about 4500 flow services (we have more), IS will need 1 hour to stand up … if JVM 2 GB of memory will not depleted earlier

I disagree. When I’ve encountered such claims it is usually due to lack of familiarity with FLOW rather than limitations with FLOW itself. See posts earlier in this thread for such an example.

IME, the number 1 reason why things don’t work right is due to poor development practices and a fundamental lack of understanding/disregard for appropriate FLOW development.

There is no such thing as intuitive programming language, IMO. One must learn it. That you can transfer knowledge and understanding from previous experience is valuable, but that’s not “intuition.”

True. Sometimes one needs to use Java. But then that Java service can be used in a higher-level FLOW service. Most integration tasks can be done with FLOW. The majority of code for the majority of solutions will be written using FLOW.

True. In my 10+ years of working with IS and Broker (ActiveWorks back in the day) I’ve never had a need to write an adapter. Not once.

Hard? It is trivial.

True. Debugging can be a pain.

Developer isn’t a Java IDE, so “real IDE” is meaningless. Using a Java IDE is better for developing Java. No surprise there.

The code generation wizard isn’t sufficient? Can you elaborate?

As above, I contend this is not a primary benefit of FLOW. It may be more or less approachable than other programming languages, but this tool is best in the hands of a programmer.

Interesting. I’ve never, ever heard anyone refer to FLOW as a declarative programming language. IMO, it is not. It is procedural.

Can you elaborate on this? I think you may be making assumptions about the implementation of the run-time environment.

Can you elaborate on any of these? Is there an assumption that “4500 flow services” is a reasonable number of services within a single IS instance? What sort of sizing exercises were done to appropriately size the environment?

True. But FLOW has been around much longer.

I disagree.

Java is intuitive? That’s quite a claim!

Can you elaborate? Where I find code hard to maintain is when people have been lax in keeping a clean pipeline. A service that ends up with tens or hundreds of variables is indeed very hard to work with–but that’s lack of programming discipline.

What do you me by “disintegrated?” Which IS bugs did you encounter?

When I come across pros/cons such as this I usually find that they are from Java programmers that have relatively little understanding of IS and FLOW. I don’t know if you fall in to that category but it sure seems that way based on these posts.

Don’t get me wrong. FLOW and Developer aren’t the perfect tools (I never look forward to mapping a relatively large document–all that clicking and dragging is tedious and annoying). But I take issue with some of the gross generalizations.

I think he is just quoting the “party line” from corporate. This is how the product is sold/marketed. :frowning:

Agree. Writing a webMethods Java API wrapper over existing Java code is easy. If you write it in an IDE (like Eclipse) getting it into Developer is a tedious process… and writing it in Developer itself is even worse!

I was due IS or Developer malfunction - we have lost our works a few times.

Maybe right. But how to write high performance system with use of slow EJBAdapter ? Or how to communicate with EIS via highly customized CORBA services ? … by writing custom connector.

It’s very hard and code obscure. We have a very complex artificial canonical model of documents not just set of simple native documents for any EIS. We have in Poland about 50 IS servers in one part of telecommunication company and I think even mode in other part, working in 5 layers with dynamic proxy services and much more features.

The code generation wizard obscure the real code and it’s not independent of logic which means it can’t be used for correction of code generated previously after any change of input/output contacts.

Flow is XML and this XML is a structure. This Flow XML when loading is not just compiled to Java code, but binding to structure of Java objects tree of execution. From JVM point of view it is a declarative description of execution path.

[/i]

Maintain the clean pipeline is not a problem for professionals I’m working with.
As I said we have big canonical document model consist also of smaller common entities which are reused in many business documents and also many services. If there is a need to change one of this highly reused small documents, what the feature of WM will help to propagate changes to all the services using this just changed document in mapping, what tool will help to validate painful manual changes ? I tell You: find, grep, sed, awk posix commands.

Sounds like IS may not be the right tool for the jobs you are using it for? But that may be a premature judgement on my part given that I don’t have the complete picture of your environment.

Excellent point. Document type changes can be a real pain.

Are there other scenarios that lead you the more general claim that “FLOW code is hard to maintain and refactor?”

“We” (in fact our Clients) have hardware load balancer, about 50 IS’es. One Client has 5 layers of IS. Layers was a need, because IS in long process of EAI development was standing up longer and longer (up to 1 hour on stong IBM blade machines), OutOfMemoryError appeared more frequently (despite advanced tuning of GC, memory parameters and Linux Kernel), IS 6.1.5 sp2 had to be restarted in 2-5 days period, and after some release IS’es was unable to start. And layers and dynamic proxy appears.
I know, it’s maybe a problem of misunderstanding primary task of EAI platform, but many people still believes that this huge artificial canonical document model and following division of services to: logical adapter, enterprise, channel, implementation of complex logic in flows was a good inventions. I don’t really know.

I don’t like Base64 coding of connection parameters and inability to easy refactorize adapter services (especially point of connection with connection definition) or names of input/output documents also.

I was using profiler, I’m almost sure. There isn’t another way:

  • (tree of) Java bytecode statements sequence (not used in IS - I’m almost sure)or,
  • declarative tree of step-objects representing one flow (I was using not documented API to walk on this, I think, structure and on structure of document definition) or,
  • life scripting-like passing of Flow XML (performance suicide - not used in IS).

It is trivial, because there is not IData, IDataCursor, only my very simple and universal wrapping class.
This example have no sense of course, just showing Java code without IData, IDataCursor.[INDENT]

 ...
        WmData data = new WmData().setData(pipeline);
...
        String name = data.popString("nsName"); // get and drop
        String rsn = data.getString("receiptSequenceNumber");
  class BrokerSettings {[INDENT]             void invoke() {[INDENT]                 WmData data = new WmData().create();
                try {[INDENT]data.setData(Service.doInvoke(NSName.create("wm.server.dispatcher.adminui:getBrokerSettings"), data.put("receiptSequenceNumber", "1").put("invocationType", "publishAndWait").getData()));
[/INDENT][INDENT]                     this.data.append(data);
[/INDENT]} catch (Exception e) {[INDENT] ...
[/INDENT]}
... 
[/INDENT]}
            WmData data;
[/INDENT]}
...
        data.close();

[/INDENT]

Interesting. Seems similar to the IDataUtil class that has various utility methods to get/set pipeline vars.

It’s very simple example of use of my code generator and use of generated by it code (skel and impl - can generate also stub classes)
Development of this generator I dropped 2 years ago.

At this moment, I’m rather thinking of creating wmflow2java compiler with dynamic replacement of flow services by precompiled java bytecode.

Generated skeleton for very simple service:

/*
 * File generated by ... WM Generator v 0.2
 * Don't change it. All changes will be lost ! 
 */
package skel.something.pub;

import com.prokom.wm.*;

/**
 * @webmethods.service name="generate" type="java/unknown" full-name="prokom.j4wm.generator.pub:generate"
 * @webmethods.node nodeTypeObj.type="service" nodeTypeObj.value="service" nodeType="2" nodeSType="service"
 * @webmethods.node.nsName fullName="prokom.j4wm.generator.pub:generate" value="prokom.j4wm.generator.pub:generate" interface="false" interface.fullName="prokom.j4wm.generator.pub" interface.value="prokom.j4wm.generator.pub"
 * @webmethods.node.values node_type="service" node_nsName="prokom.j4wm.generator.pub:generate" node_pkg="PROKOM" svc_type="java" svc_subtype="unknown" svc_sig="in:{<unnamed>;t=2;d=0}[n=targetPackage;t=1;d=0, n=nodesNames;t=1;d=1, n=targetType;t=1;d=0, n=throwOnError;t=3;d=0]/out:{<unnamed>;t=2;d=0}[n=messages;t=1;d=1]" svc_sigtype="java 3.5" icontext_policy="urn:icontext:$null" auditoption="0" auditsettings=" >>>document_data=0, startExecution=false, stopExecution=false, onError=true<<< "
 */
public abstract class GenerateSkel extends ServiceSkel {

    public String getNsFolderName() {
        return "prokom.j4wm.generator.pub";
    }
    public String getNsName() {
        return "generate";
    }
    public IRecordWrap getInRaw() {
        return getIn();
    }
    public IRecordWrap getOutRaw() {
        return getOut();
    }
    public In getIn() {
        return (In)((in==null) ? in = new In() : in);
    }
    public Out getOut() {
        return (Out)((out==null) ? out = new Out() : out);
    }
    /**
     * @webmethods.record name="null" real-name="null" type="2" java-wrapper-type="UNKNOWN" dimensions="0" path="" closed="false" expanded="false" publishable="false" recursive="false"
     * @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
     * @webmethods.node.values node_type="record" field_type="record" field_dim="0" nillable="true" rec_fields="[Lcom.wm.util.Values;@2dc78fcf"
     */
    public class In extends RecordWrap {

        public String getNsFolderName() {
            return "default";
        }
        public String getNsName() {
            return "null";
        }
        /**
         * @webmethods.field name="targetPackage" real-name="targetPackage" type="1" java-wrapper-type="UNKNOWN" dimensions="0" path="/targetPackage;1;0" nillable="true" optional="true" ui-editable="false"
         * @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
         * @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_usereditable=null, field_largerEditor=false, field_password=false<<< " field_name="targetPackage" field_type="string" field_dim="0" field_opt="true" nillable="true"
         */
        public String getTargetPackage() {
            return (String)dataUtil.get("targetPackage");
        }
        public void setTargetPackage(String value) {
            dataUtil.put("targetPackage", value);
        }
        /**
         * @webmethods.field name="nodesNames" real-name="nodesNames" type="1" java-wrapper-type="UNKNOWN" dimensions="1" path="/nodesNames;1;1" nillable="true" optional="false" ui-editable="false"
         * @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
         * @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_largerEditor=false, field_password=false<<< " field_name="nodesNames" field_type="string" field_dim="1" nillable="true"
         */
        public String[] getNodesNames() {
            return (String[])dataUtil.get("nodesNames");
        }
        public void setNodesNames(String[] value) {
            dataUtil.put("nodesNames", value);
        }
        /**
         * @webmethods.field name="targetType" real-name="targetType" type="1" java-wrapper-type="UNKNOWN" dimensions="0" path="/targetType;1;0" nillable="true" optional="false" ui-editable="false" options="STUB|SKEL"
         * @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
         * @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_usereditable=false, field_largerEditor=false, field_password=false<<< " field_name="targetType" field_type="string" field_dim="0" field_options="[Ljava.lang.String;@64ad0f99" nillable="true"
         */
        public String getTargetType() {
            return (String)dataUtil.get("targetType");
        }
        public void setTargetType(String value) {
            dataUtil.put("targetType", value);
        }
        /**
         * @webmethods.field name="throwOnError" real-name="throwOnError" type="3" java-wrapper-type="java.lang.Boolean" dimensions="0" path="/throwOnError;3.1;0" nillable="true" optional="true" ui-editable="false"
         * @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
         * @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_largerEditor=false, field_password=false<<< " field_name="throwOnError" field_type="object" field_dim="0" field_opt="true" wrapper_type="java.lang.Boolean" nillable="true"
         */
        public Boolean getThrowOnError() {
            return (Boolean)dataUtil.get("throwOnError");
        }
        public void setThrowOnError(Boolean value) {
            dataUtil.put("throwOnError", value);
        }
    }
    /**
     * @webmethods.record name="null" real-name="null" type="2" java-wrapper-type="UNKNOWN" dimensions="0" path="" closed="false" expanded="false" publishable="false" recursive="false"
     * @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
     * @webmethods.node.values node_type="record" field_type="record" field_dim="0" nillable="true" rec_fields="[Lcom.wm.util.Values;@317b0fcf"
     */
    public class Out extends RecordWrap {

        public String getNsFolderName() {
            return "default";
        }
        public String getNsName() {
            return "null";
        }
        /**
         * @webmethods.field name="messages" real-name="messages" type="1" java-wrapper-type="UNKNOWN" dimensions="1" path="/messages;1;1" nillable="true" optional="true" ui-editable="false"
         * @webmethods.node nodeTypeObj.type="record" nodeTypeObj.value="record" nodeType="4" nodeSType="record"
         * @webmethods.node.values node_type="record" node_comment="" node_hints=" >>>field_largerEditor=false, field_password=false<<< " field_name="messages" field_type="string" field_dim="1" field_opt="true" nillable="true"
         */
        public String[] getMessages() {
            return (String[])dataUtil.get("messages");
        }
        public void setMessages(String[] value) {
            dataUtil.put("messages", value);
        }
    }

}

Generated empty implementation of service with no-sense implementation:

/*
 * File generated by Prokom WM Generator v 0.2
 * Implement this initially empty service !
 */
package impl.prokom.j4wm.generator.pub;


/**
 * @webmethods.service name="generate" type="java/unknown" full-name="prokom.j4wm.generator.pub:generate"
 * @webmethods.node nodeTypeObj.type="service" nodeTypeObj.value="service" nodeType="2" nodeSType="service"
 * @webmethods.node.nsName fullName="prokom.j4wm.generator.pub:generate" value="prokom.j4wm.generator.pub:generate" interface="false" interface.fullName="prokom.j4wm.generator.pub" interface.value="prokom.j4wm.generator.pub"
 * @webmethods.node.values node_type="service" node_nsName="prokom.j4wm.generator.pub:generate" node_pkg="PROKOM" svc_type="java" svc_subtype="unknown" svc_sig="in:{<unnamed>;t=2;d=0}[n=targetPackage;t=1;d=0, n=nodesNames;t=1;d=1, n=targetType;t=1;d=0, n=throwOnError;t=3;d=0]/out:{<unnamed>;t=2;d=0}[n=messages;t=1;d=1]" svc_sigtype="java 3.5" icontext_policy="urn:icontext:$null" auditoption="0" auditsettings=" >>>document_data=0, startExecution=false, stopExecution=false, onError=true<<< "
 */
public class GenerateImpl extends skel.prokom.j4wm.generator.pub.GenerateSkel {

    public void dispatch() throws Exception {[INDENT]//throw new Exception("This is still not implemented J4WM service skel.prokom.j4wm.generator.pubGenerateImpl !");
... = getIn().getNsFolderName();
... = getIn().getSomeReferenceDocumentIfItWasAnyInInputContract().getSomeField("A");
...
///logic ...
...
getOut().setMessages({"many things", "nothing"});
...
[/INDENT]}

}

And use in service:

    public static final void generate (IData pipeline)
        throws ServiceException
    {
        // --- <<IS-START(generate)>> ---
        // @subtype unknown
        // @sigtype java 3.5
        // [i] field:0:optional targetPackage
        // [i] field:1:required nodesNames
        // [i] field:0:required targetType {"STUB","SKEL"}
        // [i] object:0:optional throwOnError
        // [o] field:1:optional messages
        com.prokom.j4wm.ServiceFinder.dispatch(pipeline);
        // --- <<IS-END>> ---
    }

Hello,
This is just a comment on the consideration of FLOW being more declarative than procedural. If you manage to think of the fact that FLOW is stored as XML in the backend, it may make sense from one view. HTML and CSS would be considered declarative. This is because the actions of these code sections tend to be heavily different based on the runtime it executes on (IE, FF, Opera, Safari). However in the same context of a code file, Javascript would not be in the same boat. This is because it has a more consistent interpretation.

Also the way that actions are described in FLOW are not of the declare and/or relation nature. FLOW lends you to do assignments, fire off precise executions of other services and do explicit branching. Now I will say. as is known, that some steps execute in a non-deterministic manner (MAP steps). That is pretty close to declarative because you say assign (what to do), but in that local scope do can’t care which one is done first (how to do).

So all in all, FLOW is a multi-paradigm language which allows for easy procedural codings and a clean Java FFI. Good day.

Yemi Bedu

Yes, it’s all true, but from Java Runtime point of view, WM Flow is not (imperative) byte-code, but a structure of objects on the heap/stack having a receipt (declarative approach or dynamic programming) of “what to do” (declarative approach), with a little “how to do” (imperative approach).

Hello,
My distinction was that the language was not declarative. Some scripting languages work without compiling and just make objects on the heap or stack to await method invocations based on there message. By declarative, we can usually see that things like ordering are not relavent in a scope and that you are thinking more of how the representation of the states should exist rather than how to compute getting to that state. Use of Loop, Branch, and Invoke are certainly not declarative constructs.

The fact that FLOW does not compile down to bytecode has no bearing. Byte-code is one up from a native CPU compile. In-memory only Jit compiling is maybe one up from bytecode and raw unoptimized interpretation is probably still one up in this relation.

I will say that it is slow and a compiler is sorely need for the simple and primitive elements of execution. I mean it should always compile when you “UNLOCK” a service. It should then show in another tab what the optimized (maybe java) code looks like. That sounds like stuff we talked about before. Keep up the good work. Good day.

Yemi Bedu

FLOW is not a declarative language in any sense. It is a procedural language. The run-time environment that executes the code does not have any bearing on this.

Hello,
I am mostly on the “FLOW is procedural” side of the swing. I argue on the language and not the implementation or runtime effect. To this end, the MAP step still holds a semi-declare nature for assigment. I do not include the use of transforms in there. Since the order of assignment can not be guaranteed, it is implicit to execution of the calls. Good day.

Yemi Bedu

Hi Forum,

I just wanted your comments with viable justifications on the million dollar question of “Java Services Vs Flow Services… like which is the better choice in production standard coding practices…?” please let me know what you guys think…?

reg,
nightfox