String tokenize

How do we tokenize strings, based on delimiter ‘\n\r’ using the service Wmpublic.string:tokenize.

The service seems to have a bug. It doesnt recognize the ‘’ character.

if the delimiter is set to null or empty it will use ‘\n\t\r’ as the delimiter.


to tokenize string, input a line feed in the field. That is

  1. right click on the field, select “Use Large Editor” and then just do an enter in that field. This will recognize the \n.
    God works in strange ways, so does webMethods.
    Have a great day!

Hi, VR. I thought about that, too, but wasn’t sure about one thing: Does the server recognize the [Enter] as \n or \r\n differently depending on its platform? So, a physical line break provided on a NT machine (\r\n) is automatically converted to \n when the code is migrated to a UNIX box?


My knowledge is limited to Windows and in windows, the line feed in the field gets recognized as ‘\n’.

I used enter before, as delimitter and it works just fine on both windows and solaris.

A couple of things about this service:

  • As with many, many built-in services, this is simply a wrapper for the corresponding Java class. In this case, java.util.StringTokenizer

  • The default behavior of StringTokenizer, if no delimiters are specified, is to use " \t\n\r\f" (note that space is part of the set). Specifying no delimiter probably won’t do what you want Suhesh.

  • StringTokenizer has behavior that can catch you off guard if you are unaware of how it works. For example, multiple delimiters that appear together in the source will not cause empty tokens to be returned. If the delimiter is '’ and you tokenize the string "12**4" you’ll get back 3 tokens, not 4. Whether or not this is an issue depends on what you’re trying to do. This behavior is exactly what you want though if you’re parsing lines terminated with \r\n.


One more thing–none of the services (with the exception of a couple in the EDI adapter) support Java character escape sequences. \n, \r, \t, etc. are special instructions to the compiler to create the character. To use these in the FLOW environment you need to convert these to the “real” characters yourself. The code for a service that converts Java and Unicode character escape literals can be found at (be sure to read the entire thread if you’re going to use that code)

Apologies for the multiple posts…

Forgot to mention that the technique Vinod describes works just fine to get \n or \t. For other chars, you’ll need the service I referenced above.