How do I find the size of data?

, ,
1

Hi everyone,

First post, so be gentle and apologies if I’m in the work place or anything!

We have a requirement to bill clients by the byte size of data flowing through our system. Billing is done on a per-client basis, so global figures are no good.

Therefore we need to know the byte size of each file/message passing through our webMethods system. (We will then log this data in Oracle for reporting on).

I’ve searched both here and advantage and haven’t managed to find anything specific to my need.

For the moment I’ve written a simple flow service that just converts the byte steam to a string, then checks the length. This works fine with small files, although I suspect it’s not a very efficient way of doing it.

But the above process gets very slow, or falls over completely with larger files.

Our average file size is probably going to be around 5-10KB, but some interfaces will have 50-100MB files.

So is there any straightforward way of getting the exact byte size of data passing through wM?

I guess we could end up with two ways of doing it, small files throught the string length route, large files through a different route alltogether.

System info:
webMethods 6.5 running on AIX 5.3
Just in case it makes any difference, we don’t use Broker.
The data can come by multiple sources, FTP, file polling, MQ etc. So something that isn’t reliant on the transport method would be preferable.
Plus we increased the Java Max mem from 512MB to 1024MB.
Also I’m relatively new to wM, so please don’t blind me techno speak :slight_smile:

Thanks in advance for any help,
Mark.

2

Hi Boothy,

Let me get this thing straight, you just want to find out the size of the stream or Byte right ? if that is the case you can create a java service and use the available() methods which takes stream as an input and returns the size of the stream. You can find this methods in FileInputStream class.

3

Well I’m not a Java person, so I’ve no idea what any of that meant, but I know a man who is, so I’ll pass your response on to him and so what he says.

Thanks for the reply.

4

There’s a Java class called java.io.File, which contains a method called length(). If your files are always stored on disk, you could create a little Java service that takes a file name as input, for example, and uses the method above to return the size of the file in bytes.

The available() method from the InputStream class is often reliable enough to tell you whether a file is empty or not. However, I wouldn’t rely on it to tell you the exact size of the file, especially if that’s driving any type of invoicing. By definition, the available() method “… returns the number of bytes that can be read (or skipped over) from this input stream without blocking by the next caller of a method for this input stream.”

  • Percio
5

The files may or may not be on disk depending on interface, so i’m really after something generic, this isn’t reliant on the files residing on disk, as that won’t always be the case.

At the moment we can’t even get Java to compile from with Wm Developer. Wm doesn’t seem to come preinstalled with Java SDK, or any means of installing it via the Wm installer. We’ve install SDK ourselves, and followed the settings in the installation guide to change ‘watt.server.compile’ and ‘watt.server.compile.unicode’ to point to the compilers, which now run, but can’t compile anything, getting lots of ‘cannot resolve symbol’ errors. I think this is related to class paths, but the installation guide doesn’t mention anything about how to add them?

Seems that the Java SDK installation instructions are incomplete!

Mark

6

Mark,

Given that the length of the files will drive billing, I’d assume the files will be archived, right? If so, where will they be archived? Maybe you can get the length of the files from there.

Regarding your “cannot resolve symbol” error, go ahead and post the entire error. Maybe we can help. My guess is that you’re not importing a class that needs to be imported.

  • Percio
7

Hi Mark,

Regarding java service compilation just open the server.bat (windows) or server.sh (linux etc.) file (based on the operating system on which IS installed) with editor either notepad or wordpad and set the java installation directory variable to your jdk installed path. Restart IS server.

i think your java services would get compile and should work fine.

Best Regards
Rishi

8

I think TN is a good candidate for your task. If you use TN as your dispatcher, you can check the content length in the processing services and log the length, partnerID and etc. into a database table. Then you can run daily report out of that table. Since this is for billing, TN transaction log will satisfy both auditors and customers’ AP department (non-repudiation!).