Filtering out a CSV flat file by distinct column(field)


I am trying to filter a CSV flat file based on a particular field in the flat file. I have a command that works fine when executed from Unix system. But when I try to invoke it from Webmethods it fails with an exit value of 1. No error or output message. Below is the command. Please suggest anything that I can try out. I am using the executeOSCommand available in ps util.

awk ‘{a[$1]++}a[$1]<2’ file > new.file

Hello Sachin,
For invoking the executeOsCommand service you need to set the parameters “allowedOSCommands” and “allowedWorkingDirectories” in the OSCommands.cnf file and reload the WmPublic Package or restart the IS.

Hope this helps.

Anwit Daityari

Hey Sachin,

PS utilities are created by webMethods professionals and if you are on wM version 8.0 and above I will suggest you not to use PS utils for executing executeOSCommand.

Use the service pub.utils:executeOSCommand which available in WmPubic. If you think the service does not work as per the design you can contact SAG global support. I don’t think SAG will provide support for PS utils.

So choice is yours :lol:

With some help I was able to write a custom executeOSCommand service. When I try the awk command I am getting the below error

awk “{a[$1]++}a[$1]<2” file > new.file

awk: (FILENAME=file FNR=677) fatal: cannot open file `">"’ for reading (No such file or directory)

But I am able to see the correct output in the outputMessage of the service. It is just not writing to a file. I can capture the output and write to a file but the output could be very large and cause the server to go out of memory.

Share your code here. And also did you use the service in WmPublic, did it help you?

Try this

awk "{a[$1]++}a[$1]<2" file >> new.file

Or provide the directory path for eg. /tmp/mahesh/file.txt

awk "{a[$1]++}a[$1]<2" /tmp/mahesh/file.txt >> /tmp/mahesh/file_1.txt