First of all, a little bit of context. I have a flow service in charge for taking csv files and performing some actions on them(extract some data basically). The service is triggered as soon as the file is located in a specific folder. Once I retrieved all I need from the file, it is moved to another folder and zipped. As I didn’t know how to use WmRoot.server.replicator:zip, I use a java service to zip my file (NB: I didn’t write the code but found it on my company repo and I can’t contact the author).
And here’s the issue, as long as I deliver my files one by one, it works correctly but if I try to deliver several files at the same time, I got this error for the 1st file to process:
com.wm.app.b2b.server.ServiceException: 2.null
at ma.sap.aladin.catalog.in.priv.utils.zip(utils.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.wm.app.b2b.server.JavaService.baseInvoke(JavaService.java:411)
......
I can’t figure why it happens even if I have a little intuition that it should be a thread/stream issue.
here’s the code I use to zip the file:
import com.wm.data.*;
import com.wm.util.Values;
import com.wm.app.b2b.server.Service;
import com.wm.app.b2b.server.ServiceException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import com.wm.data.IData;
import com.wm.data.IDataCursor;
import com.wm.data.IDataUtil;
public final class zip_SVC
{
/**
* The primary method for the Java service
*
* @param pipeline
* The IData pipeline
* @throws ServiceException
*/
public static final void zip(IData pipeline) throws ServiceException {
IDataCursor pipelineCursor = pipeline.getCursor();
String targetFolder = IDataUtil.getString( pipelineCursor, "targetFolder" );
String fileName = IDataUtil.getString( pipelineCursor, "fileName" );
pipelineCursor.destroy();
// pipeline
if (targetFolder.charAt(targetFolder.length()-1)!='\\') targetFolder+="\\";
ZipOutputStream zipOut = null;
FileOutputStream fos = null;
FileInputStream fis = null;
String pathPieces[]=fileName.split("\\\\");
String nameWoPath=pathPieces[pathPieces.length-1].substring(12);
String zipName=targetFolder+nameWoPath+"_zip.zip";
try {
fos = new FileOutputStream(zipName);
zipOut = new ZipOutputStream(fos);
File fileToZip = new File(fileName);
fis = new FileInputStream(fileToZip);
ZipEntry zipEntry = new ZipEntry(fileToZip.getName());
zipOut.putNextEntry(zipEntry);
byte[] bytes = new byte[1024];
int length;
while((length = fis.read(bytes)) >= 0) zipOut.write(bytes,0,length);
} catch (Exception e) {
e.printStackTrace();
throw new ServiceException("1."+e.getMessage());
} finally {
try {
zipOut.finish();
zipOut.flush();
fis.close();
fos.close();
zipOut.close();
} catch (Exception e) {
e.printStackTrace();
throw new ServiceException("2."+e.getMessage());
}
}
// pipeline
IDataCursor pipelineCursor_1 = pipeline.getCursor();
IDataUtil.put( pipelineCursor_1, "zipName", zipName);
pipelineCursor.destroy();
}
If someone has an idea, it would be very helpful.
Thanks !
EDIT: I finally find the solution, it seems that in some cases the java services was called before the end of the previous treatment which means it can retrieve the location of the file and cause the NullPointerException. I simply fix it with a Thread.sleep(500)just before zipping the file
WmRoot.server.replicator:zip ist not meant to be used directly by custom code and has a different scope than just zipping files.
You can check the Tech Community for the PSUtilities package which has some zip services included or you can rely on zip utilities provided by your OS by using pub.flow:executeOSCommand.
See IS Built-In-Services Reference for further informations on the executeOSCommand service.
you don´t need to use the package PSUtilities directly but you can check how the zip services were implemented to see if you can modify your code somehow to achieve what you want.
I would suggest avoiding executeOSCommand. The example in PSUtilities is the thing to follow.
We too have a policy of never using PSUtilities directly. For services we find useful, we copy them to our own “Public” package, edit them as desired, and use them. As @Holger_von_Thomsen notes, you can at least just reference PSUtilities to see how to do the work.
The code you posted has no mechanism for multiple files. Are you calling the Java service repeatedly, once for each file? The exception is thrown in the try of the finally block – something bad happening during finish/flush/close. Alas, no exception message available. Might need to change that block so you can what the exception is.
Hi reamon,
Thanks for your answer, indeed the service is called once for each file.
For now I didn’t succeed to get more infos about the exception. What I know is it’s triggered on the 1st file to be process.
I also find that if I configure the trigger responsible for calling the service to retry after failure, the process will fail on the 1st file as expected but on the retry it works.
This sounds like a multi-threading item. Is that what is happening?
Beware timing solutions being solved by sleep(). Can have unintended consequences.
Would it work if the code you’re using was updated to support multiple filenames at one time? Would avoid multiple open, write, close cycles. And timing issues.
But if the addition of files to the archive is indeed async and files that are to be placed into the archive can “arrive” at any time, including at the same time via different threads, then you’ll want to implement something that supports that.
