Unable to Uncompress the files

Guest · May 3, 2002, 6:09pm

Hi Rob,

I am using your ZIP & Unzip services to compress and uncompress the files in my flows. If I use your zip am not able to uncompress those files. If I use winzip to compress and send those file to your uncompress its able to uncompress those files. Any idea why its happening. You mentioned in your unzip service that it can’t handle large files, how to handle large files. Do I need to use some other streams to handle large files?. Please help

Thanks for your help in this.

reamon · May 3, 2002, 9:06pm

There is an error in the zipFiles code. It isn’t setting the ZipEntry sizes. I’ll post an update when I can.

The way to handle large files is to not pull the entire file, compressed or uncompressed, into memory–which is what the zipFiles and unZipFiles services do. There are couple of ways to do this.

Do everything in a single Java service. For zipping, modify zipFiles to accept a target zip filename. Instead of hooking the ZipOutputStream to a ByteArrayOutputStream, hook it to a FileOutputStream (which you might front with a BufferedOutputStream). Then you create a loop for each file to zip, read a block, write a block, etc.

For unzipping, change unZipFiles to write the files instead of creating an array of entries.

The original zipFiles was written for someone who want to send the resulting byte stream over a comm link, so that’s why it returns a byte array instead of writing it to disk.

For max flexibility, write services that mimic the java.util.zip classes and methods. The thing here is that you’d need to write service wrappers for java.io things too. This is probably overkill for most needs.

Hope this helps.

Guest · May 3, 2002, 9:36pm

Rob,

Thanks for your info. Well I was the one who asked the output as Byte Array. The reason is we want to send this byte array to a servlet for some reason they wanted like this. Their argument is by doing this way we can save space on the server. If we write the zip file or write the files when we unzip it needs the directory cleanup and we need to schedule a service to do this. I am not sure this is the best way to save the space on server.

But I used the same zipFiles service to zip the files when I tried to open it in WinZip it worked fine but not on PKZIP. java.util.zip is supported on the PKZIP only later version of 2.6G I downloaded a latest version but it did not work, even with the unZip service i was not able to uncompress thats why I created one more thread on this.
Thanks for you help on this.

reamon · May 3, 2002, 9:52pm

That’s right. I forgot it was you. Sorry 'bout that.

As I’m digging into the error, it may be the unzip service that has the problem. I’m checking into it…

reamon · May 6, 2002, 11:46pm

A new Zip Sample package is available to address the unzip problem.

I’m not sure about why pkzip doesn’t work.

Guest · May 13, 2002, 7:25pm

An Update to my last posting. We found an interesting thing with the PKZIP. But before going in to the pkzip stuffs, when we send out the Byte Array to Servlet the b2b is adding some tags to the byte array. Because the default output template was html. We converted the byte array to string with the same settings, now its not adding any tags to the compressed string. The additional tags were the problems cause.

The WINZIP takes the byte array as input (zip file), ignoring all other tags it looks for the byte array only. That’s why even the compressed files with some html tags were working fine with WINZIP. But in PKZIP it opens the file for uncompress and start reading from the begining of the file, because of the tags its not able to uncompress file.

Rob & all Thanks for your help.

Guest · July 19, 2002, 10:33pm

Hi Rob,

I using your updated Unzip Service to Unzip the files. I found out one problem with that. When it tries to unzip the file the result file is created with only some 256 characters after that everything is filled with spaces. But the file size of both original file which is got zipped and the extracted file which has spaces are same. I dont know how to fix this problem. I am posting that code here.

String zipFilename = “”;
ZipFile zf = null;

// pipeline
IDataHashCursor pipelineCursor = pipeline.getHashCursor();
if ( pipelineCursor.first( “zipFilename” ) )
{
zipFilename = (String) pipelineCursor.getValue();
}
pipelineCursor.destroy();

Vector entryVector = new Vector();

try
{
zf = new ZipFile(zipFilename);

java.util.Enumeration entries = zf.entries(); 

 
ZipEntry ze; 
while(entries.hasMoreElements()) 
{ 
 ze = (ZipEntry)entries.nextElement(); 
 byte&#91;] b = null; 

    if(!ze.isDirectory()) 
    { 
        long entrySize = ze.getSize();

int count;
if(entrySize > Integer.MAX_VALUE)
throw new ServiceException(ze.getName() + " expands to " + ze.getSize() + " and is too large for this service.");
b = new byte[(int)entrySize];
InputStream is = zf.getInputStream(ze);
is.read(b, 0, b.length);

IData entry = IDataFactory.create();
IDataHashCursor entryCursor = entry.getHashCursor();
entryCursor.last();
entryCursor.insertAfter(“fileName”, ze.getName());
entryCursor.last();
entryCursor.insertAfter(“isDirectory”, (new Boolean(ze.isDirectory())).toString());
entryCursor.last();
entryCursor.insertAfter(“outputBytes”, b);
entryCursor.destroy();
entryVector.add(entry);
}

}
catch (java.io.IOException ioe)
{
throw new ServiceException("IOException: " + ioe.getMessage());
}
finally
{
if(zf != null)
{
try
{
zf.close();

}
catch (IOException ioe)
{
// Ignore any exceptions from above close
}
zf = null;
}
}

// pipeline
IDataHashCursor pipelineCursor_1 = pipeline.getHashCursor();
pipelineCursor_1.last();
pipelineCursor_1.insertAfter( “entries”, entryVector.toArray());
pipelineCursor_1.destroy();

Thanks,
Muru.

Guest · July 22, 2002, 8:50pm

Hi Rob,

I think I fixed that problem. I used BufferedInputStream to read the contents of byte array instead InputStream. Here is the new code its works fine now.

String zipFilename = “”;
ZipFile zf = null;

// pipeline
IDataHashCursor pipelineCursor = pipeline.getHashCursor();
if ( pipelineCursor.first( “zipFilename” ) )
{
zipFilename = (String) pipelineCursor.getValue();
}
pipelineCursor.destroy();

Vector entryVector = new Vector();

try
{
zf = new ZipFile(zipFilename);

java.util.Enumeration entries = zf.entries(); 

 
ZipEntry ze; 
while(entries.hasMoreElements()) 
{ 
 ze = (ZipEntry)entries.nextElement(); 
 byte&#91;] b = null;

BufferedInputStream is = null;

    if(!ze.isDirectory()) 
    { 
        long entrySize = ze.getSize();

//if(entrySize > Integer.MAX_VALUE)
//throw new ServiceException(ze.getName() + " expands to " + ze.getSize() + " and is too large for this service.");
b = new byte[(int)entrySize];
//InputStream is = zf.getInputStream(ze);
is = new BufferedInputStream(zf.getInputStream(ze));
is.read(b, 0, b.length);

IData entry = IDataFactory.create();
IDataHashCursor entryCursor = entry.getHashCursor();
entryCursor.last();
entryCursor.insertAfter(“fileName”, ze.getName());
entryCursor.last();
entryCursor.insertAfter(“isDirectory”, (new Boolean(ze.isDirectory())).toString());
entryCursor.last();
entryCursor.insertAfter(“outputBytes”, b);
entryCursor.destroy();
entryVector.add(entry);
}

}
catch (java.io.IOException ioe)
{
throw new ServiceException("IOException: " + ioe.getMessage());
}
finally
{
if(zf != null)
{
try
{
zf.close();

}
catch (IOException ioe)
{
// Ignore any exceptions from above close
}
zf = null;
}
}

// pipeline
IDataHashCursor pipelineCursor_1 = pipeline.getHashCursor();
pipelineCursor_1.last();
pipelineCursor_1.insertAfter( “entries”, entryVector.toArray());
pipelineCursor_1.destroy();

Thanks for your help,
Muru.

reamon · July 22, 2002, 11:45pm

Muru: This is quite strange. I have duplicated this error but I’m confused as to what the exact cause is. Using BufferedInputStream should have no impact on proper operation. BufferedInputStream is derived from InputStream and simply buffers the incoming data.

I’m checking into this and will post what I find. Of course if someone sees what boneheaded thing I did in the code, please let me know.

reamon · July 23, 2002, 1:43am

The “contract” for InputStream.read(buf, offset, length) isn’t what I thought it was (my C up-bringing has bitten me again). The contract does not provide for filling the specified buf with length bytes in all but EOF or error situations as I assumed. BufferedInputStream does.

Sorry for the bug. I’ll post a fix to the sample package soon.

Thanks for pointing this out and working through it.

Guest · July 23, 2002, 2:51am

Hi Rob,
I am not clear on this part of your explanation.

“The contract does not provide for filling the specified buf with length bytes in all but EOF or error situations”

Though I understand the above code is working with BufferedInputStream but not with InputStream. My understanding of the difference between the above two was in terms of buffering before actually performing the I/O operation.

Please explain.
Thanks.

reamon · July 23, 2002, 3:55am

I’ll rephrase (my initial wording was rather terrible):

The InputStream.read(buf, offset, length) contract says that implementers can return before filling buf with length bytes. Multiple calls to read may be required if you want to fill a buf.

BufferedInputStream.read(buf, offset, length) performs multiple reads on the underlying stream in order to fill buf with length bytes. It will read until buf is filled, EOF or an exception occurs.

You can look at the code for the various input stream objects. The default stream that ZipFile.getInputStream (which is an object of a inner class named ZipFileInputStream) returns does not fill bufs completely during read. It appears to read the first 512 bytes (256 chars) from the zip file, inflate those and return.

Let me know if this explanation isn’t sufficient.

Guest · July 23, 2002, 4:32am

I am all clear now !! Thanks a lot for making me understand it.

Thus, it depends on implementors of InputStream to fill up the buffer or not, which is exactly what is said in API documentation… “Subclasses are encouraged to provide a more efficient implementation of this method”

Thanks once again.

One more question still: “object of a inner class named ZipFileInputStream”

How did you arive at this information?? I was not able to deduce this piece of information from the API itself??

Thanks a lot.

Saurabh

reamon · July 23, 2002, 4:49am

The source for the core Java packages can be downloaded from Sun.

Guest · July 23, 2002, 5:24am

Oh !! Thanks a lot for this Info.

Saurabh.

Guest · January 23, 2003, 9:56pm

Hi Rob,

I am using your Unzip service to uncompress the files in my flows.I created the java service.Now, how can i define the Input/Output (String type or…)?
How can i test the service?
I don´t know how i get the zip file, if i have to put it in some special folder, or send it to my Unzip service.
Thanks!

ricksite · April 24, 2004, 1:06am

Hi Rob

does it work with .gz files?

thanx

Asoka · October 3, 2006, 4:34pm

Hi
I need to upzip a zip file, which will have an excel spreadsheet
the services written in webMethods are not working
could any one help me out

I appreciate your early help

Thanks
Ashok

reamon · October 3, 2006, 9:17pm

Can you elaborate on “not working?”

How to request help to maximize useful responses

Asoka · October 4, 2006, 3:39am

When i copy and paste the services (ex: PSUtilities:unzip )into another java service, it is not compiling properly, it is taking the imports of other java service in that folder, it’s now working properly
if i use copy file also, when i save it is showing irrelevant errors which are not this java service errors
will PSUtilities work for GZip too
Thanks,
Ashok