Java Service to Zip/Unzip data in Memory

, ,
1

Hi,
Would anyone have a java service/s they’re willing to share which will Zip (compress) data in memory (without having to write the data to a file first) and Unzip (decompress) data in memory also ??
Ideally, what I want to do is to pass a text STRING (representing a file that has already been read into memory) into a service and that service will pass back compressed BYTEs. This data would then be Base64Encoded and sent via SOAP/WebService to a TradingPartner.
Then, on the flipside, for (Base64Encoded) data received from TradingPartner, the data would be decoded first then passed as compressed BYTEs to a service that would return a text STRING.

 I'm not all that crash hot with writing Java from scratch as I have 20+ years of COBOL/CICS/DB2 dev experience and 6/7 years of webMethods admin/dev experience.

 I've already coded what I need using the Zip/Unzip services in the PSUtilities package, but, I find it's rather ungainly having to write the text string to a file just to ZIP it up.  Then having to write the ZIPPED data to a file just to Unzip it..

Cheers,
David Krivohlavy

2

Post moved to Flow and Java services section.

I think in your case GZip will be more appropriate as there is only one file(string) which needs to be compressed. There are lot of resources out there that can help you with compress/decompress functionality in Java. I would suggest going through this article, it has some really nice explaination about Java APIs and difference between Zip and GZip.

http://www.oracle.com/technetwork/articles/java/compress-1565076.html


// pipeline
		IDataCursor pipelineCursor = pipeline.getCursor();
		String	inString = IDataUtil.getString( pipelineCursor, "inString" );
		pipelineCursor.destroy();
		 
		ByteArrayOutputStream baos = new ByteArrayOutputStream();
		try {
			GZIPOutputStream gzip = new GZIPOutputStream(baos);
			gzip.write(inString.getBytes());
		    gzip.close();
		IDataUtil.put( pipelineCursor, "result", baos.toByteArray());
		} catch (IOException e) {
			IDataUtil.put( pipelineCursor, "errorMessage", e.toString());
		}
		pipelineCursor.destroy();

You will need these in the imports section,

import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
3

Thanks Akshith,
What would be the code to decompress a byte array ??

Cheers,
David

4

Hi,
The Bank wants the files in ZIP format, which, I’ve been able to modify Akshith’s GZIP example as such:


	IDataCursor idcPipeline = pipeline.getCursor();
	String	inString = IDataUtil.getString( idcPipeline, "inString" );
	String	inFile = IDataUtil.getString( idcPipeline, "inFile" );
	idcPipeline.destroy();
		 
	ByteArrayOutputStream baos = new ByteArrayOutputStream();
	try {
		ZipOutputStream zos = new ZipOutputStream(baos);
		ZipEntry ze= new ZipEntry(inFile);
		zos.putNextEntry(ze);
		zos.write(inString.getBytes());
		zos.close();
		IDataUtil.put( idcPipeline, "result", baos.toByteArray());
		} 
	catch (IOException e) {
		IDataUtil.put( idcPipeline, "errorMessage", e.toString());
		}
	idcPipeline.destroy();

I find that I’m having difficulty with the unzip part of it. Is anyone able to assist me with the In-Memory UNZIP function ??

Thanks,
David

5

How will you be receiving the data? Is it an (MTOM)attachment to the SOAP or will the data come in as base64 encoded string in a field in the SOAP response?

6

David,

Are you receiving the ZIP file in the SOAP response call and in which encoding format they are sending?

Can you put that receiving snippet here?

HTH,
RMG

7

Hi,
Yes, data will be Base64 Encoded string received as a response from a SOAP request call. But, what will be passed to the java service will be object returned from pub.string.base64Decode. We’re changing transmission methods with the bank as part of the TransBanking conversion from Tibco to webMethods.

Also, are there any good reading for SSL and Certificates as I've just been made aware that this is also a requirement.

Cheers,
David

8

“Also, are there any good reading for SSL and Certificates as I’ve just been made aware that this is also a requirement”

Please review the ISAdministrators guide on the certs/keystore configuration setup for HTTP/S

Search this forum on the keystore (there are some threads discussed on wMUsers.com forum site)

HTH,
RMG

9

What kind of security are we talking about? Is this WS-Security? or just plain old https? You could use both of them in conjunction to secure web services. I would recommend looking at the Securing Web Services (WS-Security) section in the web services users guide.


IDataCursor pipelineCursor = pipeline.getCursor();
Object	inputBytes = IDataUtil.get( pipelineCursor, "inputBytes" );
pipelineCursor.destroy();		
		
ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream((byte[]) inputBytes));
ZipEntry entry = null;
try {
	while ((entry = zis.getNextEntry()) != null) {
		//The Assumption being there will be only one file in this zip.				
		InputStreamReader is = new InputStreamReader(zis);
		StringBuilder sb=new StringBuilder();
		BufferedReader br = new BufferedReader(is);
		String read = br.readLine();
		
		while(read != null) {
		    sb.append(read);
		    read =br.readLine();
		}
		IDataUtil.put(pipelineCursor, "decompressedData", sb.toString());
	}
} catch (FileNotFoundException e) {
		IDataUtil.put( pipelineCursor, "errorMessage", e.toString());
	} catch (IOException e) {
		IDataUtil.put( pipelineCursor, "errorMessage", e.toString());
	}
10

Sweet… Thanks a lot mate !!! Works a dream…

David

11

Hi,
It appears that the Zip/Unzip routines are stripping the CR/LF characters from the data. Would anyone know what I would need to modify to perserve the CR/LF characters in the data ??

	IDataCursor idcPipeline = pipeline.getCursor();
	String	inString = IDataUtil.getString( idcPipeline, "inString" );
	String	inFile = IDataUtil.getString( idcPipeline, "inFile" );
	idcPipeline.destroy();
		 
	ByteArrayOutputStream baos = new ByteArrayOutputStream();
	try {
		ZipOutputStream zos = new ZipOutputStream(baos);
		ZipEntry ze= new ZipEntry(inFile);
		zos.putNextEntry(ze);
		zos.write(inString.getBytes());
		zos.close();
		IDataUtil.put( idcPipeline, "result", baos.toByteArray());
		} 
	catch (IOException e) {
		IDataUtil.put( idcPipeline, "errorMessage", e.toString());
		}
	idcPipeline.destroy();

	IDataCursor idcPipeline = pipeline.getCursor();
	Object	inputBytes = IDataUtil.get( idcPipeline, "inData" );
	idcPipeline.destroy();		
		
	ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream((byte[]) inputBytes));
	ZipEntry ze = null;
	try {
		while ((ze = zis.getNextEntry()) != null) {
			//The Assumption being there will be only one file in this zip.				
			InputStreamReader is = new InputStreamReader(zis);
			StringBuilder sb=new StringBuilder();
			BufferedReader br = new BufferedReader(is);
			String fName = ze.getName();
			String read = br.readLine();
		
			while(read != null) {
			    sb.append(read);
			    read = br.readLine();
				}
			IDataUtil.put(idcPipeline, "outFileName", fName.toString());
			IDataUtil.put(idcPipeline, "outString", sb.toString());
			}
		} 
	catch (FileNotFoundException e) {
		IDataUtil.put( idcPipeline, "errorMessage", e.toString());
		} 
	catch (IOException e) {
		IDataUtil.put( idcPipeline, "errorMessage", e.toString());
		}
12

Looks like using the String builder and using br.readline() was causing that. Use whatever encoding you want in place of UTF-8 and try this


IDataCursor pipelineCursor = pipeline.getCursor();
Object	inputBytes = IDataUtil.get( pipelineCursor, "inputBytes" );
pipelineCursor.destroy();		
		
ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream((byte[]) inputBytes));
ZipEntry entry = null;
final int BUFFER = 2048;
int count = 0;
byte data[] = new byte[BUFFER];
try {
	while ((entry = zis.getNextEntry()) != null) {
		//The Assumption being there will be only one file in this zip.				
		ByteArrayOutputStream out = new ByteArrayOutputStream();
		while ((count = zis.read(data, 0, BUFFER)) != -1) {
			        out.write(data);
		    }       
	InputStream is = new ByteArrayInputStream(out.toByteArray());
	String decompressedData=new Scanner(is,"UTF-8").useDelimiter("/A").next().trim();
	IDataUtil.put(pipelineCursor, "decompressedData", decompressedData);
       }
} catch (IOException e) {
		IDataUtil.put(pipelineCursor, "errorMessage", e.toString());
}
13

Hi there, we have an error with an entirex service. From mainframe to server. We receive Nat6971 reason 9/ 10030015 Rpc Protocol Compression 1 not supported. My question is, the DEFAULTS=CODEPAGE DEFAULT_EBCDIC_IBM=ibm-1145 Broker attributes can affect this sevice? since we set this attributes the error begin. In production works well because the attributes are not set. Thanks

14

Gerardo,

Please open another new thread on your query…and appropriate EntireX/ESB section if available.

It’s not good practice you hack another discussion which is not related to your issue.

Thanks for your understanding:

HTH,
RMG

15

Hi Akshith,
Thanks mate, this doesn’t work as it appears to be scrambling the output somewhat. I’ll just revert back to using the PSUtilities file based unzip. The zip on the other hand works a dream and we will be using it…

Cheers,
David