Improperly encoded response, < sign getting converted into < but not > sign

Hi Experts,

Our client receiving the response mentioned attachment screenshot(special characters in the xml not getting getting displayed in post, so attached the screenshot )when they call webMethods WSDL

Please find screenshot for the actual response I am getting and expected response that should come

I have got actual response response by converting document to xmlstring using built-in flow service pub.xml.documentToxmlString.In pub.xml.documentToxmlString pipeleine I trie[img]d hardcoding encode variable both true and false.
I tried using different encoding styles, still I am facing same issue.

I have observed response xml in server log. There I could see improperly encoded xml as mention above. I am confused whether this is coding issue or integration server behaving like this.

We are using webMethods 9.9

Please help me to resolve the issue.

Hi Manikumar,

sounds strange.

Might be a good idea to open an incident via empower to get support team involved.

Regards,
Holger

Hi Holger,

Today we have raised incident.Waiting for their response.

Thanks,
Manikumar

Hi All,

Got the response from SoftwareAG. Justification to the issue they provided is

When tested from soapuI the repsonse is properly parsed as shown below(Ignore the raw output, what we need to look is if soapui was able to parse the respone).

<soapenv:Envelope xmlns:soapenv=“http://schemas.xmlsoap.org/soap/envelope/”>
soapenv:Body
<ser-root:encode_testResponse xmlns:ser-root=“http://server/Test.Encode_Test” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance”>
<![CDATA[ Sema 0]]>
</ser-root:encode_testResponse>
</soapenv:Body>
</soapenv:Envelope>

I don’t see any issue here atleast from product side.
Any issue with your end consumer should be not because of the respone generated with >.Please work with them

The < must be encoded to prevent the data from being interpreted as markup.
It is not necessary to encode the > because there is no matching < to confuse the parser.
The XML spec allows this, and virtually every encoder works this way.
That is how the response has been generated in Integration server from beginning.

Regards,
Manikumar

Test.zip (5.25 KB)

Manikumar,

Are you trying to send back an XML string as the response of your web service? If so, why? SOAP is already XML-based. Simply define the output of your web service as a regular document and let the IS convert it into the necessary SOAP response XML.

For example, you could set this as the output of your web service to get those same fields returned to the consumer:

Response (document type)

  • Status (document type)
    – Code (string)
    – Message (string)
    – Detail (string)

Percio

Hi,

If you really need to send XML to a web service, remember the tools you use will parse the contents for presentation.

So, you might find you have a different result when you check the output using a browser, SoapUI or another tool.

Always check the RAW result (SoapUI) or SOURCE (browser) to be absolutely sure you are getting the expected results.

Reinforcing what Percio said, transporting XML objects in web services is an anti-pattern (or code smell if you prefer) except in very, very, particular situations (mainly, the code is just a transport interface).

I’ve seen countless examples of this kind of situation and it usually boils down to bad decisions; if you can, break out of them: no reasoning will hold under thoughtful scrutiny (and the code will only get worse).

Best Regards and, good luck (you’ll need it).

Hi Percio and Gerardo,

Thanks for your suggestions.

I have tried to send the response as a regular webMethods document. Document format I have constructed is

–Result (document type)
—Response(document type)
---- -Status (document type)
----- ReturnCode (string)
----- ErrorMessage (string)
----- Details (string)

Responses captured from soapUI are.
raw xml:

HTTP/1.1 200 OK
Set-Cookie: ssnid=b5735f10b14e19159515abf3e468f780; path=/; HttpOnly
Content-Type: text/xml; charset=UTF-8
Content-Encoding: gzip
Content-Length: 409

<?xml version='1.0' encoding='UTF-8'?>

<soapenv:Envelope xmlns:soapenv=“http://schemas.xmlsoap.org/soap/envelope/”>
soapenv:Header
<tns:SoapAuthHeader xmlns:tns=“https://namespace.com”>
tns:UserNameuser</tns:UserName>
tns:Passwordpwd</tns:Password>
</tns:SoapAuthHeader>
</soapenv:Header>
soapenv:Body
<ser-root:SubmitRequestResponse xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” xmlns:ser-root=“https://namespace.com”>


0
Submit Application Successful




</ser-root:SubmitRequestResponse>
</soapenv:Body>
</soapenv:Envelope>

Noraml xml:

<soapenv:Envelope xmlns:soapenv=“http://schemas.xmlsoap.org/soap/envelope/”>
soapenv:Header
<tns:SoapAuthHeader xmlns:tns=“https://namespace.com”>
tns:UserNameuser</tns:UserName>
tns:Passwordpwd</tns:Password>
</tns:SoapAuthHeader>
</soapenv:Header>
soapenv:Body
<ser-root:SubmitRequestResponse xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” xmlns:ser-root=“https://namespace.com”>


0
Submit Application Successful




</ser-root:SubmitRequestResponse>
</soapenv:Body>
</soapenv:Envelope>

Client receiving the raw xml. But client expecting the response in different format with lt; and gt; tags.

Attached the client expected response.

Please look into the attached xml file and help if there are any options to change the format.

Thanks & Regards,
Manikumar

expected response format.xml (726 Bytes)

Hi Manikumar,

Result should be of type String in the WSDL.

Use pub.xml:documentToXmlString to convert Response document to a XML string.
This String can be encoded with the service pub.string:HTMLEncode.

Map this encoded string to the Result fielld of the Soap Request.

Regards,
Holger

Hi Holger,

I already followed the steps you have suggested.

If I use HTMLEncode of string < and > signs will be converted to &lt and &gt but if we observer in raw format &lt and &gt converted to &lt and &gt.

Please find the attached responses captured from soapUI. Please help me if you have any other suggestions. I am trying to resolve the issue from past one month.

Thanks & Regards,
Manikumar
after using HTMLEncode.txt (1.58 KB)

Hi Manikumar,

can you provide a screenshot or outline of your flow service?

This will help us to understand what is happening.

raw format looks like HTMLEncode being used twice (either manually or automatically by SOAP handler).

I have to agree with Percio and Gerardo, that this sounds like some bad design decisions which happened in the past.

Regards,
Holger

Hi Holger,

Attached the document having the screenshots and sample responses.

Thanks & Regards,
Manikumar
issue.docx (97.9 KB)

My question to you is: why?

Like Gerardo and I have tried to explain to you, including an encoded XML string in the SOAP response is an anti-pattern. The response that you pasted from when you used a regular webMethods document as an output is what a SOAP response is supposed to look like. You are the provider of the web service, are you not? You should be able to dictate what the service returns, and the consumer should comply according to the WSDL, not the other way around.

Percio

By the way, I’m catching up on the other messages, if you do return the response as a string (which you shouldn’t - read above), you don’t have to do the encoding yourself. You simply convert it to a string (i.e. documentToXMLString) and the IS will do the encoding when adding the string to the SOAP response.

Percio

For the record, we have a similar issue. We have a SOAP document, containing a ‘transaction results’, that itself contains a separate XML transaction (acknowledge or reject message) based on an industry standard. Quite why this needed to be embedded in a separate results message is lost to time. This is a legacy application that is being replaced and we are trying to replicate it without providing a knock on effect to existing consumers other than a URL change. That being said, it is perfectly valid within the XML spec to embed XML within XML.

Our SOAP message would look something like this (I’ve simplified the content dramatically which is quite large, for illustrative purposes)…

<?xml version="1.0" encoding="UTF-8"?>

<soapenv:Envelope xmlns:soapenv=“http://schemas.xmlsoap.org/soap/envelope/”>
soapenv:Body
<tns:getProductsResponse xmlns:tns=“http://tempuri.org/”>
tns:getProductsResult<?xml version="1.0" encoding="UTF-8"?>SUCCESS</tns:getProductsResult>
</tns:getProductsResponse>
</soapenv:Body>
</soapenv:Envelope>

This is obviously incorrect and I would expect webMethods to convert the response to…

<?xml version="1.0" encoding="UTF-8"?>

<soapenv:Envelope xmlns:soapenv=“http://schemas.xmlsoap.org/soap/envelope/”>
soapenv:Body
<tns:getProductsResponse xmlns:tns=“http://tempuri.org/”>
tns:getProductsResult<?xml version="1.0" encoding="UTF-8"?>SUCCESS</tns:getProductsResult>
</tns:getProductsResponse>
</soapenv:Body>
</soapenv:Envelope>

However it is returning….

<?xml version="1.0" encoding="UTF-8"?>

<soapenv:Envelope xmlns:soapenv=“http://schemas.xmlsoap.org/soap/envelope/”>
soapenv:Body
<tns:getProductsResponse xmlns:tns=“http://tempuri.org/”>
tns:getProductsResult<?xml version="1.0" encoding="UTF-8"?>SUCCESS</tns:getProductsResult>
</tns:getProductsResponse>
</soapenv:Body>
</soapenv:Envelope>

with only the < being encoded.

This according to the (purported) software AG response above, is perfectly valid and it certainly validates in XML SPY. It does however look a bit untidy (not a problem in itself) and some of our consumers are saying that their applications can’t parse it.

I’ve tried embedding it in CDATA e.g.

<?xml version="1.0" encoding="UTF-8"?>

<soapenv:Envelope xmlns:soapenv=“http://schemas.xmlsoap.org/soap/envelope/”>
soapenv:Body
<tns:getProductsResponse xmlns:tns=“http://tempuri.org/”>
tns:getProductsResultSUCCESS]]></tns:getProductsResult>
</tns:getProductsResponse>
</soapenv:Body>
</soapenv:Envelope>

but this doesn’t seem to work. It returns the CDATA correctly if I consume the WSDL directly from the integration server, however if I consume the virtualised service through mediator then webMethods is converting it back to

<?xml version="1.0" encoding="UTF-8"?>

<soapenv:Envelope xmlns:soapenv=“http://schemas.xmlsoap.org/soap/envelope/”>
soapenv:Body
<tns:getProductsResponse xmlns:tns=“http://tempuri.org/”>
tns:getProductsResult<?xml version="1.0" encoding="UTF-8"?>SUCCESS</tns:getProductsResult>
</tns:getProductsResponse>
</soapenv:Body>
</soapenv:Envelope>

It’s all very frustrating.

Again, it would appear that there is nothing technically wrong with it. It should be able to be parsed by any XML parsing engine and certainly tools like SOAP UI ‘prettify’ it to restore the CDATA construct. However when you are dealing the with legacy code of a multitude of organisations that you have no technical authority over, sometimes you just want it to respond with what we’ve actually generated rather than what webMethods thinks you want.

This is a correction of my original post, as the forum software corrected the encoding of the XML making it read incorrectly :slight_smile:

For the record, we have a similar issue. We have a SOAP document, containing a ‘transaction results’, that itself contains a separate XML transaction (acknowledge or reject message) based on an industry standard. Quite why this needed to be embedded in a separate results message is lost to time. This is a legacy application that is being replaced and we are trying to replicate it without providing a knock on effect to existing consumers other than a URL change. That being said, it is perfectly valid within the XML spec to embed XML within XML.

Our SOAP message would look something like this (I’ve simplified the content dramatically which is quite large, for illustrative purposes)…

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
	<soapenv:Body>
		<tns:getProductsResponse xmlns:tns="http://tempuri.org/">
			<tns:getProductsResult><?xml version="1.0" encoding="UTF-8"?><status>SUCCESS</status></tns:getProductsResult>
		</tns:getProductsResponse>
	</soapenv:Body>
</soapenv:Envelope>

This is obviously incorrect and I would expect webMethods to convert the response to…

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
	<soapenv:Body>
		<tns:getProductsResponse xmlns:tns="http://tempuri.org/">
			<tns:getProductsResult>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;status&gt;SUCCESS&lt;/status&gt;</tns:getProductsResult>
		</tns:getProductsResponse>
	</soapenv:Body>
</soapenv:Envelope>

However it is returning….

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
	<soapenv:Body>
		<tns:getProductsResponse xmlns:tns="http://tempuri.org/">
			<tns:getProductsResult>&lt;?xml version="1.0" encoding="UTF-8"?>&lt;status>SUCCESS&lt;/status></tns:getProductsResult>
		</tns:getProductsResponse>
	</soapenv:Body>
</soapenv:Envelope>

with only the < being encoded. This according to the (purported) software AG response above, is perfectly valid and it certainly validates in XML SPY. It does however look a bit untidy (not a problem in itself) and some of our consumers are saying that their applications can’t parse it.

I’ve tried embedding it in CDATA e.g.

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
	<soapenv:Body>
		<tns:getProductsResponse xmlns:tns="http://tempuri.org/">
			<tns:getProductsResult><![CDATA[<?xml version="1.0" encoding="UTF-8"?><status>SUCCESS</status>]]></tns:getProductsResult>
		</tns:getProductsResponse>
	</soapenv:Body>
</soapenv:Envelope>

but this doesn’t seem to work. It returns the CDATA correctly if I consume the WSDL directly from the integration server, however if I consume the virtualised service through mediator then webMethods is converting it back to

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
	<soapenv:Body>
		<tns:getProductsResponse xmlns:tns="http://tempuri.org/">
			<tns:getProductsResult>&lt;?xml version="1.0" encoding="UTF-8"?>&lt;status>SUCCESS&lt;/status></tns:getProductsResult>
		</tns:getProductsResponse>
	</soapenv:Body>
</soapenv:Envelope>

It’s all very frustrating.

Again, it would appear that there is nothing technically wrong with it. It should be able to be parsed by any XML parsing engine and certainly tools like SOAP UI ‘prettify’ it to restore the CDATA construct. However when you are dealing the with legacy code of a multitude of organisations that you have no technical authority over, sometimes you just want it to respond with what we’ve actually generated rather than what webMethods thinks you want.