Error creating REST API with encoded square brackets in JSON:API query parameters

nick_lomax24 · June 14, 2022, 9:40am

What product/components do you use and which version/fix level?

webMethods Service Designer 10.11 Fix 2

Are you using a free trial or a product with a customer license?

Free Trial

What are trying to achieve? Please describe in detail.

Trying to expose an entity in a database as RESTv2 JSON service, with the message JSON API Compliant.
Hoping to use Excel to use the JSON API compliant service to populate a spreadsheet.

Looking to emulate this type of request / response:

https://documentation.softwareag.com/webmethods/microservices_container/msc10-5/10-5_MSC_PIE_webhelp/index.html#page/integration-server-integrated-webhelp/to-config_rest_url_template_approach_5.html

Do you get any error messages? Please provide a full error message screenshot and log file.

I am trying to implement an API that uses url encoded square brackets in the query parameters with rest V2 resource and a JSON:API compliant query.
I came across this issue, as excel url encodes the square brackets in the url. Though, I think this is a valid request

http://localhost:5555/restv2/product/filter?sort=-list_price&page[limit]=2&page[number]=0

with HTTP header Accept = application/vnd.API+json

GET /restv2/product/filter
Accept: application/vnd.API+json
cache-control: no-cache
Postman-Token: 24a21b4c-8e38-4690-8e78-c1cc7d890794
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxx
User-Agent: PostmanRuntime/7.6.0
Host: localhost:5555
accept-encoding: gzip, deflate
HTTP/1.1 400
status: 400
Content-Type: application/vnd.API+json
Content-Length: 133
{“errors”:[{“status”:400,“title”:“Invalid Arguments”,“detail”:“wm.jsonapi.exceptions.JSONAPIException: Invalid Query Syntax : {0}”}]}

However, if I do not encode the square brackets, the query works

http://localhost:5555/restv2/product/filter?sort=-list_price&page[limit]=2&page[number]=0

GET /restv2/product/filter
Accept: application/vnd.API+json
cache-control: no-cache
Postman-Token: 003eccbd-676f-49ad-8445-f795c8d8e66b
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxx
User-Agent: PostmanRuntime/7.6.0
Host: localhost:5555
accept-encoding: gzip, deflate
HTTP/1.1 200
status: 200
Content-Type: application/vnd.API+json
Content-Length: 898
{“data”:[{“id”:“149”,“type”:“product”,“attributes”:{“category_name”:“Road Bikes”,“model_year”:“2018”,“brand_name”:“Trek”,“list_price”:“7499.99”,“product_name”:“Trek Domane SLR 8 Disc - 2018”},“relationships”:{},“links”:{“self”:“http://0:0:0:0:0:0:0:1:5555/restv2/bikeStore.ws:product/product/149”}},{“id”:“155”,“type”:“product”,“attributes”:{“category_name”:“Road Bikes”,“model_year”:“2018”,“brand_name”:“Trek”,“list_price”:“11999.99”,“product_name”:“Trek Domane SLR 9 Disc - 2018”},“relationships”:{},“links”:{“self”:“http://0:0:0:0:0:0:0:1:5555/restv2/bikeStore.ws:product/product/155”}}],“links”:{“self”:“http://0:0:0:0:0:0:0:1:5555/restv2/product/filter?page[number]=0&page[limit]=2”,“next”:“http://0:0:0:0:0:0:0:1:5555/restv2/product/filter?page[number]=1&page[limit]=2”,“last”:“http://0:0:0:0:0:0:0:1:5555/restv2/product/filter?page[number]=160&page[limit]=2”},“meta”:{“total-records”:“321”}}

I try to dump out the pipeline at the start of my service that should be called, but it does not generate anything. Suspect it erroring and returning before my code.

ISSERVER|| 2022-06-14 10:10:58 BST [JSONAPI.0801.0002E] (tid=286) Invalid Arguments: wm.jsonapi.exceptions.JSONAPIException: Invalid Query Syntax : {0}
ISSERVER|| 2022-06-14 10:10:58 BST [JSONAPI.0801.0002E] (tid=286) Invalid Arguments: wm.jsonapi.exceptions.JSONAPIException: Invalid Query Syntax : {0}

Have you installed latest fixes for the products

I believe so. I have recently downloaded the latest free trial for the Service Designer.

reamon · June 14, 2022, 7:32pm

URL encoding is quite often misunderstood. There are multiple parts of a URL and encoding rules differ for them. Guide to Java URL Encoding/Decoding | Baeldung has some good information. One of the caveats it mentions is “When encoding URI, one of the common pitfalls is encoding the complete URI. Typically, we need to encode only the query portion of the URI.”

Indeed, most articles on this topic treat the entire URL as though the entire URL is to be encoded all the same. This is not accurate. For example, URLEncoder should not be used to encode the path of the URL. Reserved characters in path segments are different than in query parameter values. As another example, Query string - Wikipedia describes query strings but then starts talking about URL encoding as though the entire URL has the same rules.

Another fun aspect is that there is no definitiive standard for how query strings are structured. We’re all familiar with name=value&foo=bar&x=z but there are many ways that can be used, all of which depend upon the specific server processing the query string. That said the reserved chars for a query string are defined.What characters are allowed unencoded in query strings? | 456 Berea Street has a breakdown of what is allowed where, including in query strings.

According to that article and RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax the ‘[’ and ‘]’ chars are gen-delims which are reserved and are not allowed in the query string to be unencoded. Based upon this, the call to your IS-hosted service must encode those chars.

The question now is what exactly do you do within your service with that data? Even JSON:API docs indicate the square brace chars are to be encoded in the query path. The examples they using un-encoded chars are for readability only but must be encoded IRL.

Is it possible that they way the query string is being processed in your service that when the caller has them encoded, your service is encoding them again? For example. page%5Blimit%5D from the caller becomes page%255Blimit%255D when you pass it on?

nick_lomax24 · June 15, 2022, 10:25am

Hi Reamon,

So what is getting encoded in the URI, is based on my Excel error:
Details: “Web.Contents failed to get contents from ‘http://localhost:5555/restv2/product/filter?sort=-list_price&page[limit]=2&page[number]=0’ (400): Invalid Arguments”

When I enter the unencoded URL in Excel, Excel looks like it encodes it before sending it. I have also just tried using the url with the encoded brackets in Excel and get the same error.

This is with the Accept header set to application/vnd.API+json

With HTTP header Accept set to application/json, the request is successful and I can see that the unencoded square brackets in the pipeline variables:

Here’s the postman console:

http://localhost:5555/restv2/product/filter?sort=-list_price&page%5Blimit%5D=2&page%5Bnumber%5D=0

GET /restv2/product/filter
Accept: application/json
cache-control: no-cache
Postman-Token: 31359e4b-592a-4a15-986e-b6aebc3d0de4
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxx
User-Agent: PostmanRuntime/7.6.0
Host: localhost:5555
accept-encoding: gzip, deflate
HTTP/1.1 200
status: 200
Content-Type: application/json; charset=UTF-8
Content-Length: 53029

Here’s the pipeline dump at the start of my service

ISSERVER|| 2022-06-15 10:20:34 BST [ISP.0090.0001C] (tid=30) --- START tracePipeline [15/06/2022, 10:20] --- 
ISSERVER|| 2022-06-15 10:20:34 BST [ISP.0090.0008C] (tid=30) 0 sort {java.lang.String} = '-list_price' 
ISSERVER|| 2022-06-15 10:20:34 BST [ISP.0090.0008C] (tid=30) 0 page[limit] {java.lang.String} = '2' 
ISSERVER|| 2022-06-15 10:20:34 BST [ISP.0090.0008C] (tid=30) 0 page[number] {java.lang.String} = '0' 
ISSERVER|| 2022-06-15 10:20:34 BST [ISP.0090.0002C] (tid=30) --- END tracePipeline ---

For requests with Accept application/vnd.API+json, there appears to be some pre-processing of query parameters which is struggling to handle the request.
It could be that it is being encoded twice may be, but I can’t quite work out is why if no JSON:API compliant code is run, the query parameters appear in the pipeline correct

Looking with fiddler, the encoded url looks correct from Postman in the network:

GET http://localhost:5555/restv2/product/filter?sort=-list_price&page%5Blimit%5D=2&page%5Bnumber%5D=0 HTTP/1.1
Accept: application/vnd.api+json
cache-control: no-cache
Postman-Token: a64044d3-6602-4b0e-81d6-eece4e4fe032
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxx
User-Agent: PostmanRuntime/7.6.0
Host: localhost:5555
accept-encoding: gzip, deflate
Connection: keep-alive

But get the same error response:

HTTP/1.1 400 Invalid Arguments
Content-Type: application/vnd.api+json
Content-Length: 133

{"errors":[{"status":400,"title":"Invalid Arguments","detail":"wm.jsonapi.exceptions.JSONAPIException: Invalid Query Syntax : {0}"}]}

When application/vnd.API+json is set, there appears to be some pre and post work done by the integration server code. Can debug be increased for this part of the code?
I like the idea of using the built in JSON:API parts, though have some reservation about how it sorts and paginates.
If this was a large table, would it extract the full amount from the database, only to send a fraction to the user.

Many thanks for your time,

Nick

reamon · June 15, 2022, 4:59pm

interesting set of behaviors, with query string parameters and the Accept header value impacting the result. This is beyond what I can help with. Perhaps others have additional items to review. If not, I’d suggest opening a support ticket if you have that avenue available.

nick_lomax24 · June 16, 2022, 3:27pm

It does seem odd behaviour.

When I turn on high trace, I can see the encoded REST call come in and fail when invoking Invoking service wm.jsonapi.jsonapi:processJSONAPIRequest .
Unfortunately, I do not at the moment have the ability to raise a call. However, I think that the call looks valid and is decoded later in the pipeline to the correct value. ie page[limit] and page[number]

I think I have worked out want the processJSONAPIRequest service is used for.

Might try to manually call it within a flow and may be see if I can manipulate the message before being processed by the service.

Thank you in advance for any help

Here’s an extract of the system log at high debug:

ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0039.0001D] (tid=30) GET /restv2/product/filter?sort=-list_price&page%5Blimit%5D=2&page%5Bnumber%5D=0 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0038.0002D] (tid=30) <-- Accept: application/vnd.api+json 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0038.0002D] (tid=30) <-- cache-control: no-cache 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0038.0002D] (tid=30) <-- Postman-Token: d20c3c6d-7962-4270-928a-31add0fea44b 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0038.0002D] (tid=30) <-- Authorization: Administrator:**** 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0038.0002D] (tid=30) <-- User-Agent: PostmanRuntime/7.6.0 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0038.0002D] (tid=30) <-- Host: localhost:5555 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0038.0002D] (tid=30) <-- accept-encoding: gzip, deflate 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0038.0002D] (tid=30) <-- Connection: keep-alive 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0033.0140T] (tid=30) Session b35c6ba23ce44eb982b4fd3b6520b347 - Created in memory. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0012.0003T] (tid=30) Client did not provide a session ID. Creating a new session b35c6ba23ce44eb982b4fd3b6520b347. 
15:54:43.410 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.413 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.413 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
15:54:43.415 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.415 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.415 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0012.0047T] (tid=30) No X509 certificate chain found. 
15:54:43.416 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.416 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.416 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
15:54:43.417 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.417 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.417 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
15:54:43.420 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Storing SagCredentials in shared state: UserName: Administrator;Password: present, but not shown;
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0012.9999D] (tid=30) AuthnCacheManager caching record for user Administrator 
15:54:43.422 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.422 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with shared creds.UserName: Administrator;Password: present, but not shown;
15:54:43.422 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.422 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
15:54:43.423 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.424 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with shared creds.UserName: Administrator;Password: present, but not shown;
15:54:43.425 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.426 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
15:54:43.427 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.427 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with shared creds.UserName: Administrator;Password: present, but not shown;
15:54:43.427 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.427 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
15:54:43.429 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.429 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with shared creds.UserName: Administrator;Password: present, but not shown;
15:54:43.429 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.429 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
15:54:43.430 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Login initialization started.
15:54:43.430 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with shared creds.UserName: Administrator;Password: present, but not shown;
15:54:43.430 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got credentials from SagCredentialsCallback: UserName: Administrator;Password: present, but not shown;
15:54:43.431 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Trying authentication with creds: UserName: Administrator;Password: present, but not shown;
15:54:43.431 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - enter commit()
15:54:43.433 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.principals.SagUserPrincipal - Authentication flag set to: true
15:54:43.433 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Created and stored SagUserPrincipal: Principal name: Administrator
Principal type: USER_TYPE
Properties: {}

15:54:43.434 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Got public credentials from shared credentials: UserName: Administrator;Password: present, but not shown;
15:54:43.434 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - Stored SagCredentials in Subject.privateCredentials 
15:54:43.434 [HTTP Handler 0:0:0:0:0:0:0:1] DEBUG com.softwareag.security.jaas.login.SagAbstractLoginModule - exit commit()
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0012.9999T] (tid=30) Authentication pulled username=Administrator from Subject 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0012.0009D] (tid=30) Successfully authenticated user "local/Administrator". 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0142.0001D] (tid=30) Registered Lifecycle Listener com.wm.app.b2b.server.HTTPLifecycleListener@7f6d4bb9 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service wm.jsonapi.jsonapi:processJSONAPIRequest 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0007.0004T] (tid=30) ACLManager: allow check for user "local/Administrator" on ACL "Internal" is returning true. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0048.9999D] (tid=30) Invalid Query Syntax : {0} 
ISSERVER|| 2022-06-16 15:54:43 BST [ISU.0000.9999D] (tid=30) InfraDC: Service Error Event wm.jsonapi.jsonapi:processJSONAPIRequest 
ISSERVER|| 2022-06-16 15:54:43 BST [JSONAPI.0801.0002E] (tid=6387) Invalid Arguments: wm.jsonapi.exceptions.JSONAPIException: Invalid Query Syntax : {0} 
ISSERVER|| 2022-06-16 15:54:43 BST [ISU.0000.9999D] (tid=30) InfraDC: Service Error Event wm.jsonapi.jsonapi:processJSONAPIRequest 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0007.0004T] (tid=30) ACLManager: allow check for user "local/Administrator" on ACL "Internal" is returning true. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service bikeStore.ws.product_.services:getFilteredproducts 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0007.0004T] (tid=30) ACLManager: allow check for user "local/Administrator" on ACL "Internal" is returning true. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0016.9999D] (tid=30) Cache Miss/ignored (hit list) for bikeStore.ws.product_.services:getFilteredproducts(id=1446429367) 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service pub.flow:tracePipeline. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service pub.flow:tracePipeline 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0001C] (tid=30) --- START tracePipeline [16/06/2022, 15:54] --- 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 sort {java.lang.String} = '-list_price' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 page[limit] {java.lang.String} = '2' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 page[number] {java.lang.String} = '0' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 errors[0] {[Lcom.wm.util.Values;} =>  
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 1 status {java.lang.Integer} = '400' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 1 title {java.lang.String} = 'Invalid Arguments' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 1 detail {java.lang.String} = 'wm.jsonapi.exceptions.JSONAPIException: Invalid Query Syntax : {0}' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0002C] (tid=30) --- END tracePipeline --- 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service pub.flow:debugLog. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service pub.flow:debugLog 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0003C] (tid=30) START brandCondition = %brandCondition%
brand-name = %brand-name% 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service bikeStore.flows:getOrderClause. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service bikeStore.flows:getOrderClause 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0004T] (tid=30) Branch : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0002D] (tid=30) Branch null: child -list_price not found, invoking default child $default 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0004T] (tid=30) Branch $default: index=1 depth=2 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0003D] (tid=30) Branch $default: invoking child sort = /^\-.*/ 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=3 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service pub.string:replace. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service pub.string:replace 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0004T] (tid=30) Branch $default: index=2 depth=2 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0004T] (tid=30) Branch : index=2 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISU.0000.9999D] (tid=30) InfraDC: ISDCNotificationQueue sending isextdc.logmessage to queue 
ISSERVER|| 2022-06-16 15:54:43 BST [ISU.0000.9999D] (tid=30) InfraDC: ISDCNotificationQueue sending isextdc.logmessage to queue 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service bikeStore.flows:getPageSize. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service bikeStore.flows:getPageSize 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service pub.flow:debugLog. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service pub.flow:debugLog 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0003C] (tid=30) Processing page[size] = %'page[size]'%
Processing page[size] = %"page[size]"%
Processing page[size] = %page[size]%
model_year = %model_year% 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0004T] (tid=30) Branch : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0019D] (tid=30) Branch null: no expressions evaluated true, but default child found 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0004T] (tid=30) Branch : index=3 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service bikeStore.flows:getPageNumber. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service bikeStore.flows:getPageNumber 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service pub.flow:debugLog. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service pub.flow:debugLog 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0003C] (tid=30) Processing page[number] = 0 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0004T] (tid=30) Branch : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0003D] (tid=30) Branch null: invoking child %"page[number]"% = /^[0-9]+$/ 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0004T] (tid=30) Branch : index=2 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service pub.flow:tracePipeline. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service pub.flow:tracePipeline 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0001C] (tid=30) --- START tracePipeline [16/06/2022, 15:54] --- 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 defaultPageNumber {java.lang.String} = '1' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 pageNumber {java.lang.String} = '0' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 defaultPageSize {java.lang.String} = '100' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 pageSize {java.lang.String} = '100' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 defaultSortField {java.lang.String} = 'product_id' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 orderClause {java.lang.String} = 'ORDER BY list_price DESC' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 defaultSortOrder {java.lang.String} = 'asc' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 message {java.lang.String} = 'Processing page[number] = 0' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 sort {java.lang.String} = '-list_price' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 page[limit] {java.lang.String} = '2' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 page[number] {java.lang.String} = '0' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 0 errors[0] {[Lcom.wm.util.Values;} =>  
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 1 status {java.lang.Integer} = '400' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 1 title {java.lang.String} = 'Invalid Arguments' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0008C] (tid=30) 1 detail {java.lang.String} = 'wm.jsonapi.exceptions.JSONAPIException: Invalid Query Syntax : {0}' 
ISSERVER|| 2022-06-16 15:54:43 BST [ISP.0090.0002C] (tid=30) --- END tracePipeline --- 
ISSERVER|| 2022-06-16 15:54:43 BST [ISC.0049.0005D] (tid=30) Invoke : index=1 depth=1 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0015.0005D] (tid=30) Invoking service bikeStore.adapters:getProducts. 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0014.0016T] (tid=30) Invoking service bikeStore.adapters:getProducts 
ISSERVER|| 2022-06-16 15:54:43 BST [SCC.0126.0204T] (tid=30) Connection adapters:bikeStore(1) in the <Default> partition of adapters:bikeStore is now busy 
ISSERVER|| 2022-06-16 15:54:43 BST [ISS.0142.0001D] (tid=30) Registered Lifecycle Listener com.wm.app.b2b.server.jca.WmConnectionPool@3cd7630d Name=adapters:bikeStore(1), @HashCode=1685436383 
ISSERVER|| 2022-06-16 15:54:43 BST [ADA.0001.0103T] (tid=30) Begin local transaction. 
ISSERVER|| 2022-06-16 15:54:44 BST [ADA.0001.0110T] (tid=30) Execute the SQL statement: "DECLARE @PageNumber AS INT
DECLARE @RowsOfPage AS INT
SET @PageNumber=?
SET @RowsOfPage=?
SELECT product_id
      ,product_name
      ,brand_name
      ,category_name
      ,model_year
      ,list_price
  FROM production.products p1, production.categories c1, production.brands b1
  where
  
  p1.category_id = c1.category_id and
  p1.brand_id = b1.brand_id
 
ORDER BY list_price DESC
  OFFSET (@PageNumber-1)*@RowsOfPage ROWS
FETCH NEXT @RowsOfPage ROWS ONLY;" 
ISSERVER|| 2022-06-16 15:54:44 BST [ISS.0016.9999D] (tid=30) Put id=1446429367 
ISSERVER|| 2022-06-16 15:54:44 BST [ISS.0016.9999D] (tid=30) Create List key=bikeStore.ws.product_.services:getFilteredproducts:1446429367 
ISSERVER|| 2022-06-16 15:54:44 BST [ISS.0016.9999D] (tid=30) Cache Insert id=1446429367 for bikeStore.ws.product_.services:getFilteredproducts 
ISSERVER|| 2022-06-16 15:54:44 BST [ADA.0001.0104T] (tid=30) Commit local transaction. 
ISSERVER|| 2022-06-16 15:54:44 BST [ISS.0142.0002D] (tid=30) Unregistered Lifecycle Listener com.wm.app.b2b.server.jca.WmConnectionPool@3cd7630d Name=adapters:bikeStore(1), @HashCode=1685436383 is true 
ISSERVER|| 2022-06-16 15:54:44 BST [SCC.0126.0205T] (tid=30) Connection adapters:bikeStore(1)  of adapters:bikeStore is now available 
ISSERVER|| 2022-06-16 15:54:44 BST [ISS.0142.0002D] (tid=30) Unregistered Lifecycle Listener com.wm.app.b2b.server.HTTPLifecycleListener@7f6d4bb9 is true 
ISSERVER|| 2022-06-16 15:54:44 BST [ISC.0040.0001D] (tid=30) --> HTTP/1.1 400 Invalid Arguments 
ISSERVER|| 2022-06-16 15:54:44 BST [ISC.0038.0002D] (tid=30) --> Content-Type: application/vnd.api+json 
ISSERVER|| 2022-06-16 15:54:44 BST [ISC.0038.0002D] (tid=30) --> Content-Length: 133 
ISSERVER|| 2022-06-16 15:54:44 BST [ISS.0033.0140T] (tid=30) Session b35c6ba23ce44eb982b4fd3b6520b347 - Removed from memory.

suke · June 20, 2022, 3:53pm

Hello Reamon,

Square brackets are used as delimiters for Pagination feature in JSON API.

These are not required to be encoded. Only when some data conflict with these reserved characters, in such cases, square brackets should be encoded. But these delimiters should exist in their original form.
This is as per the below spec

reamon · June 21, 2022, 3:17pm

We both referenced the same RFC. I came away with this understanding: “the ‘[’ and ‘]’ chars are gen-delims which are reserved and are not allowed in the query string to be unencoded.”

Yes, indeed. However, the JSON API docs have this note in various places, including JSON:API — Latest Specification (v1.1) in its description of paginaion: “Note: The above example URI shows unencoded [ and ] characters simply for readability. In practice, these characters must be percent-encoded, per the requirements in RFC 3986.”

nick_lomax24 · June 21, 2022, 4:54pm

Hi Sumit,

The JSON API compliant doc does seem to suggest the square brackets should be encoded in the query string. Though, reading both RFC and the JSON API compliant documentation, I can see how encoded and not encoded could be both valid.

The difficulty I have is with Excel encoding the square brackets automatically. Reading the rfc3986, it does not seem to imply the square brackets must not be encoded.

URI producing applications should percent-encode data octets that
correspond to characters in the reserved set unless these characters
are specifically allowed by the URI scheme to represent data in that
component. If a reserved character is found in a URI component and
no delimiting role is known for that character, then it must be
interpreted as representing the data octet corresponding to that
character’s encoding in US-ASCII.

I think that the URL that Excel is producing. ie encoding the brackets is a valid URL and that Integration Server should handle the encoded square brackets in the query string and consider them valid JSON:API compliant query parameters

reamon · June 21, 2022, 5:27pm

I think it is more concrete than a suggestion. It explicitly states “must be percent-encoded.”

Here again, I view the RFC as explicit. The syntax rules indicate [ and ] must be encoded. They are explicitly not included in the allowed chars for a query.

system · September 19, 2022, 5:28pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
com.webmethods.portal.system.init.InitializationException. webMethods , webMethods-BPMS , MWS-CAF-Task-Engine	12	5379	April 2, 2021
unbounded array inside an unbounded array webMethods , Mainframe-Integration , EntireX	8	967	April 2, 2021
Expose Odata feed via API gateway for Excel consumption webMethods , API-Management , API-Gateway	11	482	December 10, 2024
Adapter couldn't recover DB connection, instead it drained the thread pools and got stuck webMethods , Integration-Server-and-ESB , Adapters-and-E-Standards , Adapter-JDBC	14	974	July 22, 2024
Unable to start MWS -- java.lang.NullPointerException webMethods , webMethods-BPMS , MWS-CAF-Task-Engine	7	1989	April 2, 2021