Read CSV from URL

Hello!

Im new in the integration Service world and need some help.

Im trying to read a CSV file directly from a URL but its not working.

Here is what im doing inside my service:

pub.client:http (https:\myserver.com\file.csv, get method)
pub.flatfile:convertToValue (created FFSchema here)

When i run the server i get the error:
com.wm.app.b2b.server.ServiceExpection:java.io.IOExpection:java.net.SocketTimeoutException: Read Ttimed out

The URL/file is accessible inside the IS server

Can someone help me?

Hi Fabio,

The URL you are trying to access is HTTPS…did you try setting the secure parameters in the HTTP client service? It would be good if you can put the entire stack trace of the error that you are facing.

Thanks,
Sree

“Read Timed Out” - Possible of the socket connection is getting closed.

Remember pub.client:http will set the SSL timeout before calling the target if incase its https hosts. So verify the watt.net.ssl.server.clientHandshakeTimeout extended property and set accordingly.

how do i check/set this property?

we are using Integration Service 10.5, onpremisse installation.

Hi Fabio,

in IS Admin under Settings → Extended Settings.
There is a link to “Show and Hide …” which opens a list with known properties, search for the mentioned property, mark the checkbox in front of it an save.
After that the property will show up in the list of extended settings in the UI.
Via the “Edit …” link you can then modify the property.
After saving you might need to restart the IS to apply the modification.
See IS Administrators Guide for further informations about this.

Regards,
Holger

Hi Fabio -

If you’re getting a “read timed out,” you may have already gotten past the SSL handshake. That error means you’re waiting to receive data from the server, but no data arrived in time. There’s a “timeout” input variable in the pub.client:http service that you can play with. If you don’t specify a value for that variable, the IS will use the value from watt.net.timeout so you could also configure it there if you prefer to set it globally (see Reverb)

Having said this, in my experience, timeouts are typically due to a more deeply rooted issue, so here are some things you can try to find the root cause:

  1. Ensure you’re using the correct URL. I noticed the URL you shared in your post had back-slashes instead of forward-slashes. I’m sure that was a typo, but just in case, make sure the URL is correct.

  2. If the server allows it, temporarily change to simple HTTP instead of HTTPS to eliminate TLS as a variable.

  3. Try pulling a different file from the server to determine if this problem is specific to the file (e.g. try pulling a small text file).

  4. Try pulling the file down via a different mechanism. In other words, from the same server where the IS is running, can you download the file via a browser or via curl?

  5. Try pulling the file from a different Integration Server (e.g. one running on your local machine).

  6. If the server you’re pulling the file from is behind a load balancer, can you go directly to the server to determine if it’s an issue with the load balancer?

  7. Does anything get logged in the target server when you attempt your request?

Trying these different things should help you get closer to determining the root cause of your timeouts.

Hope this helps,
Percio

2 Likes

Hi Guys!

I have some new info to share about this problem:

1 - Setting this property doesnt workwatt.net.ssl.server.clientHandshakeTimeout;
2 - I can access the service/csv URL from the server browser (https, its a external, public URL, .gov)
3 - I had to set our internet proxy inside the IS
3.1 Set the proxyAlias inside pub.client:http

URL:
https://www4.bcb.gov.br/Download/fechamento/20220105.csv

Now im having a different error while running the service with just the pub.client:http

com.wm.app.b2b.server.ServiceException: java.io.IOException: javax.net.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider;certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Additional info:

Already inclued our certfile in Security->Certificates Client Certificates and Security → Keystore

Additional info:

tryed this command on both IS Server and Desginer machine:

keytool -import -trustcacerts -file certificate_path.crt -alias “Certificate” -keystore Software AG_directory\jvm\jvm160_32\jre\lib\security\cacerts

same error

@fabio.morciani I apologize for the late reply. Hopefully, you’ve figured this out by now. In case you haven’t, the certificate that you added to the truststore, was that the CA certificate from the target HTTPS endpoint? It wasn’t clear from the post.

Also, the Integration Server defines a separate truststore via the DEFAULT_IS_TRUSTSTORE alias. If I recall correctly, you have to add the CA cert there instead of cacerts. Better yet, you should create your own truststore file and reconfigure the DEFAULT_IS_TRUSTSTORE alias to point there. Similarly, you should create your own keystore and point DEFAULT_IS_KEYSTORE there.

Percio