Hello. We are storing big nonXML documents (ej. 6Mb, 13Mb) into Tamino with TNonXMLObject and so.
I’ve detected that when doing such storage, the JVM memory usage grows quite high (e.g. 30Mb). Monitoring the inner allocated / free memory, there are memory usage peaks matching the nonXML insertion; after the document has been stored the memory is released.
I think the problem is that the JVM is reading the whole of the nonXML into memory, instead of just piping it out to Tamino.
I also think that the problem is not in the Tamino API itself, but in its use of the Sun’s [Http]URLConnection class that comes with Java. It seems that is this class the one that, before posting anything, reads it wholly into memory, in order to set the Content-Length HTTP header.
There is at least a freeware alternative for HttpURLConnection: HTTPClient (http://www.innovation.ch/java/HTTPClient/API/HTTPClient/HttpURLConnection.html). Now, I’ve tested it and:
- By just activating it, there is no change; HTTPClient does not use the HTTP chunked encoding to send the nonXML in small parts as I hoped and still reads the whole content in memory
- It is possible however to store nonXML into Tamino without such memory consumption with HTTPClient, by not using the Tamino API, setting previously the Content-Length header and then issuing a PUT directly to Tamino
So:
- Anybody has had the same problem?
- Anybody has used HttpClient for storing nonXML into Tamino, or for any other thing?
- Anybody has used direct PUT requests to store nonXML into Tamino? Any problem?
- Anybody can think on any other solution for that memory consumption problem?
Tnahk you,
We in development know about this problem.
A similar problem (downstream) is documented in the Java bug database (4333920). They say that this will be fixed within JDK 1.4. We plan to be executable under JDK 1.4 for the next Tamino XML Server Release.
It might be worth to try JDK1.4 with your application. If it works, it might fix your problem.
Anybody else knows a different workaround for this problem?
Reading about 4333920 bug, I don’t think it will work for me. It is about reading chunked encoding, not sending it. However maybe I will try.
Anyway, would Tamino server accept a POST or PUT sent with chunked encoding? HTTPClient’s URLConnection is supposed to be able to do this automagically “if the server supports HTTP 1.1”, but in my (few) tests this did not seem to happen.
Maybe I will try to force chunked encoding and see what happens, since maybe I don’t always know the content-length.
Thanks
Still, I recommend giving JDK 1.4.1 a try, before playing around to long with the HTTP stuff.