Memory Leak in Oracle LOB Reading?? [SPR-5998]

[spring-projects-issues]

Mohan opened SPR-5998 and commented

As per the advise, we have carried out few testings.

Finally found that BLOB reading causes the problem (not the writing as mentioned here, that was a wrong assumption).

Oracle uses the so called 'outBuffer of oracle.jdbc.driver.T4C8TTIBlob' when we read blob data from the db, that is not cleared ever after closing the connection object.
The connection is cached by the Connection pool provider(Unable to reproduce the bug with Spring Driver manager data source, because that doesn't cache the connection object.),
so the outbuffer also stays with it. I got the same results with all the connection pool providers like DBCP,C3P0 and WebSphere.

When is it populated??

---

There is a call 'blob.getBytes(1, (int) blob.length())' in 'OracleLobHandler.getBlobAsBytes', which uses the outbuffer.

How to clear the buffer??

---

Wrapping the getBytes call with open and close methods (like blob writing) make sure the buffer is cleared once the job is done.

oracle.sql.BLOB oraBlob = (oracle.sql.BLOB)blob;
oraBlob.open(BLOB.MODE_READONLY);
data = oraBlob.getBytes(1, (int)b.length());
oraBlob.close();

I am not sure about this solution. I posted the same in Oracle forum and waiting for a reply.

More info available here, http://forums.oracle.com/forums/thread.jspa?messageID=3672154#3672154

As a temporary work around, i have overridden the 'OracleLobHandler.getBlobAsBytes' in our custom oracle lob handler class and applied these changes
in to it.

Is our understanding correct?? If correct then Should Spring OracleLobHandler be modified??

---

Affects: 2.5 final

Reference URL: http://forum.springsource.org/showthread.php?p=242802

Issue Links:

• OracleLobHandler: Temporary LOBs don't get freed and eat TEMP tablespace [SPR-6209] #10877 OracleLobHandler: Temporary LOBs don't get freed and eat TEMP tablespace

Referenced from: commits 5a158fb

[spring-projects-issues]

Thomas Risberg commented

I have done some testing and from what I can tell the T4C8TTIBlob objects do allocate a 32K outBuffer that it holds on to after the BLOB is read. This is a small amount of memory and since you have a limited number of connections I doubt this would cause severe issues for your application. If you have some memory issues it's more iikely to be caused by some other classes handling the BLOB objects in memory.

-Thomas

[spring-projects-issues]

Mohan commented

Hi Thomas,

Few doubts,

1. I dont think this 32K is constant. The outBuffer size is depends on size of the data we read from the database. In our test case (provided by oracle), we used 10MB data, we could able to see the outBuffer size as 10MB. In our application 10MB is on the lower side. As you can see in our posting that the buffer size as '30123012 bytes' for one of our data.

2. Regarding the limited number of connections, I agree to your point that we have limited number of connections. My question here is, when will the populated ourBuffer memory be reclaimed?.
   With my understanding, the outbuffer will be cleared only when we do update/insert blob operation on the same connection object next time. The Spring OracleLobHandler calls the blob.open which clears the outbuffer. But we do read data more often than write. So, for ex. 10 connection X 10MB = 100MB will be reserved most of the time in the JVM for 'nothing'.
   This 100MB is 4%(lower side) of our Heap size. Reclaiming this memory would postpone our OutOfMemoryError for some time :) . If you feel like our case is extreme, at lease help us to review our solution. If you foresee any problem in our solution please let us know.

Thanks in Advance,
Mohan

[spring-projects-issues]

Thomas Risberg commented

I tried calling open and then close on the CLOB/BLOB but saw some problems with this approach. Need to investigate the implications further.

[spring-projects-issues]

Thomas Risberg commented

The problems I was seeing were related to attempting to read the LOB more than once and also to call close on temporary lobs that had been freed.

If we call close() then we can't access the LOB again. Since this is a change in behavior for the LobHandler it won't be the default to call close on any LOB after read.

I added a property on the OracleLobHandler named "releaseResourcesAfterRead" which can be set to true to force open/close calls to be made on ay LOBs that are read and freeTemporary on any temporary LOBs read. This of course means that you can only read the LOBs once.