java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes (7947M) > maxPhysicalBytes (7756M)

[sep19w]

Hello,
I got this error when using javacv to segment audios. And then it cannot work any more. The CPU memory is 16G, it occured when I used about 10G memory.
I have noted the similar issues, but I have already used version 1.5.2.
Is there any way to increase the maxPhysicalBytes? or some other ways to solve this?

[saudet]

Yes, we can set the org.bytedeco.javacpp.maxPhysicalBytes system property for that:
http://bytedeco.org/javacpp/apidocs/org/bytedeco/javacpp/Pointer.html#maxPhysicalBytes

[sep19w]

Ok, can I ask why the memory will increase when using this?

[saudet]

What do you mean? If you're using an algorithm that needs that amount of memory, then you'll need to let it use that much memory.

[sep19w]

We used grabber and recorder to segment audios. The qps is about 40 per second. After few hour, the memory will increase from 2G to almost 8G. Is there any possible reason for this？

[saudet]

Using PointerScope should help with that as explained here:
http://bytedeco.org/news/2018/07/17/bytedeco-as-distribution/
But it's possible there are still some memory leaks, for example, see issue #911.

If using PointerScope doesn't help, let me know! Thanks

[sep19w]

Ok, thanks

[sep19w]

Hi~
I have already used PointerScope. It is true that there are still some memory leaks, but increase slower than before.

[saudet]

Are you still getting OutOfMemoryError?

[sep19w]

After setting properties "maxPhysicalBytes" and ”maxBytes”, there was no OutOfMemoryError when using 8G. But the memory will still grow slowly untill out of memory of jvm.

[saudet]

Could you provide a small code snippet to reproduce this issue?

[szlzhm]

Yes, we can set the org.bytedeco.javacpp.maxPhysicalBytes system property for that:
http://bytedeco.org/javacpp/apidocs/org/bytedeco/javacpp/Pointer.html#maxPhysicalBytes

There is a problem with the mechanism of javacpp free memory, which can not be released immediately as jvm, especially in high concurrency and high throughput scenarios. Because javacpp maintained a memory block of its own, the mechanism had problems.

[saudet]

@szlzhm That's what PointerScope is for, see above.

[saudet]

I've added PointerScope inside the methods of FFmpegFrameGrabber and FFmpegFrameRecorder in commit 28b90ef. Please give it a try with the snapshots: http://bytedeco.org/builds/

[saudet]

I think I've fixed all the memory leaks occurring in FFmpegFrameGrabber and FFmpegFrameRecorder.
Please give it a try with the snapshots: http://bytedeco.org/builds/

[sep19w]

Ok! I will try this later. Thank you very much

[sep19w]

I try to use this in build.gradle, but it cannot build with the errors below for example:

Could not find flandmark-android-arm.jar (org.bytedeco:flandmark:1.07-1.5.4-SNAPSHOT:20200616.071322-261).
Searched in the following locations:
https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/flandmark/1.07-1.5.4-SNAPSHOT/flandmark-1.07-1.5.4-20200616.071322-261-android-arm.jar
Could not find flandmark-android-arm64.jar (org.bytedeco:flandmark:1.07-1.5.4-SNAPSHOT:20200616.071322-261).
Searched in the following locations:
https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/flandmark/1.07-1.5.4-SNAPSHOT/flandmark-1.07-1.5.4-20200616.071322-261-android-arm64.jar

[sep19w]

My build.gradle is like this:
repositories {
maven { url = "https://oss.sonatype.org/content/repositories/snapshots" }
}

dependencies {
compile 'org.bytedeco:javacv-platform:1.5.4-SNAPSHOT'
}

[saudet]

The snapshots are a bit broken right now.
You'll need to fetch them with Maven as explained here:
http://bytedeco.org/builds/#other-options

[saudet]

@wyh0919 Ok, the snapshots have been fixed. Please try it again that way with Gradle, it should work now.

[saudet]

Fixes included in JavaCV 1.5.4. Enjoy!