MetadataRetriever#retrieveDurationUs leads to OOM

[PaulWoitaschek]

Version

Media3 pre-release (alpha, beta or RC not in this list)

More version details

1.8.0-rc01

Devices that reproduce the issue

Pixel

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Not tested

Reproduction steps

Call retrieveDurationUs twice in parallel on two MetadataRetriever instances like described in #2640 (comment)

It throws an OOM exception. In #2640 I was asked to create another bug about the OOM and it was mentioned that the media3 has likely a very memory heavy representation of the media.

#2640 (comment)

It looks like we are trying to read the whole stbl box, which is 50MB in the MP4 file according to mp4Box (and may be larger in media3's in-memory representation)

The whole issue originated here where users report that they can't see audiobooks of large app size:
PaulWoitaschek/Voice#2896

Expected result

It parses the duration

Actual result

It throws an OOM error

Media

See #2640

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

[icbaker]

When just playing my generated 80hr AAC in ExoPlayer (which succeeds), the ~50MB stbl is parsed into a TrackSampleTable with 12_446_537 entries in each of the 2 int[] and 2 long[] arrays, so assuming no overhead that is roughly (2 * 4 + 2 * 8) * 12_446_537 bytes = 298_716_888 ~= 300MB. I confirmed this by taking a heap-dump during playback.

This data is needed for extraction and playback, but not just for determining the duration. Unfortunately MetadataRetriever currently doesn't have a way to signal to Mp4Extractor that it doesn't need/expect full playback support - so this assignment is still done.

We can use this issue to track allowing MetadataRetriever to customize Mp4Extractor (and maybe also other Extractor impls) behaviour to focus only on the needed info and avoid allocating/storing info that isn't needed.

[icbaker]

@rohitjoins has modified MetadataRetriever and Mp4Extractor in the commit above to avoid parsing the stbl box if only the duration and other "non sample-level data" is needed.

That means my repro which previously OOM'd when trying to calculate the duration twice (in parallel) for the 80hr 2GB file now succeeds (see stack trace in #2640 (comment)). It also succeeds with 5x parallelism on the emulator I was using. It still fails with a (different) OOM when trying to calculate the duration 10x in parallel. This OOM comes from sniffing:

androidx.media3.exoplayer.upstream.Loader$UnexpectedLoaderException: Unexpected OutOfMemoryError: Failed to allocate a 50853640 byte allocation with 25165824 free bytes and 29MB until OOM, target footprint 531231792, growth limit 536870912
    at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:482)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
    at java.lang.Thread.run(Thread.java:1012)
Caused by: java.lang.OutOfMemoryError: Failed to allocate a 50853640 byte allocation with 25165824 free bytes and 29MB until OOM, target footprint 531231792, growth limit 536870912
    at java.util.Arrays.copyOf(Arrays.java:3585)
    at androidx.media3.extractor.DefaultExtractorInput.ensureSpaceForPeek(DefaultExtractorInput.java:225)
    at androidx.media3.extractor.DefaultExtractorInput.advancePeekPosition(DefaultExtractorInput.java:162)
    at androidx.media3.extractor.DefaultExtractorInput.advancePeekPosition(DefaultExtractorInput.java:178)
    at androidx.media3.extractor.mp4.Sniffer.sniffInternal(Sniffer.java:217)
    at androidx.media3.extractor.mp4.Sniffer.sniffFragmented(Sniffer.java:88)
    at androidx.media3.extractor.mp4.FragmentedMp4Extractor.sniff(FragmentedMp4Extractor.java:468)
    at androidx.media3.exoplayer.source.BundledExtractorsAdapter.init(BundledExtractorsAdapter.java:83)
    at androidx.media3.exoplayer.source.ProgressiveMediaPeriod$ExtractingLoadable.load(ProgressiveMediaPeriod.java:1124)
    at androidx.media3.exoplayer.upstream.Loader$LoadTask.run(Loader.java:453)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644) 
    at java.lang.Thread.run(Thread.java:1012) 

This happens because we deliberately increase the sniffing limit to read the whole moov box (which contains the stbl), in case we find an mvex box inside it which indicates the file is fragmented:

media/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/Sniffer.java

Lines 155 to 165 in 7f5e4c4

	if (atomType == Mp4Box.TYPE_moov) {
	// We have seen the moov atom. We increase the search size to make sure we don't miss an
	// mvex atom because the moov's size exceeds the search length.
	bytesToSearch += (int) atomSize;
	if (inputLength != C.LENGTH_UNSET && bytesToSearch > inputLength) {
	// Make sure we don't exceed the file size.
	bytesToSearch = (int) inputLength;
	}
	// Check for an mvex atom inside the moov atom to identify whether the file is fragmented.
	continue;
	}

And then when we peek past it, we have to store the peeked data in memory (which is what leads to the stack trace).

Because we sniff this file with both Mp4Extractor and FragmentedMp4Extractor, this results in allocating ~100MB before we even worked out which extractor to use...

I have a prototype change which tweaks the sniffing logic to say "if you find an stbl box larger than 1MB, don't peek past it, just assume the file is unfragmented". This is because fragmented MP4 files should have their stbl data split into the subsequent moof boxes (and therefore a very large stbl box suggests the file is not fragmented). I'm not terribly keen on the arbitrary 1MB threshold, but it does reduce the amount of bytes we need to sniff to decide if this file is a fragmented or non-fragmented MP4 from 50MB to 465 bytes...

[PaulWoitaschek]

Wow, great News! And super speedy 🚀
Thank you!

[icbaker]

0ef418e resolves peeking very large stbl boxes during sniffing.

I'm going to close this now, as we have mitigated the two worst causes of OOMing when working with this large MP4 file.