Crunchyroll Android Improvements #2689

armands-malejevs · 2022-05-23T08:42:57Z

This is a collection of improvements we've done for the Crunchyroll Android app:

Support disabling buffering
Fix AudioFocus bug that could cause the player to stop responding to play/pause in some instances.
Fix player crashing when it is being cleared.
Add support for customising back buffer duration and handle network errors gracefully to prevent releasing the player when network is lost.
Allow player to be init before source is provided, and later update once a source is provided.
Adds handling for providing a empty source in order to stop playback and clear out any existing content
Add support for onBufferProgress prop on Android to get buffer data even when the player is paused.
Add support for detecting if format is supported and exclude unsupported resolutions from auto quality selection and video track info in RN.
Improve error handling
Add support for L1 to L3 Widevine fallback if playback fails initially.
Reduce buffer size based on available heap
Force garbage collection when there is no available memory
Improve memory usage
Support disabling screen recording
Improved error capturing
Fix DRM init crashes
Improve progress reporting
Fix progress loss when network connection is regained

Testing

Either use the test apps in this project or link your own project to this PR and test everything related to playback.

Fix AudioFocus bug that could cause the player to stop responding to play/pause in some instances. Fixes issue react-native-video#1945 This was caused by the player requesting audio focus on each play (un-pause) and that resulted in a small window of Audio focus loss and then gain. The focus loss results in the player being paused while the player was supposed to play at the time. The solution is to keep track of Audio focus and not request new focus if we already have it.

Upgrade ExoPlayer from 2.11.4 to 2.13.2 and fix any related issues related to the upgrade and deprecated method use.

This PR adds the property disableBuffering: boolean for android. The PR was initally created as a personal fork and referenced in crunchyroll/vilos#1227 also updated RNVLoadControl constructor and method to reflect new ExoPlayer.LoadControl api related ticket: https://jira.tenkasu.net/browse/VEX-3776

Fix player crashing when it is being cleared.

Add support for customizing back buffer duration and handle network errors gracefully to prevent releasing the player when network is lost.

- Allow player to be init before source is provided, and later update once a source is provided. - Adds handling for providing a empty source in order to stop playback and clear out any existing content

Add support for onBufferProgress prop on Android to get buffer data even when the player is paused.

Add support for detecting if format is supported and exclude unsupported resolutions from auto quality selection and video track info in RN.

Improve error handling.

Add support for L1 to L3 Widevine fallback if playback fails initially.

Converts ios implementation from objective-c to swift.

Adds offline decryption key and uses it to decrypt content during offline playback Jira: VEX-5938 https://jira.tenkasu.net/browse/VEX-5938 - Update to accept scheme for key required to play offline playback - Uses provided scheme to intercept call from player and return the key - Fixes player item observer removal pattern ### Reviews - Major reviewer (domain expert): @armadilio3

#14) Add support for content tracks and improve track selection to work even during ads playback.

@nickfujita

This PR changes the behavior on old devices that have poor memory management. Jira: VEX-6030 https://jira.tenkasu.net/browse/VEX-6030 The solution implied customizing the method shouldContinueLoading from RNVLoadControl to use only the available heap, performing tests on an old Nexus 5 it was determined the ideal bytes allocation to half the reported heap, that provided some buffering during ads but smooth playback during the video with no crashes (23:39 of 23:39 at the moment of writing this, video kept playing as expected after 3 ad breaks) The fix is only targeting Marshmallow as the reduction of buffer is substantial and other versions that work properly should not get affected. Depending on the test result of VEX-5758, this same fix can be applied to Nougat Reviews Major reviewer (domain expert): @armadilio3 Minor reviewer: @nickfujita

@nickfujita

This PR is intended to prevent memory crashes when the free memory runs out Jira: VEX-6030 https://jira.tenkasu.net/browse/VEX-6030 Velocity PR: crunchyroll/velocity#2043 By forcefully calling the garbage collector the memory crashes, specially on lower resolutions (<720p) are gone on devices with 2GB of ram or less and Android 6, also testing 3GB Android 7 with this patch, so far is working as intended Reviews Major reviewer (domain expert): @nickfujita Minor reviewer: @jctorresM

Improve memory management to reduce pressure on low end devices. JIRA: VEX-6365

@nickfujita

This PR adds core actions to enable or disable screenshots Jira: VEX-6540 https://jira.tenkasu.net/browse/VEX-6540 velocity PR: crunchyroll/velocity#2248 New core actions were added to control the player surface and mark it as secure. Sending the prop to react-native-video will recreate the surface of the player changing it to a SurfaceView and enabling security Reviews Major reviewer (domain expert): @nickfujita Minor reviewer: @armadilio3

@gabriel-rivero

Updates disableDisconnectError retry condition to be more specific. Jira: VEX-6518 Reviews Major reviewer (domain expert): @armadilio3 Minor reviewer: @gabriel-rivero

freeboub · 2022-05-25T21:03:57Z

android-exoplayer/src/main/java/com/brentvatne/exoplayer/ReactExoplayerView.java

+                        public void run() {
+                            if (player != null) {
+                                double bufferedDuration = (double) (player.getBufferedPercentage() * player.getDuration() / 100);
+                                eventEmitter.bufferProgress(0d, bufferedDuration);


OK, I understand you need additionnal informations on available buffer, but it should be added to onProgress event.
the new bufferProgress has 2 paramters: 0 Ok for futur use, so not needed now.
bufferedDuration which is exactly equals to bufferedDuration in onProgress.
The only difference is the type you use double instead of long.

Another difference is the frequency. If another issue for you is linked to the the missing report when playback is pause, I invite you to check following PR: #2664

For me this new event doesn't bring new infomation for application...

android-exoplayer/src/main/java/com/brentvatne/exoplayer/ReactExoplayerView.java

README.md

cobarx · 2022-05-30T19:06:28Z

Agree with @armadilio3's suggestion that we record the improvements and take them up one by one. Our team has limited capacity to devote to changes, so picking them up incrementally would be much better.

gabriel-rivero

double checked the changes in test-harness, features are working with no issues

armands-malejevs · 2022-05-31T07:18:10Z

@freeboub I cleaned up some stuff 👍 I had some questions on some of your comments, let me know how you want to handle those

armands-malejevs · 2022-06-02T12:13:18Z

@freeboub Can you take another look? I fixed/responded to some of the feedback

freeboub · 2022-06-03T16:40:48Z

PR Tested on sample provided in exemple/basic:
I have following crash when playing local file (require('./broadchurch.mp4'))

[Fri Jun 03 2022 18:39:31.221] LOG {"error":{"errorStackTrace":"com.google.android.exoplayer2.ExoPlaybackException: Unexpected runtime error\n\tat com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:587)\n\tat android.os.Handler.dispatchMessage(Handler.java:102)\n\tat android.os.Looper.loop(Looper.java:223)\n\tat android.os.HandlerThread.run(HandlerThread.java:67)\nCaused by: java.lang.IllegalStateException\n\tat com.google.android.exoplayer2.util.Assertions.checkState(Assertions.java:86)\n\tat com.google.android.exoplayer2.source.ProgressiveMediaPeriod.selectTracks(ProgressiveMediaPeriod.java:281)\n\tat com.google.android.exoplayer2.source.MaskingMediaPeriod.selectTracks(MaskingMediaPeriod.java:186)\n\tat com.google.android.exoplayer2.MediaPeriodHolder.applyTrackSelection(MediaPeriodHolder.java:292)\n\tat com.google.android.exoplayer2.ExoPlayerImplInternal.reselectTracksInternal(ExoPlayerImplInternal.java:1643)\n\tat com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:503)\n\t... 3 more\n","errorCode":"2001","errorException":"com.google.android.exoplayer2.ExoPlaybackException: Unexpected runtime error","errorString":"ExoPlaybackException type : 2"}}

Please notice also that after the error occurs, onBufferProgress is still sent in loop

Edit: it is reproduced with all mp4 content
Edit2: In this PR I also add m3u and live streams samples it would be intersting to test with these new streams.

Thanks

armands-malejevs · 2022-06-07T09:24:01Z

@freeboub I fixed single track playback. Can you test again?

freeboub · 2022-06-07T20:25:19Z

@freeboub I fixed single track playback. Can you test again?

I confirm this is fixed, thanks

freeboub · 2022-06-07T20:34:04Z

android-exoplayer/src/main/java/com/brentvatne/exoplayer/ReactExoplayerView.java

+                        public void run() {
+                            if (player != null) {
+                                double bufferedDuration = (double) (player.getBufferedPercentage() * player.getDuration() / 100);
+                                eventEmitter.bufferProgress(0d, bufferedDuration);


@armadilio3 This is the last point to handle I think.
I still don't understand why you need to duplicate this event as information is already available in onProgress event.
once again there is a bug in onProgress which is blocking even when player is paused, it has to be solved at least for live playback.
Duplicating event can overload the bridge, so having too much event is a bad idea.

The advatage of onProgress is that timeout can already be configured.
I also think having 1 event with multiple parameters is lighter than have 2 events in terms of CPU consumption

armands-malejevs · 2022-06-08T08:32:56Z

@freeboub I'm still not sold on having the an independent value buffer tied to on progress events which seems non intuitive if the two are unrelated to one another, but to get this PR moving forward I removed it. We can open a separate PR to properly implement the feature and discuss separating or leaving in the progress event.

freeboub · 2022-06-08T16:44:19Z

Thank you @armadilio3 !
In the same time, can you please review this PR and check if you are now well receiving the event even if player is paused or have connectivity issue : #2664

EyMaddis · 2022-12-14T15:57:49Z

Force garbage collection when there is no available memory

I think this might be the culprit why our FireTV Sticks starting having frame drops, this works on 5.x versions of react-native-video

armands-malejevs and others added 30 commits March 15, 2021 16:51

Upgrade exoplayer to 2.13.2 (#2)

6100981

Upgrade ExoPlayer from 2.11.4 to 2.13.2 and fix any related issues related to the upgrade and deprecated method use.

VEX-4556: The app crashes when the premium episode is loaded (#4)

f6cce0d

Fix player crashing when it is being cleared.

VEX-4579: Network loss handling (#5)

8087310

Add support for customizing back buffer duration and handle network errors gracefully to prevent releasing the player when network is lost.

VEX-5044: Allow exoplayer to preinit with empty source (#6)

cba88fa

- Allow player to be init before source is provided, and later update once a source is provided. - Adds handling for providing a empty source in order to stop playback and clear out any existing content

VEX-3245: Buffer Progress UI While Paused (#7)

93604b2

Add support for onBufferProgress prop on Android to get buffer data even when the player is paused.

VEX-5545: Some Android codecs don't support specific resolutions (#8)

a0b679c

Add support for detecting if format is supported and exclude unsupported resolutions from auto quality selection and video track info in RN.

VEX-5825: Android player error handling improvements (#10)

2fcc1de

Improve error handling.

VEX-5798: Implement DRM L1 to L3 fallback (#9)

3bcf2c6

Add support for L1 to L3 Widevine fallback if playback fails initially.

VEX-5682: iOS Swift Conversion (#11)

8b75438

Converts ios implementation from objective-c to swift.

VEX-6030-reduce-buffer-size-based-on-heap

0ab7ff4

VEX-6030-reduce-buffer-size-based-on-heap

40e450b

add config parameter maxHeapAllocationPercent

7dbc5eb

VEX-6011: Align AVOD resolutions with available resolutions on Content (

f712eec

#14) Add support for content tracks and improve track selection to work even during ads playback.

force garbage collector when free memory reaches 0

2b310cb

typo

ab202c9

VEX-6365: Improve memory management (#17)

2ab0299

Improve memory management to reduce pressure on low end devices. JIRA: VEX-6365

add useSecureView prop to prevent screen recording

2af360a

Capture more errors from DRM session manager

b6f3652

Updates disableDisconnectError retry condition to be more specific (#22)

39dee21

Updates disableDisconnectError retry condition to be more specific. Jira: VEX-6518 Reviews Major reviewer (domain expert): @armadilio3 Minor reviewer: @gabriel-rivero

Move DRM init to a separate thread

88f32ae

Clean up

2f77765

Fix build issues

142521b

Fix typo

cca2744

Fix runtime crash

7067afa

hueniverse mentioned this pull request May 25, 2022

Provide a way to specify minimum DRM level for Widevine #2696

Closed

freeboub requested changes May 25, 2022

View reviewed changes

README.md Show resolved Hide resolved

gabriel-rivero approved these changes May 31, 2022

View reviewed changes

hueniverse added 1 and removed 2 labels May 31, 2022

Remove timeline reference from videoTrackInfo

9b04cfa

armands-malejevs requested a review from freeboub May 31, 2022 07:18

armands-malejevs added 2 commits June 2, 2022 15:08

Dont download dash manifest if contentStartTime is not set

0330943

Prevent unnecessary work when creating video track info

c7b7390

Fix single track playback

e90ab21

freeboub requested changes Jun 7, 2022

View reviewed changes

Remove buffer progress event

8a057c5

armands-malejevs requested a review from freeboub June 8, 2022 08:33

freeboub approved these changes Jun 8, 2022

View reviewed changes

hueniverse added bug feature and removed review requested labels Jun 9, 2022

hueniverse merged commit bf11bb9 into TheWidlarzGroup:master Jun 9, 2022

hueniverse added this to the 6.0.0-alpha1 milestone Jun 9, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Crunchyroll Android Improvements #2689

Crunchyroll Android Improvements #2689

armands-malejevs commented May 23, 2022

freeboub May 25, 2022

cobarx commented May 30, 2022

gabriel-rivero left a comment

armands-malejevs commented May 31, 2022

armands-malejevs commented Jun 2, 2022

freeboub commented Jun 3, 2022 •

edited

Loading

armands-malejevs commented Jun 7, 2022

freeboub commented Jun 7, 2022

freeboub Jun 7, 2022

armands-malejevs commented Jun 8, 2022

freeboub commented Jun 8, 2022

EyMaddis commented Dec 14, 2022 •

edited

Loading

Crunchyroll Android Improvements #2689

Crunchyroll Android Improvements #2689

Conversation

armands-malejevs commented May 23, 2022

Testing

freeboub May 25, 2022

Choose a reason for hiding this comment

cobarx commented May 30, 2022

gabriel-rivero left a comment

Choose a reason for hiding this comment

armands-malejevs commented May 31, 2022

armands-malejevs commented Jun 2, 2022

freeboub commented Jun 3, 2022 • edited Loading

armands-malejevs commented Jun 7, 2022

freeboub commented Jun 7, 2022

freeboub Jun 7, 2022

Choose a reason for hiding this comment

armands-malejevs commented Jun 8, 2022

freeboub commented Jun 8, 2022

EyMaddis commented Dec 14, 2022 • edited Loading

freeboub commented Jun 3, 2022 •

edited

Loading

EyMaddis commented Dec 14, 2022 •

edited

Loading