Trying to record screen and stream the feed on Youtube Live, no errors but feed not showing up.

[sanketsirohiya]

Hi, I am trying to record my android screen and send the feed to YouTube Live. There are no errors that I see, but the feed is not visible on youtube. I have started a stream on YouTube Studio and used the stream URL and stream key in the code while calling the srsFlvMuxer.start(rtmpUrl) method. The permission for screen recording was also successfully given and recording notification is in place.

Have tried multiple things but am not able to get it to work. Would really appreciate some help here. Attaching the code files. Thanks in advance.

ScreenRecording.zip

[pedroSG94]

Hello,

First of all, that library version is really old. I recommend you use the last library version and replace srsFlvMuxer to RtmpClient.

Also, you have a code example to stream screen without implement encoder and others things yourself:
https://github.com/pedroSG94/RootEncoder/tree/master/app/src/main/java/com/pedro/streamer/screen

Anyway, I did a fast check in your code and I could see that you are not using setSPSandPPS method. You need do it when mediacodec return you "on output format changed":
https://github.com/pedroSG94/RootEncoder/blob/master/encoder/src/main/java/com/pedro/encoder/BaseEncoder.java#L197
You can get sps and pps from MediaFormat like here:
https://github.com/pedroSG94/RootEncoder/blob/master/encoder/src/main/java/com/pedro/encoder/video/VideoEncoder.java#L305

[sanketsirohiya]

Hi, thanks for your response. I tried to implement the code from the suggested link without the encoder etc. and updated the library version to the latest ( implementation 'com.github.pedroSG94.RootEncoder:library:2.4.9' ), updated my kotlin version to 2.0.0 that the library supports, but then ran into some errors from react-native-gesture-handler due to the change in kotlin version.

Error: Only safe (?.) or non-null asserted (!!.) calls are allowed on a nullable receiver of type 'com.swmansion.gesturehandler.core.GestureHandler<*>?'.

I tried updating react-native-gesture-handler to latest version but that did not work as well.

Can you please help with with an alternate version of your library that would work with Kotlin 1.8.0 and still let me use the approach without the encoder?

Or would you suggest I continue with the encoder approach from before and implement the setSPSandPPS method?

[pedroSG94]

Hello,

Did you try with kotlin 1.9.0? It is working for me

[pedroSG94]

If the problem persist I can create you a version with kotlin 1.8.0 but this is an error in react native.
This should be solved by them.

[pedroSG94]

Also, I found that an user resolved your error here:
software-mansion/react-native-gesture-handler#2427 (comment)

[sanketsirohiya]

Hi, thanks again for your prompt response.

I do agree that the error is from react-native-gesture-handler side, but I'm guessing they might take a while to fix this.

I also did try the last link you've shared to fix the error, but there were other errors in different files that came up.

You suggested building a version with Kotlin 1.8.0, if thats easy enough to do, it would mean a lot. Or I could go through the encoding steps myself by adding the setSPSandPPS method in my previous implementation.

What do you suggest?

[pedroSG94]

In this case I did a release using Kotlin 1.8.22.

This is an exception and I have no plan to continue the compilation in the future so try to migrate to use last Kotlin version as soon as possible.

implementation 'com.github.pedroSG94.RootEncoder:library:2.5.0-1.8.22'

[sanketsirohiya]

Hi, I am able to stream my screen to Youtube with the above version.

Really appreciate you taking the effort to compile it for me. I will try to upgrade to the latest whenever possible.

Can't thank you enough, cheers!

[sanketsirohiya]

Hi,

Regret to reopen this issue - the stream to Youtube worked fine, except for 1 issue: -

I have a live stream app, which is joined by other broadcasters apart from myself. My audio and my screen is perfectly streamed to Youtube but the incoming audio from the other broadcasters are not audible on the Youtube Live Stream. Is there a solution for this?

Please let me know if its simpler for you to track this in a separate issue, I can open it as a different one.

[pedroSG94]

Hello,

I don't understand your question.

What do you mean with "app which is joined by other broadcasters apart from myself"?
A video call or meet?

Keep in mind 2 things:

• If you are using Microphone audio source the audio provided by the phone is not detected. Sometimes you have an exception if the audio from speakers is captured by the microphone.

• If you are using Internal audio source, the audio/video capture can be restricted by the app. For example, if you open Netflix, the audio and video can't be captured because the app block it.

[sanketsirohiya]

Hi, thanks again for your quick response.

Yes it is like a video call which is joined by me + other broadcasters. Audio from my microphone is captured but the incoming audio from the other broadcasters which is being output from my phone is not captured.

Is there a solution for this?

[pedroSG94]

Hello,

Currently the library only support audio from microphone or from the device. Get both audios is not possible for now.
I have plan to implement this feature but it is not easy because I need mix both PCM audios and sync it.

[sanketsirohiya]

I understand, although it breaks my heart to know this.

Are you aware of any other source that I can use for this case? Alternatively, just to get a sense, do you have any rough idea about when we can expect the library to support both the device and mic audio?

[pedroSG94]

I'm not sure about the time. I'm already working on it and I have a first version but with really bad results.

The audio is missed and sync but the result has a really bad quality but I don't know the exact reason so the release date for this feature depend of my free time to do it and the luck debugging the case.

Also, for now. I only tested in mono and 44100 sample rate so maybe after solve it for this config. I need fix it for others configurations.

You can follow the development and collaborate if you want. This is the PR:
#1571

[noho501]

Hi @pedroSG94

Is this the same issue where, when I turn on the microphone, the sound from the PUBG game isn’t recorded, but when I turn off the microphone, the game sound is recorded? Please see the attached video: https://drive.google.com/file/d/12JBlI0wn-AgErcMbONOjc9CMFsU46HiC/view?usp=drivesdk

This is code we use:

var muted = false

fun toggleMute() {
    if (muted) {
      genericStream.changeAudioSource(MicrophoneSource())
      muted = false
    } else {
      mediaProjection?.let {
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
          genericStream.changeAudioSource(InternalAudioSource(it))
          muted = true
        }
      }
    }
}

[pedroSG94]

Yes, this is the same case.

In your code you change microphone to internal audio.
Internal audio is used to capture audio from apps and microphone capture only audio from the device microphone.

I'm working on a source that mix both audios, allowing capture both at the same time.

[noho501]

Thank you for your reply. Do you know how to switch to the brand feature/audio-mix-source branch so we can test it with our source?

[pedroSG94]

Hello,

I just finish the development. I'm currently doing tests in different sample rate, channels, etc to confirm that all is working fine).
That branch currently allow you use AudioMixSource that include both. Microphone and InternalAudio sources. Also, allow control the volume of both sources.
If you want use the branch, clone the project with git and change to the branch using:

git checkout feature/audio-mix-source

Remember do a git pull if you already have the project cloned.
Also, you can download the project directly in that branch as zip, unzip and open it in Android studio:
https://github.com/pedroSG94/RootEncoder/archive/refs/heads/feature/audio-mix-source.zip

[noho501]

Thank you so much. We will check it out and get back to you.

[sanketsirohiya]

Hi, great to hear the progress on the mix audio source, eagerly waiting!

In the meantime, could you please help me with one other problem? The stream on Youtube is coming up in Landscape mode, is there a way to switch it to Portrait though code or any other setting that you are aware of?

Thanks!

[pedroSG94]

Hi, great to hear the progress on the mix audio source, eagerly waiting!

In the meantime, could you please help me with one other problem? The stream on Youtube is coming up in Landscape mode, is there a way to switch it to Portrait though code or any other setting that you are aware of?

Thanks!

Hello,

You only need set rotation in prepareVideo method to 90. In the app example using rotation example you can change it in the menu -> orientation -> vertical.

Remember that using the app this change have effect while you are not recording/streaming, because you need configure it in prepareVideo, and you can't call it while recording/streaming.

https://github.com/pedroSG94/RootEncoder/blob/master/app/src/main/java/com/pedro/streamer/rotation/CameraFragment.kt#L160

[sanketsirohiya]

Hi, setting the rotation works! However to capture the full screen I retrieved the device height and width (2209 and 1080 respectively) and on passing this to the prepareVideo function, I am getting the following error: -

Create VideoEncoder failed. java.lang.IllegalArgumentException:

It works with the hardcoded height = 480, width = 640, as seen on the sample app but I there was either a lot of unnecessary whitespace around the recorded screen/ or sometimes I even noticed a part of the screen being captured as expected by the landscape nature of the 480 Height and 640 Width. Am I missing something?

I referred to the top few answers here to get the height and width -
https://stackoverflow.com/questions/4743116/get-screen-width-and-height-in-android#:~:text=Display%20display%20%3D%20getWindowManager().,x%3B%20int%20height%20%3D%20size.

[pedroSG94]

hello,

2209 is not a valid value you need set even values. Also, depend of your video source if you are using camera2 the value must be between the min and the max camera values. You have the rule here:
https://github.com/pedroSG94/RootEncoder/blob/master/library/src/main/java/com/pedro/library/util/sources/video/Camera2Source.kt#L65

[pedroSG94]

My recommendation is use a value listed in getCameraResolutions(face) method

[sanketsirohiya]

Using even numbers worked! Thanks again for the quick support.

Hate to trouble again and again, but is there also a way to save the recorded media on the device as an mp4 or something?

[sanketsirohiya]

Youtube allows downloading the Stream after a few hours and that sometimes poses as a blocker.

[pedroSG94]

Hello,

Try using startRecord and stopRecord methods. You can start or stop while streaming or without streaming.

Example here:

https://github.com/pedroSG94/RootEncoder/blob/master/app/src/main/java/com/pedro/streamer/rotation/CameraFragment.kt#L132

[sanketsirohiya]

Thanks for sharing this. Tried it out and it works really well. :)

[pedroSG94]

MixAudioSource merged to master.
You can test it in the app example -> screen example -> menu -> mix audio

Closing as solved, you can reopen it if you find a bug or create other issue

[noho501]

Hello,

I just finish the development. I'm currently doing tests in different sample rate, channels, etc to confirm that all is working fine). That branch currently allow you use AudioMixSource that include both. Microphone and InternalAudio sources. Also, allow control the volume of both sources. If you want use the branch, clone the project with git and change to the branch using:

git checkout feature/audio-mix-source

Remember do a git pull if you already have the project cloned. Also, you can download the project directly in that branch as zip, unzip and open it in Android studio: https://github.com/pedroSG94/RootEncoder/archive/refs/heads/feature/audio-mix-source.zip

Hi @pedroSG94

It’s not working in my case. When playing PUBG, if I turn on the in-game microphone and then enable the microphone in my app, no audio is recorded in the video or live stream. However, if I turn off my app’s microphone, the audio is recorded properly. Any suggestions?

[pedroSG94]

Hello,

Did you try using master branch with screen example and change audio source using the menu? Can you try open YouTube instead of PUBG and tell me the result?

[noho501]

Hi @pedroSG94,

YouTube is working fine, but the problem occurs when the in-game or in-app microphone is enabled. I believe this is related to granting microphone permissions to the application, which results in our app being unable to record audio to the video file.

[pedroSG94]

I see.

PUBG app get the microphone producing no audio result. This has logic because the new class wait capture the microphone and device audio and then mix it. If no audio is received, never produce audio.

I'm not sure if we can reproduce this case with other audio app. Use PUBG sounds like a pain to debug.
Can you confirm me if this happens with an app to record audio?

[pedroSG94]

I think that capturing the microphone at the same time that other apps may not be possible but I can try to recover the microphone audio after the app stops using it and meanwhile only sends device audio

[noho501]

Hi @pedroSG94

Do you think we should use an external microphone to avoid conflicts with the in-game or in-app microphone during recording?

P/s: Recorder app the same pubg game. No audio is recorded to live stream or video file.

https://stackoverflow.com/questions/74135716/is-there-any-way-to-determine-which-mic-of-android-phone-is-being-used-for-recor#:~:text=Most%20android%20smartphone%20has%20two,and%20One%20at%20the%20bottom.

[sanketsirohiya]

Hi @pedroSG94 , Glad to hear that the Mix Audio Support is now complete. Can you confirm if there is a new version that you have released with this? I tried to sync my project but didn't see the MixAudioSource as part of the source.

To remind, I am using this version currently, that you had released for kotlin 1.8 :-
implementation 'com.github.pedroSG94.RootEncoder:library:2.5.0-1.8.22'

[pedroSG94]

Hello,

No, it is not released yet because a bug was reported where the audio is not working if other app capture microphone.
In this case this app lose the access to microphone and audio stop send. I want fix it first.

For now, a solution is copy the source code of the new Audio source:
https://github.com/pedroSG94/RootEncoder/blob/master/library/src/main/java/com/pedro/library/util/sources/audio/MixAudioSource.kt

If you have a suggestion or fix, it is welcome.

[sanketsirohiya]

Hi @pedroSG94 ,

I tried to copy the source code for MixAudioSource.kt and VolumeEffect.kt as suggested, but am not able to use some methods/members (Screenshot Attached)

Am I missing something?

Thanks.

[pedroSG94]

Hi @pedroSG94 ,

I tried to copy the source code for MixAudioSource.kt and VolumeEffect.kt as suggested, but am not able to use some methods/members (Screenshot Attached)

Am I missing something?

Thanks.

Hi @pedroSG94 ,

I tried to copy the source code for MixAudioSource.kt and VolumeEffect.kt as suggested, but am not able to use some methods/members (Screenshot Attached)

Am I missing something?

Thanks.

Hi @pedroSG94 ,

I tried to copy the source code for MixAudioSource.kt and VolumeEffect.kt as suggested, but am not able to use some methods/members (Screenshot Attached)

Am I missing something?

Thanks.

Aaah, true. That methods was added in the same PR that this new source so it is not available. For now, test it using the default app example.
I have plan to fix it and upload a new version with all the code as soon as all bugs are solved

[sanketsirohiya]

Sure, I'll wait for the positive news then. Hoping you can release one with kotlin 1.8.22 as well. :)

[pedroSG94]

Hi @pedroSG94

Do you think we should use an external microphone to avoid conflicts with the in-game or in-app microphone during recording?

P/s: Recorder app the same pubg game. No audio is recorded to live stream or video file.

https://stackoverflow.com/questions/74135716/is-there-any-way-to-determine-which-mic-of-android-phone-is-being-used-for-recor#:~:text=Most%20android%20smartphone%20has%20two,and%20One%20at%20the%20bottom.

Hello,

I was looking about this case. According with my tests if other app try to capture audio from the microphone the microphone receive silence but the device audio works. After close the app that block the microphone the audio is automatically restored and work as expected.
Depend of audio source set to the microphone instance you can modify a bit the way this works. You have the info about it here:
https://github.com/pedroSG94/RootEncoder/blob/master/encoder/src/main/java/com/pedro/encoder/input/sources/audio/MixAudioSource.kt#L52

No matter what you can't capture the audio in two apps at the same time but maybe using an external microphone you can solve it. I'm not sure if you can capture audio in both apps using the external microphone or you need use both microphones at the same time to work.
I haven't an external microphone to test it for now.

For example, the record screen from the system with Pixel device (that allow record both, device and microphone audio at the same time) seems to use MediaRecorder.AudioSource.VOICE_COMMUNICATION because if you try record audio from other app you always receive silence.

[noho501]

Hi @pedroSG94 ,

Thank you so much for your help. With the MediaRecorder.AudioSource.DEFAULT and MediaRecorder.AudioSource.MIC options, the microphone can be restored once another app releases it. I tested it with another app, and it seems there’s no way to capture audio in two apps simultaneously, as can be done on iOS. Based on this, I think we are ready to proceed with updating the release version.

Best regards,

[pedroSG94]

Hello,

I have plans to release the version this week.

This version changes the package of few classes so you will need to remove the import and import again.

[sanketsirohiya]

Hey @pedroSG94, just wanted to follow up on the release activity for Mix Audio Source? When can we expect it?

[pedroSG94]

Hello,

MixAudioSource is already available in the version 2.5.1

I'm closing this issue as completed. Open other issue if you have questions or problems