fix(YouTube): Fix video playback by switching to ReVanced GmsCore vendor

[oSumAtrIX]

About

This PR switches to the ReVanced GmsCore vendor to fix playback issues. The previous vendor did not update GmsCore, resulting in missing features required for playback, specifically PoToken, which was added to requests recently. Because the PoToken was missing, playback failed.

Related:

• [com.google.android.youtube] Video playback fails with HTTP error code 403 microg/GmsCore#1870
• POST request GetPoIntegrityTokenRequest returns no GetPoIntegrityTokenResponse body  microg/GmsCore#2253

Fixes #293

Todo

• Implement migration code or a dialog for com.mgoogle

Depends on:

• Bump to latest version GmsCore#2

[LisoUseInAIKyrios]

Any Gms option migration must be done in patches, since there is no integrations settings for Gms.

The simplest is to reject Vanced MicroG in the option validation of gmsCoreVendorOption:

            it!!.matches(Regex("^[a-z]\\w*(\\.[a-z]\\w*)+\$"))
                    && it != "com.mgoogle"

But to most users this will show what looks like an error in the patching logs (even though everything is ok):

INFO: Setting patch options 
WARNING: Could not set option value for the GmsCore support patch: The option value "com.mgoogle" failed validation for gmsCoreVendor 
INFO: Merging integrations 

Alternatively it could validate outside the patch options instance, and use the new default if the old Vanced MicroG is set.
Then no warnings or anything of concern, But there is no feedback the obsolete Vanced MicroG selection was ignored.

[oSumAtrIX]

Can't integrations show a dialog at start? The migration is only relevant for those that have actually touched the patch options or ReVanced CLI users in case they saved the option value to the json.

Additionally, the Logger facade can be used, or traditionally, the patch option validator, but that comes with its own merrits

[oSumAtrIX]

Perhaps ReVanced Manager can check for option values that aren't present anymore, unless the user chose a custom value

[LisoUseInAIKyrios]

If the user does not have the updated Gms core installed (and it was patched using the updated Gms), then
integrations cannot show a dialog because the app crashes immediately after the Gms hook is called. Integrations avoids the crash by calling System.exit(). Even attempting to show a dialog before the System.exit won't work, because Android throws a "Unable to add window" exception since the app is still starting up and there is nothing for the dialog to attach to.

The only thing that can be shown is a toast, since it's queued up before the app exits. A toast is already shown but it is very brief and only says "Gms Core is not installed. Install it", without any explanation that their old Vanced MicroG is obsolete and needs updating.

[oSumAtrIX]

On first launch the app will start a query for the vendor that was used for MicroG. The user is asked to install it via a toast as well. Once that is done, the app will launch normally at which point the dialog can be shown. The user would be told that the vendor they are using can not be used anymore properly and asked to switch the vendor by using the default vendor (app.revanced) for example.

[LisoUseInAIKyrios]

Or maybe temporarily change the website that is opened during launch.

Right now Integrations opens the website of https://github.com/revanced/gmscore/releases/latest if Gms is not installed. But this could be temporarily changed to a more informative website, explaining they need to uninstall any old MicroG and install this fixed version.

Then a few months from now the url can be changed back to the github latest releast page.

[oSumAtrIX]

Maybe I am misunderstanding something, but Vanced used com.mgoogle and ReVanced uses app.revanced as the group id. Users that have previously used Vanced, but haven't changed the patch options, will automatically be asked to install GmsCore forked by us, whereas those who have set the patch option to Vanced explicitly, will be asked to install GmsCore by Vanced. In both cases, once done, the application starts normally

[LisoUseInAIKyrios]

If the user is allowed to patch with the obsolete option of using Vanced MicroG, then Integrations can show an update dialog explaining the user needs to install an updated Gms Core and they need to repatch again and pick ReVanced as the Gms provider.

That would work, but why not forbid using Vanced MicroG during patching and save the user a lot of steps? Then they only need to install ReVanced Gms and they're done.

[oSumAtrIX]

Mainly because we have little options to do so before patching as you have explained. The user would see an error while patching and not know how to solve it, unlike with a dedicated popup at launch. Am I missing something?

[LisoUseInAIKyrios]

The patch options can be validated, and rejected if they are set to Vanced MicroG. Either silently reject (validate and override in the patch itself), or reject with more verbosity by validating in the patch options.

If the patch options are still set to Vanced MicroG, I think for now the best solution is to quietly ignore the patch option and use ReVanced Gms during patching. Then the user gets the correct Gms set during patching, no confusing error messages, and no stupid posts of "I updated and playback is still broken".

The integrations toast that normally says GmsCore is not installed. Please install it, can be temporarily changed to GmsCore is outdated. Please update. and it opens the usual github GmsCore release page.

[oSumAtrIX]

My problem with that is that we would forbid using Vanced MicroG whereas with the dialog on app launch the user is warned. Using the validator to check the input but returning true isn't necessary though as the option value can be checked in the execute method. But then either the patch is aborted with a message that they are forced to move away from Vanced MicroG, or the patch continues and the user ignores it or has to repatch leading to the same outcome as just displaying the dialog. Forbidding using Vanced MicroG or unexpectedly migrating off it is something that we should avoid

[LisoUseInAIKyrios]

I just checked, and a dialog cannot be shown during the GmsCore check in integrations because the context is not suitable for a dialog. (it is an Activity context, and not the context for an Android View).

The use case for allowing Vanced MicroG as a patch option:

1. User is informed playback is fixed (in app announcement, Discord, whatever)
2. User repatches as usual (but does not change their existing patch options saved in Manager)
3. User lauches app, sees a toast or a website is opened telling them to update GmsCore, and reset their patch options and repatch again
4. User installs updated GmsCore
5. User relaunches ReVanced Manager (again)
6. User changes patch options to default (be prepared for users to complain "what do I click on? how? huh?")
7. User patches YouTube (again)
8. Done

The use case for quietly migrating Vanced MicroG patch options to ReVanced GmsCore:

1. User is informed playback is fixed (in app announcement, Discord, whatever)
2. User repatches as usual (but does not change their existing patch options saved in Manager) -- their outdated Vanced option is silently updated, with no warnings during patching
3. User lauches app, sees a warning telling them to update GmsCore (but they do not need to repatch again)
4. User installs updated GmsCore
5. Done

[LisoUseInAIKyrios]

This is the simplest approach, to keep updating as simple and foolproof as possible.

Since there is no easy way to show a dialog in the Integrations Gms check, allowing Vanced MicroG makes it much more difficult informing the user what to do to fix their app.

[oSumAtrIX]

Regarding logging, you can use the regular logger facade by Java:

Logger.info()

[oSumAtrIX]

To clarify why I prefer to not hardcore com.mgoogle and not allow patching with it is because every vendor should be treated equally. While we deem com.mgoogle as unreliable or useless, what speaks of anyone creating their own fork with that group id. Why would we restrict exactly that group id then? Note, that there are vendors of GmsCore apart from Vanced that used the same group id. While they are something that we do not take into consideration, I am trying to clarify the problem that I am seeing with preventing to use an arbitrary group id.

I understand the overhead of the user needing to repatch, but this overhead will not be avoidable. You suggest to delay the process by silently migrating to GmsCore. Eventually you suggest to remove the migration code which means that at that point the user will face the issue you would be delaying til the time you remove the migration code because there is actually no way to migrate. Neither the patch nor the app at runtime can change the saved option value in ReVanced Manager and ReVanced Manager can not intervene as it is agnostic to patches. Introducing an API that solves this so that the patches can do exactly that would be a very fruitless change introduced just for the sake of fulfilling a single purpose instead of a generalized one (there is no practice case apart from this scenario).

I just realized, the shortcut by migrating directly to ReVanced is to not require the user to repatch an additional time. Regardless, this still prevents the user from using Vanced MicroG or any other vendor even if by choice. (For example unaffected users, those who have issues with our fork or who need the Huawei sign in)

[LisoUseInAIKyrios]

Nearly all unaffected users will soon become affected users. As of right now, there is no other forks with the updated GmsCore fix.

Could Manager do a one off migration, where on startup only once it clears the option values of GmsCore support? Because that would fix all of this and still allow specifying a different Gms if they intentionally changed it.

[LisoUseInAIKyrios]

I just realized, that since SpoofSignature was removed from client spoof, that means 99% of clients will have issues unless they also use the updated GmsCore (since signature spoofing is no more)

[oSumAtrIX]

Unfortunately ReVanced Manager follows the strict policy of being agnostic to any patch. Every patch that ReVanced Manager sees is the super type "Patch". What can be done instead is solve an existing open issue. ReVanced Manager can save what kind of option was selected. A preset or a custom value. In either case, it can handle the scenario in which a preset is removed such as in the current scenario. If the user did not manually enter com.mgoogle and instead selected the preset, when the preset is deleted from the patch, ReVanced Manager can for example fall back to the default value (with a warning dialog for example). ReVanced Manager now also has introduced an update dialog when ever an update is available which makes more users aware of when an update exists. Additionally we are working on counting how many users are on older versions and how many on newer versions so we can properly migrate off deprecated APIs (if you noticed, ReVanced Patches uses the deprecated ones as to not break patches for users that haven't updated ReVanced Manager to be able to use the new APIs).

Til now ReVanced Manager has not saved what kind of option value was selected (preset or custom value). In this case we need to default to either behavior. The correct one would be to treat them as custom values as to not surprise the user with fallbacking to a default value, but then this makes it useless for this current scenario, as this case would also be treated as a custom value whereas the fallback never happens.

[oSumAtrIX]

Adding to my previous comment, if ReVanced Manager would've implemented that feature, the migration code in the patch you pushed would not be needed anymore

[oSumAtrIX]

I just realized, that since SpoofSignature was removed from client spoof, that means 99% of clients will have issues unless they also use the updated GmsCore (since signature spoofing is no more)

That doesn't mean we should invalidate anyone else creating an updated version of GmsCore with the group id com.mgoogle. Previous to the last commit, any vendor was treated equally. The design was intentional to be flexible and free, but with the latest commit com.mgoogle is now impossible to use. I understand the reason behind that, but perhaps there is a better solution (such as the proposal with ReVanced Manager falling back to default value). For now, we can keep this, as to not waste time, and if we find a better solution along the way, we can quickly implement that 👍

[oSumAtrIX]

The problem with the migration idea is, even if it is temporary, once it is reverted or removed, the next time the user patches, they will be asked to install Vanced MicroG again. There is no way to persist the setting for those who have manually selected Vanced as the vendor (which is a little amount of people likely), unless the feature with falling back to default is added to ReVanced Manager (ideally for ReVanced CLi as well, thus subject to being implemented in ReVanced Library) for example.

[oSumAtrIX]

I think you are assuming that setting the value will also save it in ReVanced Manager. That is not the case, the option values are only set at the beginning but not read to be saved after patching, this can be a great addition to the API though. An onChanged handler parameter could be added to options so that ReVanced Patches can set the values and ReVanced Manager and co can handle that. Kotlin supports properly delegation which means the setter can be delegated, one that can be hooked by ReVanced Manager. Its slightly off topic for now though

[LisoUseInAIKyrios]

Then the migration logic could be kept until Manager better handles defaults options that change between patch releases.

If there was logging available to patches it would be helpful to print out something informing an option migration is applied (and keep the language in a "everything is ok" tone). But as far as I know only system println can be used and I don't know if that works with Manager.

I understand that changing the option value during patches does not change what is saved to disk. It would be useful if patches could alter the saved options, but the use case is pretty limited.

[oSumAtrIX]

@LisoUseInAIKyrios The regular Java logging facade can be used by patches. Everything else is automatically handled, I can push that in a minute

[MarcaDian]

I uninstalled Vanced MicroG, installed ReVanced GmsCore (https://github.com/ReVanced/GmsCore/releases/tag/v0.3.1.1.240913), and added a Google account to it.
I added all the permissions I had.

I patched the latest dev.13 recommended version of YouTube. I open it and it crashes.
Tell me what I did wrong.

full log 5s. of crash (my first log), if add findstr "revanced" it empty
log.txt

[LisoUseInAIKyrios]

@MarcaDian The logs don't show anything ReVanced, except some weird window error when shutting down the app.

Try deleting your options.json file, then repatch with only GmsCore (--exclusive -i "GmsCore support") and verify the app launches.

[MarcaDian]

@LisoUseInAIKyrios same issue, app crashes

[ {
  "patchName" : "GmsCore support",
  "options" : [ {
    "key" : "gmsCoreVendorGroupId",
    "value" : "app.revanced"
  } ]
} ]

PS D:\Android\cli> java -jar revanced-cli-4.6.0-dev.1-all.jar patch --exclusive -i "GmsCore support" -b revanced-patches-4.4.0-dev.13.jar -m revanced-integrations-1.5.0-dev.9.apk -o out.apk youtube_19.09.37.apk
INFO: Loading patches
INFO: Decoding app manifest
INFO: Remember video quality excluded
INFO: Playback speed excluded
INFO: Restore old video quality menu excluded
INFO: Video ads excluded
INFO: Hide ads excluded
INFO: Disable resuming Shorts on startup excluded
INFO: Disable auto captions excluded
INFO: SponsorBlock excluded
INFO: Change start page excluded
INFO: Theme excluded
INFO: Alternative thumbnails excluded
INFO: Disable fullscreen ambient mode excluded
INFO: Hide Shorts components excluded
INFO: Hide breaking news shelf excluded
INFO: Hide album cards excluded
INFO: Hide layout components excluded
INFO: Disable suggested video end screen excluded
INFO: Hide floating microphone button excluded
INFO: Hide crowdfunding box excluded
INFO: Hide endscreen cards excluded
INFO: Comments excluded
INFO: Disable rolling number animations excluded
INFO: Hide timestamp excluded
INFO: Player flyout menu excluded
INFO: Hide info cards excluded
INFO: Hide 'Load more' button excluded
INFO: Hide filter bar excluded
INFO: Hide seekbar excluded
INFO: Return YouTube Dislike excluded
INFO: Tablet mini player excluded
INFO: Enable tablet layout excluded
INFO: Wide searchbar excluded
INFO: Disable player popup panels excluded
INFO: Remove player controls background excluded
INFO: Custom player overlay opacity excluded
INFO: Navigation buttons excluded
INFO: Hide captions button excluded
INFO: Hide autoplay button excluded
INFO: Hide video action buttons excluded
INFO: Hide player buttons excluded
INFO: Hide cast button excluded
INFO: Custom branding excluded
INFO: Change header excluded
INFO: Restore old seekbar thumbnails excluded
INFO: Spoof app version excluded
INFO: Always repeat excluded
INFO: Announcements excluded
INFO: Bypass URL redirects excluded
INFO: Open links externally excluded
INFO: Disable zoom haptics excluded
INFO: Client spoof excluded
INFO: Spoof device dimensions excluded
INFO: Minimized playback excluded
INFO: Enable debugging excluded
INFO: Remove tracking query parameter excluded
INFO: Remove viewer discretion dialog excluded
INFO: Copy video URL excluded
INFO: Downloads excluded
INFO: Swipe controls excluded
INFO: Enable slide to seek excluded
INFO: Seekbar tapping excluded
INFO: Disable precise seeking gesture excluded
INFO: Spoof SIM country excluded
INFO: Remove screenshot restriction excluded
INFO: Spoof Wi-Fi connection excluded
INFO: Override certificate pinning excluded
INFO: Enable Android debugging excluded
INFO: Change package name excluded
INFO: Export all activities excluded
INFO: Predictive back gesture excluded
INFO: Remove screen capture restriction excluded
INFO: Setting patch options
INFO: Merging integrations
INFO: Deleting existing temporary files directory
INFO: Decoding resources
INFO: Executing patches
INFO: GmsCore support succeeded
INFO: Change package name succeeded
INFO: Compiling patched dex files
INFO: Compiled classes.dex
INFO: Compiled classes2.dex
INFO: Compiled classes3.dex
INFO: Compiled classes4.dex
INFO: Compiled classes5.dex
INFO: Compiled classes6.dex
INFO: Compiled classes7.dex
INFO: Compiled classes8.dex
INFO: Compiling modified resources
INFO: Aligning APK
INFO: Signing APK
INFO: Saved to D:\Android\cli\out.apk

[LisoUseInAIKyrios]

Try clearing the app data? The logs should show something if the app crashes.

[MarcaDian]

I'm doing a clean install

log.txt

maybe

03-26 12:12:31.817 26754 26775 I SpoofUtils: Package name of app.revanced.android.youtube spoofed to com.google.android.youtube
03-26 12:12:31.818 26754 26775 I SpoofUtils: Package name of app.revanced.android.youtube spoofed to com.google.android.youtube
03-26 12:12:31.822 26754 26775 I SpoofUtils: Package signature of app.revanced.android.youtube spoofed to 24bb24c05e47e0aefa68a58a766179d9b613a600
03-26 12:12:31.822 26754 26775 W GmsAuthProvider: Not granting extended access to [app.revanced.android.youtube], signature: 24bb24c05e47e0aefa68a58a766179d9b613a600
...
03-26 12:12:31.823 27829 27829 E Auth    : [GoogleAuthUtil] Exception when getting accounts
03-26 12:12:31.823 27829 27829 E Auth    : java.lang.SecurityException: Access denied, missing google package permission or GET_ACCOUNTS

[cyberboh]

@MarcaDian Could be caused by this PR ReVanced/revanced-integrations#589 isn't merged yet

[aliernfrog]

I'm doing a clean install

log.txt

maybe

03-26 12:12:31.817 26754 26775 I SpoofUtils: Package name of app.revanced.android.youtube spoofed to com.google.android.youtube
03-26 12:12:31.818 26754 26775 I SpoofUtils: Package name of app.revanced.android.youtube spoofed to com.google.android.youtube
03-26 12:12:31.822 26754 26775 I SpoofUtils: Package signature of app.revanced.android.youtube spoofed to 24bb24c05e47e0aefa68a58a766179d9b613a600
03-26 12:12:31.822 26754 26775 W GmsAuthProvider: Not granting extended access to [app.revanced.android.youtube], signature: 24bb24c05e47e0aefa68a58a766179d9b613a600
...
03-26 12:12:31.823 27829 27829 E Auth    : [GoogleAuthUtil] Exception when getting accounts
03-26 12:12:31.823 27829 27829 E Auth    : java.lang.SecurityException: Access denied, missing google package permission or GET_ACCOUNTS

Granting contacts permission to patched YouTube works. But it doesn't show any accounts even after logging in.

[oSumAtrIX]

The issue is known but kinda difficult to fix in certain terms. Basically you are missing YouTube on your device. How the fork works is, if you launch the patched app, GmsCore checks the app for whitelisted signatures. Because YouTube is modified, it fails these checks. This is why the fork includes a small workaround by asking for patched apps to include a spoof package name and signature meta in the manifest. GmsCore checks for that and spoofs the app to the original app. But because you don't have the original app installed GmsCore can not check for whatever it checks. Now, we could go as far as to simply check if the app is installed on the device but that would mean changing more in GmsCore than we should for the purpose of spoofing. We added the necessary code to spoof and workaround the whitelist, but checking if the original app exists in GmsCore is something that we arent supposed to look into, even if it's easy to do.

To note, because we work around the whitelist checks of GmsCore, this is an open hole for any kind of malware to abuse the same spoofing method to gain access to your Google account. This was the case with the previous vendor of GmsCore as well so we have no regression here.

The solution to that, while offtopic, is to make GmsCore ask the user for confirmation to use the spoofed app. If malware attempts to use GmsCore, the user would first need to grant permission, same would be in the case of the GmsCore support patch.

[MarcaDian]

@oSumAtrIX Yes, my factory YouTube was frozen. I just installed standard YouTube from the market and everything worked. Thank you very much.

[oSumAtrIX]

I don't know what frozen means, but even when installed as a system app and disabled on Android, it should work just fine

[MarcaDian]

I'll probably leave this here for future users who will also encounter this. Maybe this will be useful for them.

I used the XiaomiADBFastbootTools program (A simple tool for Xiaomi devices, utilising ADB and Fastboot) to freeze standard applications pre-installed by the vendor. I don’t know the exact commands that it sends, but after them the standard YouTube disappears from the installed ones, but the files remain in the firmware, since after the reset everything returns as it was.

[BlueStarBRS]

What do you mean by changing more then it should be in gmscore ?
And what have google changed that have this big impact?

[ILoveOpenSourceApplications]

Is there a reason why #2878 is still left open? (Wanted to ask there but it's locked so asking over here)

[oSumAtrIX]

It was kept open so that people don't open duplicates, can be closed now.