[Bug] application deploy on Android 11 emulator fails with Mono.AndroidTools.AdbException: secure_mkdirs failed: Permission denied

[pmahend1]

Description

Last week upgraded Visual Studio 2019(Professional) which shipped Android 11 SDK updates
I changed the target SDK in both manifest and project application properties.
When I try to deploy the app I get the error. This error doesnt show up for Android Pie and Q emulators

Steps to Reproduce

1. Update Visual Studio 2019 to 16.7
2. Update all android SDKs
3. Create an x86 emulator with Android 11(R)
4. Build and Run Android app selecting Android 11 emulator

Expected Behavior

App should deploy and run successfully.

Actual Behavior

Severity	Code	Description	Project	File	Line	Suppression State
Error		ADB1000:  Deployment failed
Xamarin.AndroidTools.AndroidDeploymentException: FailedToSynchronizeFastDevAssemblies ---> Mono.AndroidTools.AdbException: secure_mkdirs failed: Permission denied
   at Mono.AndroidTools.Util.AggregateAsyncResult.CheckError(CancellationToken token)
   at Mono.AndroidTools.Adb.AdbSyncClient.EndPushSyncItems(IAsyncResult result)
   at Mono.AndroidTools.Internal.AdbClientTaskExtensions.InnerInvoke(TaskCompletionSource`1 tcs, Func`2 endMethod, IAsyncResult l)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Xamarin.AndroidTools.AndroidDeploySession.<InstallAssemblies>d__121.MoveNext()
   --- End of inner exception stack trace ---
   at Xamarin.AndroidTools.AndroidDeploySession.<InstallAssemblies>d__121.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Xamarin.AndroidTools.AndroidDeploySession.<FastDevAsync>d__118.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Xamarin.AndroidTools.AndroidDeploySession.<RunAsync>d__110.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Xamarin.AndroidTools.AndroidDeploySession.<RunLoggedAsync>d__108.MoveNext()		 	0	

Basic Information

• Version with issue: Xamarin Forms 4.8.0.XXXX
• Last known good version: Not sure
• IDE: Visual Studio 2019 Professional
• Platform Target Frameworks:
  • iOS: N/A
  • Android: Android 11
  • UWP: N/A
• Android Support Library Version:
• Nuget Packages: Xamarin Forms 4.8.0.XXXX Xamarin.Android.Support 28.0.0.3
• Affected Devices:

Screenshots

Reproduction Link

Workaround

Turn Use Fast Deployment Off
Reference : https://forums.xamarin.com/discussion/11383/deployment-failed-due-to-an-error-in-fastdev-assembly-synchronization

[pmahend1]

Update

I noticed this exception goes away if we turn Use Fast Deployment off

So this can be thought as a workaround

[pmahend1]

And I could find related Visual Studio bug as raised in community as well

https://developercommunity.visualstudio.com/content/problem/1075071/xamarinandroid-deployment-often-fails.html

[dellis1972]

@pmahend1 can you provide a full diagnostic deployment log? It would be interesting to see where its trying to fast deploy the assemblies to. I suspect we no longer have write permissions to the directory we try to use for fast deployment.

The easiest way to get use a deployment log is to turn fast dev back on, un install the app from the device/emulator then do the following from the Visual Studio command prompt.

msbuild <your android project.csproj> /t:Install /bl

This will provide a msbuild.binlog file which we can then investigate.

You can also capture this from the IDE, but you'll need to make sure that the log output is set to diagnostic and that you capture the deployment, not the build log as the build log won't have the data we need.

[pmahend1]

@dellis1972 logs have been shared with you over email.

[dellis1972]

@pmahend1 thanks for the logs.

Ok so the odd thing is this directory it is trying to create /storage/emulated/0/Android/data/com.companyname.app5/files/.__override__ is a perfectly valid directory.
It looks like Android R has changed the external storage system/capabilities https://developer.android.com/preview/privacy/storage. Looks like apps with no longer be able to use the external storage directory.

There are some commands to try , this for example will enable management of external storage.

adb shell appops set --uid PACKAGE_NAME MANAGE_EXTERNAL_STORAGE allow

You could also try adding preserveLegacyExternalStorage to your manifest. see if that helps. Although you probably don't want to do that for release apps.

We are working on a new Fast Deployment system which will be compatible with Android R, but that is not slated for release until 16.9 (16.8 is the one currently in preview).

[Mertsch]

Caution: After you update your app to target Android 11 (API level 30),
the system ignores the requestLegacyExternalStorage attribute when your app is running on Android 11 devices,
so your app must be ready to support scoped storage and to migrate app data for users on those devices.

https://developer.android.com/training/data-storage/use-cases

[pmahend1]

@Mertsch Thanks . yes, looks like setting requestLegacyExternalStorage doesnt do any good for apps running on Android 11. Looks like its only for devices with API=<29 which have been built with API 30.

[dellis1972]

More investigation https://developer.android.com/reference/android/os/Environment#getExternalStorageDirectory().

This method was deprecated in API level 29.
To improve user privacy, direct access to shared/external storage devices is deprecated. When an app targets Build.VERSION_CODES.Q, the path returned from this method is no longer directly accessible to apps. Apps can continue to access content stored on shared/external storage by migrating to alternatives such as Context#getExternalFilesDir(String), MediaStore, or Intent#ACTION_OPEN_DOCUMENT.

So when targeting Q, the external storage directory will not work.

This will be addressed in PR #4690 with the new Fast Deployment system which does not use the external storage directory.

The best thing to do at this time is to disable Fast Deployment when targeting Android Q :(.

[dellis1972]

@pmahend1 So I've been doing some more investigation. I have a version of our shared runtime which used newer api's to get the external storage path.
If you have time would you mind testing out the following packages [1]. Its a zip file which contains all the Mono.Android.DebugRuntime-*.apk files. you will need to extract them and replace the ones in your d16-7 installation directory. Be sure to backup the old ones just in case.

The directory should be in C:\Program Files (x86)\Microsoft Visual Studio\2019\<edition>\MSBuild\Xamarin\Android

Let me know if this works. If it does then I might be able to get this into the next d16-8 preview.

[1] https://1drv.ms/u/s!Amsoudv0Ow8JnXGGZbXp_-kPHtT9?e=v2qEZc