Add a retry attempt when copying files #245
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dellis1972
commented
Oct 15, 2024
| /// set. | ||
| /// </summary> | ||
| /// <returns>The value of DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS or the default of DEFAULT_FILE_WRITE_RETRY_ATTEMPTS</returns> | ||
| public static int GetFileWriteRetryAttempts () |
There was a problem hiding this comment.
I made these public so we can use them from XA.
There was a problem hiding this comment.
Do we have an other environment variables like this? I wonder if they should go in a class like KnownEnvironment.cs.
jonpryor
reviewed
Oct 15, 2024
$(LangVersion)=13 breaks CI: ``` Invalid option '13.0' for /langversion. Use '/langversion:?' to list supported values. ``` Let's try 12.0?
dellis1972
changed the title
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Oct 18, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
This was referenced Oct 18, 2024
jonathanpeppers
approved these changes
Oct 22, 2024
| /// set. | ||
| /// </summary> | ||
| /// <returns>The value of DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS or the default of DEFAULT_FILE_WRITE_RETRY_ATTEMPTS</returns> | ||
| public static int GetFileWriteRetryAttempts () |
There was a problem hiding this comment.
Do we have an other environment variables like this? I wonder if they should go in a class like KnownEnvironment.cs.
[Microsoft.Android.Build.BaseTasks] retry when copying files (#245)
Context: https://github.com/dotnet/android/issues/9133
Context: https://learn.microsoft.com/visualstudio/msbuild/copy-task?view=vs-2022
We sometimes get collisions between the Design-Time-Build (or
AntiVirus) and our main build. This can result in errors such as:
Error (active) XALNS7019 System.UnauthorizedAccessException: Access to the path 'D:\Projects\MauiApp2\obj\Debug\net9.0-android\android\assets\armeabi-v7a\MauiApp2.dll' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.InternalDelete(String path, Boolean checkHost)
at Microsoft.Android.Build.Tasks.Files.CopyIfChanged(String source, String destination) in /Users/runner/work/1/s/xamarin-android/external/xamarin-android-tools/src/Microsoft.Android.Build.BaseTasks/Files.cs:line 125
at Xamarin.Android.Tasks.MonoAndroidHelper.CopyAssemblyAndSymbols(String source, String destination) in /Users/runner/work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Utilities/MonoAndroidHelper.cs:line 344
at Xamarin.Android.Tasks.LinkAssembliesNoShrink.CopyIfChanged(ITaskItem source, ITaskItem destination) in /Users/runner/work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/LinkAssembliesNoShrink.cs:line 161
at Xamarin.Android.Tasks.LinkAssembliesNoShrink.RunTask() in /Users/runner/work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/LinkAssembliesNoShrink.cs:line 76
at Microsoft.Android.Build.Tasks.AndroidTask.Execute() in /Users/runner/work/1/s/xamarin-android/external/xamarin-android-tools/src/Microsoft.Android.Build.BaseTasks/AndroidTask.cs:line 25 MauiApp2 (net9.0-android) C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.99.0-preview.6.340\tools\Xamarin.Android.Common.targets 1407
If we look at the [MSBuild `<Copy/>` task][0] we see that it has a
retry system in the cases of `UnauthorizedAccessException` or
`IOException` when the code is `ACCESS_DENIED` or
`ERROR_SHARING_VIOLATION`. The `<Copy/>` task also has public
`Retries` and `RetryDelayMilliseconds` properties to control behavior.
Duplicate that kind of logic into our `Files.Copy*IfChanged()` helper
methods. This should give our builds a bit more resiliency to these
kinds of issues.
Instead of adding new `Files.Copy*IfChanged()` method overloads which
accept "retries" and "retryDelay" parameters, we instead use
environment variables to allow overriding these values:
* `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`: The number of times
to try to retry a copy operation; corresponds to the
`Copy.Retries` MSBuild task property.
The default value is 10.
* `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`: The amount of time,
in milliseconds, to delay between attempted copies; corresponds
to the `Copy.RetryDelayMilliseconds` MSBuild task property.
The default value is 1000 ms.
[0]: https://github.com/dotnet/msbuild/blob/main/src/Tasks/Copy.cs#L897 |
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Yes?
I'm not sure what a |
jonpryor
added a commit
to dotnet/android
that referenced
this pull request
Oct 22, 2024
Changes: dotnet/android-tools@ab2165d...60fae19 * dotnet/android-tools@60fae19: [Microsoft.Android.Build.BaseTasks] retry when copying files (dotnet/android-tools#245)
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Oct 22, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
jonathanpeppers
pushed a commit
to dotnet/android
that referenced
this pull request
Oct 22, 2024
Changes: dotnet/android-tools@ab2165d...60fae19 * dotnet/android-tools@60fae19: [Microsoft.Android.Build.BaseTasks] retry when copying files (dotnet/android-tools#245)
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Oct 23, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
jonathanpeppers
pushed a commit
to dotnet/android
that referenced
this pull request
Oct 23, 2024
Changes: dotnet/android-tools@ab2165d...60fae19 * dotnet/android-tools@60fae19: [Microsoft.Android.Build.BaseTasks] retry when copying files (dotnet/android-tools#245)
jonpryor
pushed a commit
to dotnet/java-interop
that referenced
this pull request
Oct 23, 2024
Changes: dotnet/android-tools@ab2165d...60fae19 * dotnet/android-tools@60fae19: [Microsoft.Android.Build.BaseTasks] retry when copying files (dotnet/android-tools#245) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Oct 29, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Nov 7, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Nov 24, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Nov 29, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Dec 4, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
github-actions bot
pushed a commit
to dellis1972/maui
that referenced
this pull request
Dec 4, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
dellis1972
added a commit
to dellis1972/maui
that referenced
this pull request
Dec 5, 2024
… by another process Fixes dotnet#25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second).
mattleibow
pushed a commit
to dotnet/maui
that referenced
this pull request
Dec 6, 2024
… by another process (#25374) * Creating .NET MAUI Project sometimes causes error *.png is being used by another process Fixes #25207 Context dotnet/android-tools#245 dotnet/android#9409 So there is a problem where the design time builds (DTB) of android sometimes lock files. This can happen when two processes try to write to the same file. This is not a great experience for our users as it just fails the build. So lets try a few things to fix this. 1. Move the Resizetizer output for a DTB into the android `$(IntermediateOutputPath)designtime` folder. This will keep the DTB files completely separate. This should prevent clashes. 2. Add some retry code to the `SkiaSharpTools.Save` method. This will catch `UnauthorizedAccessException` exceptions as well as specific `IOException` types (Access Denied and Sharing Violation). This will allow us to catch when this happens and retry the write. There is a small delay before attempting to write the file again. Note these code uses the Identical code as we are going to use in Android. We have introduced two new environment variables which can be used to control the new behavior. 1. `DOTNET_ANDROID_FILE_WRITE_RETRY_ATTEMPTS`. Integer, controls the number of times we try to write the file. The default is 10. 2. `DOTNET_ANDROID_FILE_WRITE_RETRY_DELAY_MS`. Integer, controls the delay in milliseconds between retry attempts. Default is 1000ms (or 1 second). * Back out the change which moves the 'r' directory to designtime * Slight change to make sure we do not loop forever * Exit if all is well
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context dotnet/android#9133.
We get collisions between the DTB (or AV) and our main build. This can result in the following error
If we look at the MSBuild Copy task we see that it has a retry system in the cases of
UnauthorizedAccessExceptionorIOExceptionwhen the code isACCESS_DENIEDorERROR_SHARING_VIOLATION. So lets duplicate that kind of logic in ourCopy*IfChangedhelper methods. This should give our builds a bit more resiliency to these kinds of issues.