Android: AOT compiler fails on Windows in directory named `BuildAot AndÜmläüts` #88625

jonathanpeppers · 2023-07-10T19:59:11Z

Description

This happened during code flow here: dotnet/android#8176

Microsoft.Android.Sdk.Aot.targets(109,5): error : The system cannot find the file specified.
Microsoft.Android.Sdk.Aot.targets(109,5): error : AOT of image C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot And├£ml├ñ├╝ts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\_Microsoft.Android.Resource.Designer.dll failed.
Microsoft.Android.Sdk.Aot.targets(109,5): error : Mono Ahead of Time compiler - compiling assembly C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot And├£ml├ñ├╝ts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\_Microsoft.Android.Resource.Designer.dll
Microsoft.Android.Sdk.Aot.targets(109,5): error : AOTID 289CA072-6C75-216D-F0DC-CB1E0B63A631
Microsoft.Android.Sdk.Aot.targets(109,5): error : Using profile data file 'dotnet.aotprofile'
Microsoft.Android.Sdk.Aot.targets(109,5): error : Added 0 methods from profile.
Microsoft.Android.Sdk.Aot.targets(109,5): error : Executing opt: "C:\a\_work\1\s\bin\Release\dotnet\packs\Microsoft.NETCore.App.Runtime.AOT.win-x64.Cross.android-x86\8.0.0-preview.7.23359.1\Sdk\..\tools\opt" -f -disable-tail-calls -passes='default<O2>,place-safepoints' -spp-all-backedges -o "obj\Release\android-x86\aot\x86\_Microsoft.Android.Resource.Designer\temp.opt.bc" "obj\Release\android-x86\aot\x86\_Microsoft.Android.Resource.Designer\temp.bc"
Microsoft.Android.Sdk.Aot.targets(109,5): error : Precompiling failed for C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot AndÜmläüts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\UnnamedProject.dll with exit code 1.
Microsoft.Android.Sdk.Aot.targets(109,5): error : The system cannot find the file specified.
Microsoft.Android.Sdk.Aot.targets(109,5): error : AOT of image C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot And├£ml├ñ├╝ts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\UnnamedProject.dll failed.
Microsoft.Android.Sdk.Aot.targets(109,5): error : Mono Ahead of Time compiler - compiling assembly C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot And├£ml├ñ├╝ts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\UnnamedProject.dll
Microsoft.Android.Sdk.Aot.targets(109,5): error : AOTID 1991E3CD-E6C4-BA31-1115-20CDEA468513
Microsoft.Android.Sdk.Aot.targets(109,5): error : Executing opt: "C:\a\_work\1\s\bin\Release\dotnet\packs\Microsoft.NETCore.App.Runtime.AOT.win-x64.Cross.android-x86\8.0.0-preview.7.23359.1\Sdk\..\tools\opt" -f -disable-tail-calls -passes='default<O2>,place-safepoints' -spp-all-backedges -o "obj\Release\android-x86\aot\x86\UnnamedProject\temp.opt.bc" "obj\Release\android-x86\aot\x86\UnnamedProject\temp.bc"

Reproduction Steps

mkdir 'BuildAot AndÜmläüts' && cd 'BuildAot AndÜmläüts'
dotnet new android
dotnet build -c Release

Expected behavior

Build succeeds.

Actual behavior

<MonoAOTCompiler/> MSBuild task fails.

Regression?

Yes

Known Workarounds

You could disable AOT with -p:RunAOTCompilation=false, but this is a default setting for Release builds.

Configuration

.NET SDK 8.0.100-preview.7.23360.1

Other information

It appears this may only be an issue on Windows?

The text was updated successfully, but these errors were encountered:

Context: dotnet/runtime#88625

lewing · 2023-07-11T03:28:17Z

cc @radical

radical · 2023-07-11T06:05:25Z

From the binlog:

: Precompiling failed for C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot AndÜmläüts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\_Microsoft.Android.Resource.Designer.dll with exit code 1. 
The system cannot find the file specified. 
AOT of image C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot And├£ml├ñ├╝ts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\_Microsoft.Android.Resource.Designer.dll failed. 
Mono Ahead of Time compiler - compiling assembly C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot And├£ml├ñ├╝ts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\_Microsoft.Android.Resource.Designer.dll 
AOTID 289CA072-6C75-216D-F0DC-CB1E0B63A631 
Using profile data file 'C:\a\_work\1\s\bin\Release\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.0-ci.darc-main-33fa432d-4d4a-4d5d-9e54-6af849b9f4e5.367\targets\dotnet.aotprofile' 
Added 0 methods from profile. 
Executing opt: "C:\a\_work\1\s\bin\Release\dotnet\packs\Microsoft.NETCore.App.Runtime.AOT.win-x64.Cross.android-x86\8.0.0-preview.7.23359.1\Sdk\..\tools\opt" -f -disable-tail-calls -passes='default<O2>,place-safepoints' -spp-all-backedges -o "obj\Release\android-x86\aot\x86\_Microsoft.Android.Resource.Designer\temp.opt.bc" "obj\Release\android-x86\aot\x86\_Microsoft.Android.Resource.Designer\temp.bc"

: Precompiling failed for C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot AndÜmläüts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\UnnamedProject.dll with exit code 1. 
The system cannot find the file specified. 
AOT of image C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot And├£ml├ñ├╝ts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\UnnamedProject.dll failed. 
Mono Ahead of Time compiler - compiling assembly C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot And├£ml├ñ├╝ts_armeabiv7ax86_True_True\obj\Release\android-x86\linked\UnnamedProject.dll 
AOTID 1991E3CD-E6C4-BA31-1115-20CDEA468513 
Executing opt: "C:\a\_work\1\s\bin\Release\dotnet\packs\Microsoft.NETCore.App.Runtime.AOT.win-x64.Cross.android-x86\8.0.0-preview.7.23359.1\Sdk\..\tools\opt" -f -disable-tail-calls -passes='default<O2>,place-safepoints' -spp-all-backedges -o "obj\Release\android-x86\aot\x86\UnnamedProject\temp.opt.bc" "obj\Release\android-x86\aot\x86\UnnamedProject\temp.bc"

The first line in the output (after the Precompiling failed..) is The system cannot find the file specified. . This looks suspicious - is this from mono-aot-cross?
The failure (of Exec) seems to be from Executing opt run by mono-aot-cross, but no output from that AFAICS.

The runtime changes in that PR are 3e87528...3110693 .

It also includes b3d2503 - with the new as-prefix parameter, which is set via AsPrefix for the MonoAOTCompiler task. Does the same need to be passed to MonoAOTCompiler in:

https://github.com/xamarin/xamarin-android/blob/a850d8147d6c1d066f3849e7cf8fab231497c64b/src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.Aot.targets#L109-L130

cc @steveisok

steveisok · 2023-07-11T11:13:35Z

as-prefix is optional and I don't think is something they need to set.

@directhex could this possibly be related to LLVM changes?

jonathanpeppers · 2023-07-11T13:46:07Z

Sorry, forgot to post the .binlog: aot.zip (to make it easier to find)

It appears that only this one test with Ümläüts in the project directory is failing on Windows. So AOT is working in other cases.

Context: dotnet/runtime#88625

directhex · 2023-07-11T14:27:18Z

Where can I most easily download a suitably equipped .NET SDK 8.0.0-preview.7.23359.1 for windows to repro with?

jonathanpeppers · 2023-07-11T14:36:13Z

I think the exact build from the PR above is: https://dotnetbuilds.azureedge.net/public/Sdk/8.0.100-preview.7.23360.1/dotnet-sdk-8.0.100-preview.7.23360.1-win-x64.exe

Then you should be able to do:

dotnet workload install android --source https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json --source https://api.nuget.org/v3/index.json
mkdir 'BuildAot AndÜmläüts'
cd 'BuildAot AndÜmläüts'
dotnet new android
dotnet build -c Release

^^ untested, but hopefully that repros?

directhex · 2023-07-11T16:20:28Z

I can't satisfy myself that my environment is dependably "correct", but my testing seems to imply LLVM 16 is not the cause of issues.

Can someone who isn't me:

Repro the failures
Repro (or not) the failures with -p:EnableLLVM=false
Repro (or not) the failures in a new project in a folder with neither spaces nor non-ASCII symbols
Repro (or not) the failures in a new project in a folder with neither spaces nor non-ASCII symbols with -p:EnableLLVM=false

steveisok · 2023-07-12T02:45:55Z

I was able to confirm this has nothing to do with LLVM and has everything to do with the aot compiler not supporting non-ASCII symbols. @jonathanpeppers, I'm not sure how this test passed before.

The problem in either case is that the aot compiler tries to fopen the assembler file and fails. Only the non-llvm path checks if the file handle is valid and reports if it is not. The llvm path seems to fall through and that's why you see The system cannot find the file specified..

The fix would be to modify the aot compiler to support unicode files. I think that would be a pretty big, at the very least, validation task, so this is not something we can fix in the short term.

radical · 2023-07-12T02:59:53Z

Would passing relative paths to the aot compiler help? Though I think we had some problems with that in the past. And it would help only with unicode chars in the containing project directory name.

Update: and I can't say where it would fail next down the build pipeline!

jonathanpeppers · 2023-07-12T13:58:11Z

Would passing relative paths to the aot compiler help?

I suspect this is how it somehow worked before. We've had this test passing since 2020 or so: dotnet/android@28add0d

Maybe something changed from a relative path to a full path recently?

radical · 2023-07-12T14:22:27Z

Would passing relative paths to the aot compiler help?

I suspect this is how it somehow worked before. We've had this test passing since 2020 or so: xamarin/xamarin-android/commit/28add0dddf34ba99a97bac77fcfd23eb9f4e1697

Maybe something changed from a relative path to a full path recently?

cc @mdh1418

radical · 2023-07-12T14:32:59Z

The problem in either case is that the aot compiler tries to fopen the assembler file and fails. Only the non-llvm path checks if the file handle is valid and reports if it is not. The llvm path seems to fall through and that's why you see The system cannot find the file specified..

Could you please point to the code for this? I'm wondering what file is it trying to load, and how it got that path.
Also, at minimum I think the aot-compiler is fail at that point.

steveisok · 2023-07-12T14:53:45Z

Could you please point to the code for this? I'm wondering what file is it trying to load, and how it got that path. Also, at minimum I think the aot-compiler is fail at that point.

runtime/src/mono/mono/mini/aot-compiler.c

Lines 15358 to 15367 in 693eef6

 } else { 

 acfg->tmpbasename = g_build_filename (acfg->aot_opts.temp_path, "temp", (const char*)NULL); 

 acfg->tmpfname = g_strdup_printf ("%s.s", acfg->tmpbasename); 

 acfg->fp = fopen (acfg->tmpfname, "w+"); 

 } 

 } 

 if (acfg->fp == 0 && !acfg->aot_opts.llvm_only) { 

 aot_printerrf (acfg, "Unable to open file '%s': %s\n", acfg->tmpfname, strerror (errno)); 

 return 1; 

 }

radical · 2023-07-12T14:56:47Z

So aot_opts.temp_path then? This is what gets passed to the compiler:
temp-path=obj\Release\android-arm\aot\armeabi-v7a\_Microsoft.Android.Resource.Designer

steveisok · 2023-07-12T15:06:01Z

I don't know why yet, but the unicode name ends up in the path. For example:

Unable to open file 'obj\Release\net8.0-android\android-x64\aot\x86_64\BuildAot And├£ml├ñ├╝ts\temp.s': No such file or directory

radical · 2023-07-12T15:09:06Z

runtime/src/mono/mono/mini/aot-compiler.c

Lines 8776 to 8777 in 693eef6

 } else if (str_begins_with (arg, "temp-path=")) { 

 opts->temp_path = clean_path (g_strdup (arg + strlen ("temp-path=")));

runtime/src/mono/mono/mini/aot-compiler.c

Lines 8550 to 8563 in 693eef6

 static gchar * 

 clean_path (gchar * path) 

 { 

 if (!path) 

 return NULL; 

 if (g_str_has_suffix (path, G_DIR_SEPARATOR_S)) 

 return path; 

 gchar *clean = g_strconcat (path, G_DIR_SEPARATOR_S, (const char*)NULL); 

 g_free (path); 

 return clean; 

 }

radical · 2023-07-12T15:10:40Z

oh, sorry, I thought clean_path was making the path absolute.

radical · 2023-07-12T15:40:31Z

I don't know why yet, but the unicode name ends up in the path. For example:

Unable to open file 'obj\Release\net8.0-android\android-x64\aot\x86_64\BuildAot And├£ml├ñ├╝ts\temp.s': No such file or directory

Do you have the binlog handy? I want to check obj\Release\net8.0-android\android-x64\aot\x86_64\BuildAot And├£ml├ñ├╝ts\ is being passed as temp-path, if so then something we could change in the task. Else, it is getting appended in aot-compiler somehow.

steveisok · 2023-07-12T15:59:24Z

@radical here you go. @jonathanpeppers, I noticed the binlog you provided and the reproduction steps differ in that the latter will have a unicode named assembly and working directory versus just a unicode named working directory. I still think the end result is the same though.

llvm-and-nollvm-binlogs.zip

radical · 2023-07-12T16:12:15Z

So the MonoAOTCompiler is passing temp-path=obj\Release\net8.0-android\android-x86\aot\x86\BuildAot AndÜmläüts for compiling obj\Release\net8.0-android\android-x86\linked\BuildAot AndÜmläüts.dll.

We could sanitize that temp path before passing it.. but I don't know what else the aot-compiler will get stuck on! I can't reproduce myself, so somebody else will have to try this.

jonathanpeppers · 2023-07-12T16:20:47Z

Our test might be setting something like AssemblyName=UnnamedProject, and using dotnet new wouldn't do that.

I would focus on just the project directory being the issue -- as customers a lot of the time will have something like C:\Users\MyNameäü\Desktop\Code in project paths.

radical · 2023-07-12T16:23:36Z

Our test might be setting something like AssemblyName=UnnamedProject, and using dotnet new wouldn't do that.

I would focus on just the project directory being the issue -- as customers a lot of the time will have something like C:\Users\MyNameäü\Desktop\Code in project paths.

How does that case fail? project directory C:\Users\MyNameäü\Desktop\Code, and no assemblies with unicode chars.
Do you have a binlog for that?

jonathanpeppers · 2023-07-12T16:28:01Z

It is the aot.zip above, I got it from our CI.

Project is C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot AndÜmläüts_armeabiv7ax86_True_True\UnnamedProject.csproj
<AssemblyName>UnnamedProject</AssemblyName>
Assembly is obj\Release\android-arm\linked\UnnamedProject.dll
WorkingDirectory = C:\a\_work\1\a\TestRelease\07-10_18.28.32\temp\BuildAot AndÜmläüts_armeabiv7ax86_True_True

Changes: dotnet/installer@28d4a6b...ca467d6 Changes: dotnet/runtime@3e87528...ae99bb2 Changes: dotnet/emsdk@60a73f9...afbff08 Changes: dotnet/cecil@3e46711...1a6a83a Updates: * Microsoft.Dotnet.Sdk.Internal: from 8.0.100-preview.7.23330.16 to 8.0.100-preview.7.23364.32 * Microsoft.NETCore.App.Ref: from 8.0.0-preview.7.23329.8 to 8.0.0-preview.7.23364.3 * Microsoft.NET.Workload.Emscripten.Current.Manifest-8.0.100.Transport: from 8.0.0-preview.7.23326.1 to 8.0.0-preview.7.23361.2 * Microsoft.NET.ILLink.Tasks: from 8.0.0-preview.7.23329.8 to 8.0.0-preview.7.23364.3 * Microsoft.DotNet.Cecil: from 0.11.4-alpha.23327.1 to 0.11.4-alpha.23360.2 ~~ Other changes ~~ * Update `.apkdesc` files for app size changes * Ignore several tests around an LLVM build issue on Windows: dotnet/runtime#88625 * Workarounds for behavior changes around `$DOTNET_ROOT`: This appears to be the way to get `<Csc/>` to pick the right `dotnet`: https://github.com/dotnet/roslyn/pull/68918/files#diff-ec99cd9453fe657dd1ea08e93e94b0bb51c51484b8ad25882d312564e5fe8e26R61 * [tests] disable failing Java.Interop tests under LLVM dotnet/runtime#89190 * Bump to xamarin/java.interop/main@62efc0980 Changes: dotnet/java-interop@151b03e...62efc09 Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>

dotnet-issue-labeler bot added the area-Codegen-AOT-mono label Jul 10, 2023

ghost added the untriaged New issue has not been triaged by the area owner label Jul 10, 2023

jonathanpeppers added a commit to dotnet/android that referenced this issue Jul 10, 2023

[tests] ignore BuildAotApplicationAndÜmläüts on Windows

5ef90a8

Context: dotnet/runtime#88625

jonathanpeppers added a commit to dotnet/android that referenced this issue Jul 11, 2023

[tests] ignore BuildAotApplicationWithNdkAndBundleAndÜmläüts

bce52f1

Context: dotnet/runtime#88625

radical changed the title ~~Android: AOT compiler fails in directory named BuildAot AndÜmläüts~~ Android: AOT compiler fails on Windows in directory named BuildAot AndÜmläüts Jul 11, 2023

jonathanpeppers added a commit to dotnet/android that referenced this issue Jul 11, 2023

[tests] ignore NoSymbolsArgShouldReduceAppSize

c235d60

Context: dotnet/runtime#88625

steveisok added this to the Future milestone Jul 12, 2023

ghost removed the untriaged New issue has not been triaged by the area owner label Jul 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Android: AOT compiler fails on Windows in directory named `BuildAot AndÜmläüts` #88625

Android: AOT compiler fails on Windows in directory named `BuildAot AndÜmläüts` #88625

jonathanpeppers commented Jul 10, 2023 •

edited

Loading

lewing commented Jul 11, 2023

radical commented Jul 11, 2023

steveisok commented Jul 11, 2023

jonathanpeppers commented Jul 11, 2023

directhex commented Jul 11, 2023

jonathanpeppers commented Jul 11, 2023

directhex commented Jul 11, 2023

steveisok commented Jul 12, 2023

radical commented Jul 12, 2023 •

edited

Loading

jonathanpeppers commented Jul 12, 2023

radical commented Jul 12, 2023

radical commented Jul 12, 2023

steveisok commented Jul 12, 2023

radical commented Jul 12, 2023

steveisok commented Jul 12, 2023

radical commented Jul 12, 2023

radical commented Jul 12, 2023

radical commented Jul 12, 2023

steveisok commented Jul 12, 2023

radical commented Jul 12, 2023

jonathanpeppers commented Jul 12, 2023

radical commented Jul 12, 2023

jonathanpeppers commented Jul 12, 2023

Android: AOT compiler fails on Windows in directory named BuildAot AndÜmläüts #88625

Android: AOT compiler fails on Windows in directory named BuildAot AndÜmläüts #88625

Comments

jonathanpeppers commented Jul 10, 2023 • edited Loading

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

lewing commented Jul 11, 2023

radical commented Jul 11, 2023

steveisok commented Jul 11, 2023

jonathanpeppers commented Jul 11, 2023

directhex commented Jul 11, 2023

jonathanpeppers commented Jul 11, 2023

directhex commented Jul 11, 2023

steveisok commented Jul 12, 2023

radical commented Jul 12, 2023 • edited Loading

jonathanpeppers commented Jul 12, 2023

radical commented Jul 12, 2023

radical commented Jul 12, 2023

steveisok commented Jul 12, 2023

radical commented Jul 12, 2023

steveisok commented Jul 12, 2023

radical commented Jul 12, 2023

radical commented Jul 12, 2023

radical commented Jul 12, 2023

steveisok commented Jul 12, 2023

radical commented Jul 12, 2023

jonathanpeppers commented Jul 12, 2023

radical commented Jul 12, 2023

jonathanpeppers commented Jul 12, 2023

Android: AOT compiler fails on Windows in directory named `BuildAot AndÜmläüts` #88625

Android: AOT compiler fails on Windows in directory named `BuildAot AndÜmläüts` #88625

jonathanpeppers commented Jul 10, 2023 •

edited

Loading

radical commented Jul 12, 2023 •

edited

Loading