Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build issue typemaps.x86_64.ll: error: Could not open input file: no such file or directory #8967

Closed
hansmbakker opened this issue May 20, 2024 · 26 comments · Fixed by #9174
Closed
Assignees
Labels
Area: App+Library Build Issues when building Library projects or Application projects. need-attention A xamarin-android contributor needs to review

Comments

@hansmbakker
Copy link

hansmbakker commented May 20, 2024

Android application type

.NET Android (net7.0-android, net8.0-android, etc.)

Affected platform version

VS 2022 17.10 Preview 7

Description

When trying to debug my app, I sometimes get an issue like typemaps.x86_64.ll: error: Could not open input file: no such file or directory.
Another error I got was typemaps.armeabi-v7a.ll: error: Could not open input file: no such file or directory

This happens both in Visual Studio and on the command line using MSBuild.

The build error code is XA3006.

Steps to Reproduce

Not sure, but it is related to debugging on different devices having different processor architectures.

It happens e.g. when I try to deploy to my x86_64 emulator on my development machine after debugging via USB on a physical phone (which has an ARM processor).

Did you find any workaround?

No

Relevant log output

Please see the relevant tail of the logs below. The part before this (building the app) is common to all target devices and is working fine.

Skipping target "_LinkAssembliesNoShrink" because all output files are up-to-date with respect to the input files.
Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
Skipping target "_ManifestMerger" because all output files are up-to-date with respect to the input files.
Skipping target "_ConvertCustomView" because all output files are up-to-date with respect to the input files.
Skipping target "_AddStaticResources" because all output files are up-to-date with respect to the input files.
Skipping target "_GenerateEmptyAndroidRemapNativeCode" because all output files are up-to-date with respect to the input files.
Touching "obj\Debug
et8.0-android\stamp\_GeneratePackageManagerJava.stamp".
Skipping target "_GenerateAndroidAssetsDir" because all output files are up-to-date with respect to the input files.
Skipping target "_PrepareCreateBaseApk" because all output files are up-to-date with respect to the input files.
Skipping target "_CreateBaseApk" because all output files are up-to-date with respect to the input files.
Skipping target "_CompileJava" because all output files are up-to-date with respect to the input files.
[llc.EXE stderr] C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.95\tools\binutils\bin\llc.exe: error: C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.95\tools\binutils\bin\llc.exe: typemaps.x86_64.ll: error: Could not open input file: no such file or directory
XA3006: Could not compile native assembly file: typemaps.x86_64.ll

stderr | C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.95\tools\binutils\bin\llc.exe: error: C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.95\tools\binutils\bin\llc.exe: typemaps.x86_64.ll: error: Could not open input file: no such file or directory


Done building project "Anonymized.App.csproj" -- FAILED.
Build FAILED.
2>Deployment failed to Small_Phone_API_34.
2>Deploy failed
========== Build: 1 succeeded, 0 failed, 5 up-to-date, 0 skipped ==========
========== Build completed at 13:32 and took 11,768 seconds ==========
========== Deploy: 0 succeeded, 1 failed, 0 skipped ==========
========== Deploy completed at 13:32 and took 11,768 seconds ==========
@hansmbakker hansmbakker added Area: App+Library Build Issues when building Library projects or Application projects. needs-triage Issues that need to be assigned. labels May 20, 2024
@hansmbakker
Copy link
Author

hansmbakker commented May 20, 2024

@grendello let's discuss the build issue here.

I recorded a binlog and can share it confidentially (it contains client information).

@hansmbakker
Copy link
Author

The logs are shared with @grendello's Microsoft account via OneDrive

@grendello
Copy link
Contributor

@hansmbakker can you modify the C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.95\tools\Xamarin.Android.Common.targets file and apply the following diff:

diff --git a/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets b/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets
index 5d81e3f75..5c35118e6 100644
--- a/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets
+++ b/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Common.targets
@@ -1443,7 +1443,7 @@ because xbuild doesn't support framework reference assemblies.
 <Target Name="_GenerateJavaStubs"
     DependsOnTargets="$(_GenerateJavaStubsDependsOnTargets);$(BeforeGenerateAndroidManifest)"
     Inputs="@(_AndroidMSBuildAllProjects);@(_ResolvedUserMonoAndroidAssemblies);$(_AndroidManifestAbs);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"
-    Outputs="$(_AndroidStampDirectory)_GenerateJavaStubs.stamp">
+    Outputs="$(_AndroidStampDirectory)_GenerateJavaStubs.stamp;@(_TypeMapAssemblySource)">
 
   <PropertyGroup>
     <_ManifestOutput Condition=" '$(AndroidManifestMerger)' == 'legacy' ">$(IntermediateOutputPath)android\AndroidManifest.xml</_ManifestOutput>

And see if that fixes the issue?

@grendello grendello added need-info Issues that need more information from the author. and removed needs-triage Issues that need to be assigned. labels May 23, 2024
@hansmbakker
Copy link
Author

I'm off until Monday and will reply when I have a chance to do it

@microsoft-github-policy-service microsoft-github-policy-service bot added need-attention A xamarin-android contributor needs to review and removed need-info Issues that need more information from the author. labels May 23, 2024
@hansmbakker
Copy link
Author

hansmbakker commented May 24, 2024

@grendello I tried your change and it seems to help. I had to restart Visual Studio before it picked it up.

I updated the reproduction steps - it had something to do with switching debugging between devices that have different architectures.

Please note that xamarin/Xamarin.Forms#15719 and dotnet/maui#14531 (comment) seem to be related.

@dezryth
Copy link

dezryth commented May 25, 2024

Just popping in to say I'm getting this error too, but don't think I have anything helpful to add. Let me know if there is something I could provide to assist though.

@dezryth
Copy link

dezryth commented May 26, 2024

@grendello I tried the change as well and it worked for me. I did not have to restart Visual Studio.

@crui3er
Copy link

crui3er commented May 31, 2024

Thanks @grendello. It helped to me too.

grendello added a commit that referenced this issue Jun 4, 2024
Fixes: #8967

For some reason, sometimes the `typemap*.ll` files are sometimes
removed from the `obj/` directory which leads to build errors
similar to:

    typemaps.x86_64.ll: error: Could not open input file: no such file or directory

Files are generated by the `GenerateJavaStubs` tasks, which is invoked
by the `_GenerateJavaStubs` target.  However, the target doesn't specify
the `*.ll` files in its `Outputs` parameter and, therefore, whenever the
files are removed but the `_GenerateJavaStubs.stamp` file is newer than
the items/files specified in the target's `Inputs` parameter, the native
assembly files aren't regenerated leading to the above error.

To fix this, we need to add the `typemap*.ll` files to the target's
`Outputs` set, thus forcing their regeneration should they be no longer
where they are expected to be.
grendello added a commit that referenced this issue Jun 5, 2024
Fixes: #8967

For some reason, sometimes the `typemap*.ll` files are sometimes
removed from the `obj/` directory which leads to build errors
similar to:

    typemaps.x86_64.ll: error: Could not open input file: no such file or directory

Files are generated by the `GenerateJavaStubs` tasks, which is invoked
by the `_GenerateJavaStubs` target.  However, the target doesn't specify
the `*.ll` files in its `Outputs` parameter and, therefore, whenever the
files are removed but the `_GenerateJavaStubs.stamp` file is newer than
the items/files specified in the target's `Inputs` parameter, the native
assembly files aren't regenerated leading to the above error.

To fix this, we need to add the `typemap*.ll` files to the target's
`Outputs` set, thus forcing their regeneration should they be no longer
where they are expected to be.
@ur3an0
Copy link

ur3an0 commented Jun 9, 2024

gracias @grendello, también me funciono a mi esa solución

@sabrigo
Copy link

sabrigo commented Jun 13, 2024

thank you... this solved the issue

@bodeg
Copy link

bodeg commented Jun 15, 2024

@grendello
you are a lifesaver. Thanks!

@wdcossey
Copy link

Just to confirm, it's [still] broken in Microsoft.Android.Sdk.Windows 34.0.113

The fix by @grendello still applies

C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\Xamarin.Android.Common.targets

@rhires
Copy link

rhires commented Jun 24, 2024

The fix shown in the diff worked for me.

@dellis1972
Copy link
Contributor

For all of those hitting this issue (and for whom the fix worked) it would be really nice to figure out what is deleting these files in the first place.

If you can a chance can you install the "Project System Tools" extension [1] and turn on the build logging and set it to capture All logs. Then try to repo the issue (without the fix). We would love to figure out what is causing the problem as the code we have should be working.

[1]https://github.com/dotnet/project-system-tools

@kidmar
Copy link

kidmar commented Jun 26, 2024

This happened to me after deleting bin and obj folders.
Restarting Visual Studio without changing anything solved the problem.

@Orgbrat
Copy link

Orgbrat commented Jun 28, 2024

This happened to me just switching between building Windows or iOS to an Android build and deploy to emulator.
Restarting Visual Studio without changing anything solved the problem.

@dellis1972
Copy link
Contributor

@Orgbrat if you can reliably reproduce it would be really handy to have all the binlog files for that process (see #8967 (comment)). Something in that process is causing files to be deleted.

@Orgbrat
Copy link

Orgbrat commented Jun 28, 2024

@dellis1972 I have installed project-system-tools as requested. I have to be away until after the holiday so I will try to reproduce and send you the logs with my details as soon as I am able. Although it does work now, I am curious if building one of the other platforms has anything to do with this issue.

@hansmbakker
Copy link
Author

For me it seemed to happen when switching not between platforms per se (I did not try building for windows or iOS) but switching between targets devices.

I was switching between different android phones - two physical and one emulator. There were different processor architectures.

grendello added a commit that referenced this issue Jul 8, 2024
Fixes: #8967

For some reason, sometimes the `typemap*.ll` files are sometimes
removed from the `obj/` directory which leads to build errors
similar to:

    typemaps.x86_64.ll: error: Could not open input file: no such file or directory

Files are generated by the `GenerateJavaStubs` tasks, which is invoked
by the `_GenerateJavaStubs` target.  However, the target doesn't specify
the `*.ll` files in its `Outputs` parameter and, therefore, whenever the
files are removed but the `_GenerateJavaStubs.stamp` file is newer than
the items/files specified in the target's `Inputs` parameter, the native
assembly files aren't regenerated leading to the above error.

To fix this, we need to add the `typemap*.ll` files to the target's
`Outputs` set, thus forcing their regeneration should they be no longer
where they are expected to be.
@pjt33
Copy link

pjt33 commented Jul 10, 2024

For all of those hitting this issue (and for whom the fix worked) it would be really nice to figure out what is deleting these files in the first place.

I think it's not copying them in the first place. If I rebuild the basic template project then the only .ll files copied into obj/Debug/net8.0-android/android/ are .arm64-v8a.ll files. When I then try to debug on an x86_x64 emulator, it copies the .x86_x64.ll versions of all except typemaps. However, the fix doesn't work for me so maybe there are multiple issues at play here.

Edit: a workaround for me for development is to add <AndroidCreatePackagePerAbi>True</AndroidCreatePackagePerAbi> to the csproj, but I suspect that I won't want that when I come to compile a bundle for release.

@dellis1972
Copy link
Contributor

The reason only the x86.64.ll file is used when debugging is we detect the device abi which you are debugging agsinst and only build for that abi. This is different from release packages where we build for all the selected abis.

@ziomek64
Copy link

For all of those hitting this issue (and for whom the fix worked) it would be really nice to figure out what is deleting these files in the first place.

I think it's not copying them in the first place. If I rebuild the basic template project then the only .ll files copied into obj/Debug/net8.0-android/android/ are .arm64-v8a.ll files. When I then try to debug on an x86_x64 emulator, it copies the .x86_x64.ll versions of all except typemaps. However, the fix doesn't work for me so maybe there are multiple issues at play here.

Edit: a workaround for me for development is to add <AndroidCreatePackagePerAbi>True</AndroidCreatePackagePerAbi> to the csproj, but I suspect that I won't want that when I come to compile a bundle for release.

This workaround worked for me

grendello added a commit that referenced this issue Jul 26, 2024
Fixes: #8967

For some reason, sometimes the `typemap*.ll` files are sometimes
removed from the `obj/` directory which leads to build errors
similar to:

    typemaps.x86_64.ll: error: Could not open input file: no such file or directory

Files are generated by the `GenerateJavaStubs` tasks, which is invoked
by the `_GenerateJavaStubs` target.  However, the target doesn't specify
the `*.ll` files in its `Outputs` parameter and, therefore, whenever the
files are removed but the `_GenerateJavaStubs.stamp` file is newer than
the items/files specified in the target's `Inputs` parameter, the native
assembly files aren't regenerated leading to the above error.

To fix this, we need to add the `typemap*.ll` files to the target's
`Outputs` set, thus forcing their regeneration should they be no longer
where they are expected to be.
@jonathanpeppers
Copy link
Member

Does someone know the repro steps to cause this problem?

I thought that this would cause it inside Visual Studio:

  • F5 to an arm64 device
  • F5 to an x86_64 emulator

But that seems to work fine.

@PredatH0r
Copy link

For me it happened just now the first time after switching from the Android emulator (on x64) to a Samsung device (arm64) and then back again to the emulator.
The fix now allows me to run my app in the emulator again.

jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this issue Aug 6, 2024
Fixes: dotnet#8967
Context: dotnet#9001

You can cause a build error in .NET 8 by doing:

* Start an x86 or x86_64 emulator
* `dotnet build -t:Run`
* Close the emulator, attach an arm or arm64 device
* `dotnet build -t:Run`

Emits the error:

    C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\Xamarin.Android.Common.targets(2063,3): error XA3006: Could not compile native assembly file: typemaps.x86_64.ll
    stderr | C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: error: C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: typemaps.x86_64.ll: error: Could not open input file: no such file or directory

This works fine in Visual Studio, but not at the command-line.

The underlying problem is due to `_GeneratePackageManagerJava` target
running, while the `_GenerateJavaStubs` target *did not*:

    Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
    ...
    Input file "obj\Debug\net8.0-android\resolvedassemblies.hash" is newer than output file "obj\Debug\net8.0-android\stamp\_GeneratePackageManagerJava.stamp".

These two targets should almost always run together, so we should
ensure that they do.

This problem also doesn't happen .NET 9, as both targets are
invalidated for a different reason:

    Target Name=_LinkAssembliesNoShrink Project=UnnamedProject.csproj
    Building target "_LinkAssembliesNoShrink" completely.
    Output file "obj\Debug\android\assets\x86_64\UnnamedProject.dll" does not exist.

Since, we build per-RID in .NET 9, `obj\Debug\android\assets\x86_64\UnnamedProject.dll`
has an `x86_64` in the path, which is not present in the .NET 8 build.

Reviewing the two sets of `Inputs`:

    <Target Name="_GenerateJavaStubs"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);@(_ResolvedUserMonoAndroidAssemblies);$(_AndroidManifestAbs);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"
    ...
    <Target Name="_GeneratePackageManagerJava"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);$(_ResolvedUserAssembliesHashFile);$(MSBuildProjectFile);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"

Let's align them more closely by:

* Track assembly changes in the exact same way:

    $(_ResolvedUserAssembliesHashFile);@(_ResolvedUserMonoAndroidAssemblies);

This way if an assembly is added, removed, or timestamp changed: both
targets are in sync.

* Don't track `$(MSBuildProjectFile)`, this should already be in
  `@(_AndroidMSBuildAllProjects)`.

With this change in place manually, I'm not able to reproduce the
problem any longer.

PR dotnet#9001 may also have fixed this issue, but it could cause targets to
run on every incremental build -- a performance issue.
jonathanpeppers added a commit that referenced this issue Aug 6, 2024
…ets (#9174)

Fixes: #8967
Context: #9001

You can cause a build error in .NET 8 by doing:

* Start an x86 or x86_64 emulator
* `dotnet build -t:Run`
* Close the emulator, attach an arm or arm64 device
* `dotnet build -t:Run`

Emits the error:

    Xamarin.Android.Common.targets(2063,3): error XA3006: Could not compile native assembly file: typemaps.x86_64.ll
    stderr | C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: error: C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: typemaps.x86_64.ll: error: Could not open input file: no such file or directory

This works fine in Visual Studio, but not at the command-line.

The underlying problem is due to `_GeneratePackageManagerJava` target
running, while the `_GenerateJavaStubs` target *did not*:

    Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
    ...
    Input file "obj\Debug\net8.0-android\resolvedassemblies.hash" is newer than output file "obj\Debug\net8.0-android\stamp\_GeneratePackageManagerJava.stamp".

These two targets should almost always run together, so we should
ensure that they do.

This problem also doesn't happen .NET 9, as both targets are
invalidated for a different reason:

    Target Name=_LinkAssembliesNoShrink Project=UnnamedProject.csproj
    Building target "_LinkAssembliesNoShrink" completely.
    Output file "obj\Debug\android\assets\x86_64\UnnamedProject.dll" does not exist.

Since, we build per-RID in .NET 9, `obj\Debug\android\assets\x86_64\UnnamedProject.dll`
has an `x86_64` in the path, which is not present in the .NET 8 build.

Reviewing the two sets of `Inputs`:

    <Target Name="_GenerateJavaStubs"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);@(_ResolvedUserMonoAndroidAssemblies);$(_AndroidManifestAbs);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"
    ...
    <Target Name="_GeneratePackageManagerJava"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);$(_ResolvedUserAssembliesHashFile);$(MSBuildProjectFile);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"

Let's align them more closely by:

* Track assembly changes in the exact same way:

    $(_ResolvedUserAssembliesHashFile);@(_ResolvedUserMonoAndroidAssemblies);

This way if an assembly is added, removed, or timestamp changed: both
targets are in sync.

* Don't track `$(MSBuildProjectFile)`, this should already be in
  `@(_AndroidMSBuildAllProjects)`.

With this change in place manually, I'm not able to reproduce the
problem any longer.

PR #9001 may also have fixed this issue, but it could cause targets to
run on every incremental build -- a performance issue.
jonathanpeppers added a commit that referenced this issue Aug 6, 2024
…ets (#9174)

Fixes: #8967
Context: #9001

You can cause a build error in .NET 8 by doing:

* Start an x86 or x86_64 emulator
* `dotnet build -t:Run`
* Close the emulator, attach an arm or arm64 device
* `dotnet build -t:Run`

Emits the error:

    Xamarin.Android.Common.targets(2063,3): error XA3006: Could not compile native assembly file: typemaps.x86_64.ll
    stderr | C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: error: C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools\binutils\bin\llc.exe: typemaps.x86_64.ll: error: Could not open input file: no such file or directory

This works fine in Visual Studio, but not at the command-line.

The underlying problem is due to `_GeneratePackageManagerJava` target
running, while the `_GenerateJavaStubs` target *did not*:

    Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
    ...
    Input file "obj\Debug\net8.0-android\resolvedassemblies.hash" is newer than output file "obj\Debug\net8.0-android\stamp\_GeneratePackageManagerJava.stamp".

These two targets should almost always run together, so we should
ensure that they do.

This problem also doesn't happen .NET 9, as both targets are
invalidated for a different reason:

    Target Name=_LinkAssembliesNoShrink Project=UnnamedProject.csproj
    Building target "_LinkAssembliesNoShrink" completely.
    Output file "obj\Debug\android\assets\x86_64\UnnamedProject.dll" does not exist.

Since, we build per-RID in .NET 9, `obj\Debug\android\assets\x86_64\UnnamedProject.dll`
has an `x86_64` in the path, which is not present in the .NET 8 build.

Reviewing the two sets of `Inputs`:

    <Target Name="_GenerateJavaStubs"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);@(_ResolvedUserMonoAndroidAssemblies);$(_AndroidManifestAbs);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"
    ...
    <Target Name="_GeneratePackageManagerJava"
        ...
        Inputs="@(_AndroidMSBuildAllProjects);$(_ResolvedUserAssembliesHashFile);$(MSBuildProjectFile);$(_AndroidBuildPropertiesCache);@(AndroidEnvironment);@(LibraryEnvironments)"

Let's align them more closely by:

* Track assembly changes in the exact same way:

    $(_ResolvedUserAssembliesHashFile);@(_ResolvedUserMonoAndroidAssemblies);

This way if an assembly is added, removed, or timestamp changed: both
targets are in sync.

* Don't track `$(MSBuildProjectFile)`, this should already be in
  `@(_AndroidMSBuildAllProjects)`.

With this change in place manually, I'm not able to reproduce the
problem any longer.

PR #9001 may also have fixed this issue, but it could cause targets to
run on every incremental build -- a performance issue.
@zijianhuang
Copy link

@grendello has given a good answer about modifying file in tools\Xamarin.Android.Common.targets

The problem has come back to me upon VS 17.11.2, now go to the targets file in
C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\34.0.113\tools

Not sure if MS should fix this?

@grendello
Copy link
Contributor

@zijianhuang it's fixed in #9174

@github-actions github-actions bot locked and limited conversation to collaborators Oct 4, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Area: App+Library Build Issues when building Library projects or Application projects. need-attention A xamarin-android contributor needs to review
Projects
None yet