native aot regression in .net 8.0

[ivanjx]

Description

i am using this library to generate strings with templates. it can work fine using JsonDocument as the model under native aot with .net 7. however, somehow when i upgraded my dotnet version to 8.0 it breaks with System.ArgumentException: The handle is invalid.. i know that this library does not support native aot directly but im sure this is a regression bug in .net

Reproduction Steps

nativeaot.zip

• first, compile with dotnet publish -c Release and run the generated binary.
• change the TargetFramework to net7.0 to make it work.

Expected behavior

works on net 8.0

Actual behavior

crashed under native aot

Regression?

net 7.0

Known Workarounds

No response

Configuration

No response

Other information

No response

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

i am using this library to generate strings with templates. it can work fine using JsonDocument as the model under native aot with .net 7. however, somehow when i upgraded my dotnet version to 8.0 it breaks with System.ArgumentException: The handle is invalid.. i know that this library does not support native aot directly but im sure this is a regression bug in .net

Reproduction Steps

nativeaot.zip

• first, compile with dotnet publish -c Release and run the generated binary.
• change the TargetFramework to net7.0 to make it work.

Expected behavior

works on net 8.0

Actual behavior

crashed under native aot

Regression?

net 7.0

Known Workarounds

No response

Configuration

No response

Other information

No response

Author:	ivanjx
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[ivanjx]

it seems that this also works on net 8.0 preview 4. from preview 5 all the way to preview 7 it broke.

# Build.
FROM mcr.microsoft.com/dotnet/sdk:8.0.100-preview.4-bookworm-slim-amd64 as build
RUN dpkg --add-architecture arm64 && \
    apt-get update && \
    apt-get install -y \
    clang zlib1g-dev zlib1g-dev:arm64 \
    binutils-aarch64-linux-gnu gcc-aarch64-linux-gnu
COPY ./ nativeaot
WORKDIR nativeaot
ARG TARGETPLATFORM
RUN ARCH=$(echo $TARGETPLATFORM | cut -d '/' -f 2) && \
    PREFIX=$(echo $ARCH | awk '{if ($0 == "arm64") print "aarch64-linux-gnu-";}') && \
    dotnet publish \
    -c Release \
    -r linux-$ARCH \
    -p:ObjCopyName=${PREFIX}objcopy \
    -o /output && \
    rm /output/*.dbg

# Runtime.
FROM mcr.microsoft.com/dotnet/runtime-deps:8.0.0-preview.4-bookworm-slim as runtime
COPY --from=build /output /app
ENTRYPOINT [ "/app/nativeaot" ]

[agocke]

The problem seems to be that the app is using Linq Expressions in a trim-incompatible way.

The exception is:

$ ./bin/Release/net8.0/win-x64/publish/nativeaot.exe
Unhandled Exception: HandlebarsDotNet.HandlebarsCompilerException: An unhandled exception occurred while trying to compile the template
 ---> HandlebarsDotNet.HandlebarsCompilerException: An unhandled exception occurred while trying to compile the template
 ---> HandlebarsDotNet.HandlebarsCompilerException: An unhandled exception occurred while trying to compile the template
 ---> HandlebarsDotNet.HandlebarsCompilerException: An unhandled exception occurred while trying to compile the template
 ---> System.ArgumentException: The handle is invalid.
   at System.Reflection.Runtime.General.ReflectionCoreCallbacksImplementation.GetMethodFromHandle(RuntimeMethodHandle, RuntimeTypeHandle) + 0xdd
   at HandlebarsDotNet.Compiler.PathBinder.VisitPathExpression(PathExpression) + 0x97f
   at HandlebarsDotNet.Compiler.PathBinder.VisitStatementExpression(StatementExpression) + 0x5d
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor, BlockExpression) + 0x44
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression) + 0x18
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor, BlockExpression) + 0x44
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression) + 0x18
   at HandlebarsDotNet.Compiler.FunctionBuilder.Reduce(Expression, CompilationContext, IReadOnlyList`1&) + 0x1b6
   at HandlebarsDotNet.Compiler.FunctionBuilder.CreateExpression(IEnumerable`1, CompilationContext, IReadOnlyList`1&) + 0xf4
   --- End of inner exception stack trace ---
   at HandlebarsDotNet.Compiler.FunctionBuilder.CreateExpression(IEnumerable`1, CompilationContext, IReadOnlyList`1&) + 0x1ea
   at HandlebarsDotNet.Compiler.FunctionBuilder.Compile(IEnumerable`1, CompilationContext, IReadOnlyList`1&) + 0x17
   --- End of inner exception stack trace ---
   at HandlebarsDotNet.Compiler.FunctionBuilder.Compile(IEnumerable`1, CompilationContext, IReadOnlyList`1&) + 0xe1
   at HandlebarsDotNet.Compiler.IteratorBinder.VisitIteratorExpression(IteratorExpression) + 0xb7
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitBlockExpressions(ExpressionVisitor, BlockExpression) + 0x44
   at System.Linq.Expressions.ExpressionVisitor.VisitBlock(BlockExpression) + 0x18
   at HandlebarsDotNet.Compiler.FunctionBuilder.Reduce(Expression, CompilationContext, IReadOnlyList`1&) + 0xfa
   at HandlebarsDotNet.Compiler.FunctionBuilder.CreateExpression(IEnumerable`1, CompilationContext, IReadOnlyList`1&) + 0xf4
   --- End of inner exception stack trace ---
   at HandlebarsDotNet.Compiler.FunctionBuilder.CreateExpression(IEnumerable`1, CompilationContext, IReadOnlyList`1&) + 0x1ea
   at HandlebarsDotNet.Compiler.FunctionBuilder.Compile(IEnumerable`1, CompilationContext, IReadOnlyList`1&) + 0x17
   --- End of inner exception stack trace ---
   at HandlebarsDotNet.Compiler.FunctionBuilder.Compile(IEnumerable`1, CompilationContext, IReadOnlyList`1&) + 0xe1
   at HandlebarsDotNet.Compiler.HandlebarsCompiler.Compile(ExtendedStringReader, CompilationContext) + 0x9a
   at HandlebarsDotNet.HandlebarsEnvironment.Compile(TextReader) + 0x12a
   at HandlebarsDotNet.HandlebarsEnvironment.Compile(String) + 0x5f
   at Program.<Main>$(String[] args) + 0xee
   at nativeaot!<BaseAddress>+0x331a69

This is likely to produce random failures -- either the program happens to visit all the necessary types that LINQ needs and everything works, or it doesn't keep the right thing, and the app crashes, like this.

My suspicion is that we have made some improvements in the LINQ area to trim even more unnecessary code away. The app may have been relying on that code to be present.

[vitek-karas]

The app produces AOT compatibility warnings when published for AOT. Our design is that we only guarantee compatibility if there are no warnings. Apps which produce warnings may work, but we're free to break them with upcoming releases.

Andy is right that the likely reason in this case is something around Linq.Expressions changed, but it's not a bug in the system - the design is such that if there are warnings the compiler is free to change the behavior if it's beneficial for other reasons.

[ivanjx]

thank you for the responses. is there any templating library that is aot friendly currently on dotnet?

[MichalStrehovsky]

System.ArgumentException: The handle is invalid. is likely a compiler bug. We made a RuntimeMethodHandle that doesn't resolve. I'll take a look.

[MichalStrehovsky]

I have a fix at #91382, thanks for the report! The warnings muddied the water a bit - what others said is true though - if there are warnings generated, the code is not AOT friendly and may not work in general. You need to thoroughly test with AOT enabled. It is not a great experience. With the fix, the demo app seems to work though.

[ivanjx]

thanks a lot @MichalStrehovsky. can i expect the fix to be in rc1? i will close this issue now.

[MichalStrehovsky]

Let's not close before it merges (github will autoclose on merge).

I don't think it will make RC1. Looks like RC2 at this point.

[ivanjx]

ah okay thats fine