Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NativeAOT-LLVM] Merge to August 6 #2647

Merged
merged 593 commits into from
Aug 9, 2024

Conversation

SingleAccretion
Copy link

@SingleAccretion SingleAccretion commented Aug 6, 2024

Notable changes:

  1. WASI SDK version update to 24 and the switch to WASI P2 (following upstream). Includes using wasmtime for testing.
  2. Emscripten update to 3.1.56 (following upstream).
  3. Adjust the warning check for old WASI SDKs to work around VERSION file missing from WASI SDK 24 WebAssembly/wasi-sdk#471.

akoeplinger and others added 30 commits July 24, 2024 12:54
Fixes an issue where we were still harcoding the Debug config for tasks even though that changed with dotnet/runtime#84931.
Also align the Makfile to make it more similar to the iOS-NativeAOT one.
UseCompilerGeneratedDocXmlFile already defaults to
true. Remove the two hits to avoid confusion.
* [RISC-V] Fix System.Net.Sockets.Tests on Qemu

Before this change there are 8 failures from System.Net.Sockets.Tests with following reports:

root@69fa7050f168:/runtime/artifacts/bin/System.Net.Sockets.Tests/Release/net9.0-unix# /runtime/artifacts/bin/testhost/net9.0-linux-Release-riscv64/dotnet  exec --runtimeconfig System.Net.Sockets.Tests.runtimeconfig.json --depsfile  System.Net.Sockets.Tests.deps.json  xunit.console.dll System.Net.Sockets.Tests.dll -xml testResults.xml -nologo -notrait category=nonnetcoreapptests -notrait category=nonlinuxtests -notrait category=failing -maxthreads 32
  Discovering: System.Net.Sockets.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Net.Sockets.Tests (found 1672 of 1820 test cases)
  Starting:    System.Net.Sockets.Tests (parallel test collections = on [32 threads], stop on fail = off)
    System.Net.Sockets.Tests.CreateSocket.Ctor_Raw_NotSupported_ExpectedError [SKIP]
      Condition(s) not met: "NotSupportsRawSockets"
    System.Net.Sockets.Tests.KeepAliveTest.Socket_Get_KeepAlive_Time_AsByteArray_OptionLengthZero_Failure [FAIL]
      System.Net.Sockets.SocketException : Bad address
      Stack Trace:
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3737,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3728,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketOptionErrorAndThrowException(SocketError error, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(2145,0): at System.Net.Sockets.Socket.GetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionLength)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/KeepAliveTest.cs(136,0): at System.Net.Sockets.Tests.KeepAliveTest.Socket_Get_KeepAlive_Time_AsByteArray_OptionLengthZero_Failure()

           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
    System.Net.Sockets.Tests.ArgumentValidation.Connect_ConnectTwice_NotSupported(invalidatingAction: 1) [FAIL]
      System.Net.Sockets.SocketException : Protocol not available
      Stack Trace:
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3737,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3728,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketOptionErrorAndThrowException(SocketError error, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3466,0): at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue, Boolean silent)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(1966,0): at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/ArgumentValidationTests.cs(809,0): at System.Net.Sockets.Tests.ArgumentValidation.Connect_ConnectTwice_NotSupported(Int32 invalidatingAction)
           at InvokeStub_ArgumentValidation.Connect_ConnectTwice_NotSupported(Object, Span`1)
           at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    System.Net.Sockets.Tests.SocketOptionNameTest.MulticastInterface_Set_AnyInterface_Succeeds [FAIL]
      System.Net.Sockets.SocketException : Unknown socket error
      Stack Trace:
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3737,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3728,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketOptionErrorAndThrowException(SocketError error, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3479,0): at System.Net.Sockets.Socket.SetMulticastOption(SocketOptionName optionName, MulticastOption MR)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(2021,0): at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Object optionValue)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/SocketOptionNameTest.cs(96,0): at System.Net.Sockets.Tests.SocketOptionNameTest.MulticastInterface_Set_Helper(Int32 interfaceIndex)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/SocketOptionNameTest.cs(71,0): at System.Net.Sockets.Tests.SocketOptionNameTest.MulticastInterface_Set_AnyInterface_Succeeds()
        --- End of stack trace from previous location ---
    System.Net.Sockets.Tests.KeepAliveTest.Socket_Get_KeepAlive_Time_AsByteArray_BufferNullOrTooSmall_Failure(buffer: null) [FAIL]
      System.Net.Sockets.SocketException : Bad address
      Stack Trace:
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3737,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3728,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketOptionErrorAndThrowException(SocketError error, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(2121,0): at System.Net.Sockets.Socket.GetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Byte[] optionValue)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/KeepAliveTest.cs(156,0): at System.Net.Sockets.Tests.KeepAliveTest.Socket_Get_KeepAlive_Time_AsByteArray_BufferNullOrTooSmall_Failure(Byte[] buffer)

           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(178,0): at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
    System.Net.Sockets.Tests.KeepAliveTest.Socket_Get_KeepAlive_Time_AsByteArray_BufferNullOrTooSmall_Failure(buffer: []) [FAIL]
      System.Net.Sockets.SocketException : Bad address
      Stack Trace:
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3737,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3728,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketOptionErrorAndThrowException(SocketError error, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(2121,0): at System.Net.Sockets.Socket.GetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Byte[] optionValue)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/KeepAliveTest.cs(156,0): at System.Net.Sockets.Tests.KeepAliveTest.Socket_Get_KeepAlive_Time_AsByteArray_BufferNullOrTooSmall_Failure(Byte[] buffer)
           at InvokeStub_KeepAliveTest.Socket_Get_KeepAlive_Time_AsByteArray_BufferNullOrTooSmall_Failure(Object, Span`1)
           at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    System.Net.Sockets.Tests.SocketOptionNameTest.MulticastOption_CreateSocketSetGetOption_GroupAndInterfaceIndex_SetSucceeds_GetThrows [FAIL]
      System.Net.Sockets.SocketException : Unknown socket error
      Stack Trace:
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3737,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3728,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketOptionErrorAndThrowException(SocketError error, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3479,0): at System.Net.Sockets.Socket.SetMulticastOption(SocketOptionName optionName, MulticastOption MR)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(2021,0): at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Object optionValue)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/SocketOptionNameTest.cs(61,0): at System.Net.Sockets.Tests.SocketOptionNameTest.MulticastOption_CreateSocketSetGetOption_GroupAndInterfaceIndex_SetSucceeds_GetThrows()

           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /runtime/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
    System.Net.Sockets.Tests.SocketOptionNameTest.MulticastInterface_Set_IPv6_AnyInterface_Succeeds [FAIL]
      System.Net.Sockets.SocketException : Protocol not available
      Stack Trace:
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3737,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3728,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketOptionErrorAndThrowException(SocketError error, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3466,0): at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue, Boolean silent)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(1966,0): at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/SocketOptionNameTest.cs(199,0): at System.Net.Sockets.Tests.SocketOptionNameTest.MulticastInterface_Set_IPv6_Helper(Int32 interfaceIndex)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/SocketOptionNameTest.cs(129,0): at System.Net.Sockets.Tests.SocketOptionNameTest.MulticastInterface_Set_IPv6_AnyInterface_Succeeds()
        --- End of stack trace from previous location ---
    System.Net.Sockets.Tests.ArgumentValidation.ConnectAsync_ConnectTwice_NotSupported(invalidatingAction: 1) [FAIL]
      System.Net.Sockets.SocketException : Protocol not available
      Stack Trace:
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3737,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3728,0): at System.Net.Sockets.Socket.UpdateStatusAfterSocketOptionErrorAndThrowException(SocketError error, String callerName)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(3466,0): at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue, Boolean silent)
        /runtime/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs(1966,0): at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue)
        /home/d.jurczak2/runtime/src/libraries/System.Net.Sockets/tests/FunctionalTests/ArgumentValidationTests.cs(842,0): at System.Net.Sockets.Tests.ArgumentValidation.ConnectAsync_ConnectTwice_NotSupported(Int32 invalidatingAction)
           at InvokeStub_ArgumentValidation.ConnectAsync_ConnectTwice_NotSupported(Object, Span`1)
           at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
  Finished:    System.Net.Sockets.Tests
=== TEST EXECUTION SUMMARY ===
   System.Net.Sockets.Tests  Total: 2874, Errors: 0, Failed: 8, Skipped: 1, Time: 139.280s

Those failures are caused by Qemu's buggy and/or implementation defined behavior like:
https://gitlab.com/qemu-project/qemu/-/issues/2410
https://gitlab.com/qemu-project/qemu/-/issues/2390
https://gitlab.com/qemu-project/qemu/-/issues/1837

In this patch we add couple of workarounds to make all System.Net.Sockets.Tests passing.

---------

Co-authored-by: Jan Kotas <jkotas@microsoft.com>
* Convert Perl scripts to C# in coreclr

* Delete obsolete IL script

* Add license header and regenerate

* Add a README

* Mention generator in ilasm readme and cleanups
…104461)

* Increase timeout for tracing/eventcounter/runtimecounters

This test expects that all runtime counters are already prepared when timeout has passed,
however, it might be untrue for debug/checked builds and for VisionFive2 (riscv64).

* Check if event are already prepared each second (60 seconds max)
* Update SDK to preview 6

* Update Shared.csproj

Fix `error NU1903: Package 'System.Text.Json' 8.0.0 has a known high severity vulnerability`

* Fix with existing version.

---------

Co-authored-by: Ilona Tomkowicz <32700855+ilonatommy@users.noreply.github.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Change DefaultMaximumErrorResponseLength to KB from Byte

* Handle overflow

* Review feedback
Fixes warning code when a generic type whose type parameters have DAM
annotations is used with MakeGenericType, over a type that doesn't
have matching annotations.

The code IL2070 used to mention the 'this' argument. Instead it should
have been IL2071 which mentions the generic argument as the cause of
the mismatch. Similar for MakeGenericMethod with IL2090 and IL2091.
Co-authored-by: Vladimir Sadov <vsadov@microsoft.com>
…n isolated context (#105337)

* Add support for isolated load context in LoadInMemoryAssemblyInContext by passing -1 as loadContext
* Have ijwhost check a runtime config parameter to determine if it should run in an isolated load context
* Added test for ijwhost isolated load context runtime config option
* Add tests

* Fix passing float and uint arguments in VM

* Change test lib name so it doesn't clash with managed DLL on Windows
We need to make sure the attribute for MacCatalyst comes _after_ the iOS one due to how MacCatalyst is a superset of iOS: https://learn.microsoft.com/en-us/dotnet/standard/analyzers/platform-compat-analyzer#platform-inclusion

This caused an error in aspnetcore in the latest dependency flow because the analyzer thought AesGcm is _only_ supported on MacCatalyst:
> error CA1416: (NETCORE_ENGINEERING_TELEMETRY=Build) This call site is reachable on all platforms. 'AesGcm.Decrypt(ReadOnlySpan<byte>, ReadOnlySpan<byte>, ReadOnlySpan<byte>, Span<byte>, ReadOnlySpan<byte>)' is only supported on: 'maccatalyst' 13.0 and later.
* Remove zlib from requirements instructions

* Remove zlib from native requirements installation script
…unt (#105289)

* Ensure that WaitForPendingFinalizers has seen the expected Full GC

* NativeAOT and some renames

* a testcase

* make the test not unsafe and make OuterLoop

* Use unsigned math when comparing collection ticks

* cast the diff to int when comparing gc ticks
…d license mentions (second attempt) (#105371)

* Remove zlib/
* Remove zlib-intel/
* Remove third party notice
* Remove patches
* Remove version txts
* Remove cgmanifest.json entries
* Remove installer third party notice
* Update docs
---------
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
* Edit test + trigger relink.

* Remove logging to speed up the test + decrease loop runs to prevent "Browser has been disconnected" error.

* Feedback - properties are not bool-only anymore.

* Fix: workload needed when heap size set.

---------

Co-authored-by: Larry Ewing <lewing@microsoft.com>
* Set xunit env var to not print output for passing tests

Fixes dotnet/runtime#103445

* Update xunit.console.targets

* Update xunit.console.targets
* Fix IDE0056 on corelib (indexing can be simplified)

* Fix IDE (null check can be simplified)

* Fix IDE0078 (use pattern matching)

* Fix IDE0019 (use pattern matching)

* Fix IDE0066 (use switch expression)

* Fix IDE0250 (struct can be made readonly)

* Fix nullability warning and address PR feedback

* Address PR feedback and revert a downlevel change

* Wrap any `?? throw new`s that go beyond 120 characters
On https://github.com/dotnet/runtime/blob/main/docs/project/dotnet-standards.md the Partition with Notes download links using HTTP protocol fail to download in Chrome:

>Mixed Content: The site at 'https://github.com/' was loaded over a secure connection, but the file at 'https://download.microsoft.com/download/7/3/3/733AD403-90B2-4064-A81E-01035A7FE13C/MS%20Partition%20I.pdf' was redirected through an insecure connection. This file should be served over HTTPS. See https://blog.chromium.org/2020/02/protecting-users-from-insecure.html for more details.

(It might be reasonble for somebody else to followup fixing all domains on all pages with regex `\(http://([.a-z0-9-]+)` replacing with `(https://$1`, but I didn't test each blog site supports HTTPS.)
@SingleAccretion SingleAccretion force-pushed the merge-aug-6 branch 3 times, most recently from 58d4aad to 210ada5 Compare August 7, 2024 19:05
@SingleAccretion SingleAccretion marked this pull request as ready for review August 8, 2024 23:40
@SingleAccretion
Copy link
Author

@dotnet/nativeaot-llvm

Copy link
Contributor

@yowl yowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just looked at the resolve conflict commit so far.

Copy link
Contributor

@yowl yowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments

Copy link
Contributor

@yowl yowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!, minor comments is all.

@maraf maraf added the area-NativeAOT-LLVM LLVM generation for Native AOT compilation (including Web Assembly) label Aug 9, 2024
Also delete duplicate cmake arguments.
@hez2010
Copy link

hez2010 commented Aug 9, 2024

Can you seek the merge to the latest commit (particularly, after 99f7f934927c7e2c2af2491cb9b9aa2063a562d2)? I would like to test the Avalonia wasm size saving after it.

@jkotas jkotas merged commit 5be5460 into dotnet:feature/NativeAOT-LLVM Aug 9, 2024
11 checks passed
@jkotas
Copy link
Member

jkotas commented Aug 9, 2024

Thank you!

@SingleAccretion
Copy link
Author

Can you seek the merge to the latest commit (particularly, after 99f7f934927c7e2c2af2491cb9b9aa2063a562d2)? I would like to test the Avalonia wasm size saving after it.

@hez2010 Historically, we've preferred not to do merges too frequently (they are disruptive). An exception can always be made: #2653.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-NativeAOT-LLVM LLVM generation for Native AOT compilation (including Web Assembly)
Projects
None yet
Development

Successfully merging this pull request may close these issues.