Merge upstream #2

* Fix artifact match string. * Completed first attempt for a complete Maui SOD run. Added ios_scenarios tests, setup file copies, uploads, and downloads; and setup perf branch to use as partner testing branch. * Reset performance-setup.ps1 file to use main perf branch. * Reenable yml jobs disabled for testing.

…pecified (#60988) * diable sending NT Authority if specific trust was specified * feedback from review * add missing file * Apply suggestions from code review Co-authored-by: Cory Nelson <phrosty@gmail.com> * one more style update Co-authored-by: Cory Nelson <phrosty@gmail.com>

* All of System.Security.Cryptography.Primitives has been moved * AsymmericKeyExchangeFormatter moved from S.S.C.Algorithms * This just starts the process to keep the infrastructure pieces separate from code layout pieces.

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Remove "block" from VNApplySelectorsAssign It was unused and all the callers passed the current block anyway. Also some renaming to make Assign and AssignTypeCoerce methods consistent. * Improve the readability of VNForMapStore By naming parameters properly. * Improve the readability of VNForMapSelect By naming parameters properly. * Pull ASG numbering out of fgValueNumberTree In my upcoming substantive changes to this code the unreasonable level of nesting started to significantly degrade the readability of code. So, move it out of the main numbering function. The result is still pretty huge and could be split up further, but this will do for now. * Refactor VNApplySelectors Make it use a loop instead of recursion. Rename parameters to match the "Assign" variants. Add standard header comments.

This change enables device runs on CI by building each test app on the helix instance it was deployed to. In past attempts, we looked at enhancing what takes place on the build machine via build tricks, compressing / cleaning up files early, etc and we could not overcome the need for excessively long timeouts and far more disk space. The change also adopts the patterns established in the wasm test build, giving us the opportunity to support different scenario runs as well as workloads testing in the future.

* Remove quasi passwords * oops

* Fix gcc warnings during mono linux-x64 build main with Debug configuration: 822 warnings - http://sprunge.us/2GzrDE PR with Debug configuration: 3 warnings related to deprecated sys/sysctl.h includes - http://sprunge.us/JuyA3K after fixing Debug warnings, there were 13 additional warnings in Release configuration: http://sprunge.us/PJCivP PR with Release configuration: (same) 3 warnings - http://sprunge.us/NwKHNE * Address CR feedback

…code sent (#59313) HTTP2 loopback server WaitForCancellationAsync should validate error code sent

…xtensions.DependencyModel` is empy (#60842) * Setting framework name as nullable. * Setting path to nullable. * Setting runtime to nullable. * Setting path to nullable. * Setting Target Name, RuntimeTargetEntrySub Path and TargetLibrary Name as nullable (string) properties. * Adding missing line of code. * Adding new string resource and throwing argument exception on null/empty values. Adding unit tests to check for specific null/empty value cases * Removing Argument from string resource. Updating PR according to comments * Updating unit tests to use FormatException * Rollback of ArgumentExceptions in DependencyModels * Added missing usings * Updating all exceptions to FormatException * Moving FormatException while reading target name * Updating PR with new comments. * Update src/libraries/Microsoft.Extensions.DependencyModel/src/DependencyContextJsonReader.cs

These packages are unused as described in #60794 Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>

This adds support for setting per-project optimization flags, and sets them for these projects so they don't OOM. Microsoft.Extensions.Logging.Generators.Roslyn3.11.Tests Microsoft.Extensions.Logging.Generators.Roslyn4.0.Tests System.Text.Json.SourceGeneration.Roslyn3.11.Unit.Tests System.Text.Json.SourceGeneration.Roslyn4.0.Unit.Tests System.Text.RegularExpressions.Generators.Tests Some individual ones are disabled due to #58226, and #60899 . Additionally, if a AOT build fails, then on linux it dumps the last few lines from dmesg, to help identify it was an oom-kill. Fixes #51961 . Co-authored-by: Larry Ewing <lewing@microsoft.com>

* Disable couple of randomly failing PerformanceCounter tests * fix: disabled the wrong test

#61102) #61099

…GetField` method (#53645) * Reorder checks for TypeBuilder.GetConstructor() * Reorder checks for TypeBuilder.GetField() * Adjust unit test to reordered TypeBuilder checks * Use 'not' keyword for pattern matching * Apply the same Type checks for Mono GetConstructor and GetField had the same incorrect checks order comparing to GetMethod as in the CoreCLR. To keep the consistent solution, Mono has also been adjusted. * Check whether a declaring type isn't type's generic definition * Make generic type if contains generic parameters * Suppres UnrecognizedReflectionPattern for GetField and GetConstructor * Move GetMethodType validation method at original order * Test GetConstructor against throwing an exception * Fix: copy-paste error 'type' -> 'constructor' * Align Mono and CoreCLR with nameof for GetConstructor * Use type for argument exception and test against type parameter * Get tests passing on both coreclr and mono. Ensure checks are consistent across GetConstructor, GetField, and GetMethod for both coreclr and mono. * Fix mono exception messages to use SR resources. Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>

…#61108) Contributes to #61017

Co-authored-by: Elinor Fung <elfung@microsoft.com>

* Fix refcount when a composite r2r image is loaded from a bundle * PR feedback - better comments.

* Add TFM and version support query to marshaller interface. * If any marshaller fails to support the current tfm version, fallback to using the built-in DllImport attribute. * Add tests for fallback generation on downlevel or generation if marshallers are supported on the target framework. * Convert CreateCompilationWithReferenceAssemblies to CreateCompilation overload. Co-authored-by: Elinor Fung <elfung@microsoft.com>

* added configSrc - path to MonoConfig json file * added Module.onConfigLoaded callback * added Module.onDotNetReady replacing MonoConfig.loaded_cb * removed loaded_cb, assembly_list, runtime_assets, runtime_asset_sources from MonoConfig * simplified all sample and test projects * renamed no_global_exports to disableDotNet6Compatibility * implement default Module.preInit and Module.onRuntimeInitialized which could be overriden by user code (like Blazor) * re-enable the interp lane

Fixes three warnings: ```sh [ 56%] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/utils/mono-mmap.c.o In file included from /runtime/src/mono/mono/utils/mono-mmap.c:23: /usr/include/x86_64-linux-gnu/sys/sysctl.h:21:2: warning: #warning "The <sys/sysctl.h> header is deprecated and will be removed." [-Wcpp] 21 | #warning "The <sys/sysctl.h> header is deprecated and will be removed." | ^~~~~~~ # snip [ 59%] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/utils/mono-time.c.o In file included from /runtime/src/mono/mono/utils/mono-proclib.c:42: /usr/include/x86_64-linux-gnu/sys/sysctl.h:21:2: warning: #warning "The <sys/sysctl.h> header is deprecated and will be removed." [-Wcpp] 21 | #warning "The <sys/sysctl.h> header is deprecated and will be removed." | ^~~~~~~ [ 59%] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/utils/mono-uri.c.o [ 59%] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/utils/mono-poll.c.o In file included from /runtime/src/mono/mono/utils/mono-time.c:107: /usr/include/x86_64-linux-gnu/sys/sysctl.h:21:2: warning: #warning "The <sys/sysctl.h> header is deprecated and will be removed." [-Wcpp] 21 | #warning "The <sys/sysctl.h> header is deprecated and will be removed." | ^~~~~~~ ``` With this PR: 0 warnings - http://sprunge.us/96kyq9 Like in src/libraries/Native, skip linux from sys/sysctl.h check. For other systems, `check_include_files` (instead of `check_include_file` (singular)) when a header depends on another one. Also, simplified few conditions: "is linux" check supersedes "is android".

… Xamarin.MacCatalyst.dll as well. (#60854) Ref: xamarin/xamarin-macios#13092 Ref: mono/mono#21261

* Draft of hidden attribute decorator use. * Removed checking custom attributes in the runtime each time the breakpoint is set and moved it to the constructor which is more efficient. * Added test for DebuggerHidden decorator. * By adding a line: using System.Diagnostics; IntAdd method moved down one line what resulted in CreateGoodBreakpoint tests failures. All row numbers in these tests had to be incremented. * Reverted edition of tests not connected with HiddenAttribute. * Added visible method to the HiddenAttribute test. * Applying thaystg review suggestion.

* fix HttpClientHandlerTest.RemoteServer.cs * fix HttpClientHandlerTest.Authentication.cs * fix HttpClientHandlerTest.cs

We were hitting both transforms in this block, but the first transform changes the node from GT_ADD to GT_SUB, while the second transform is valid only for GT_ADD. It is difficult to have natural code that hits both of these transforms, so I have just made them mutually exclusive instead of rechecking the oper in the second one. No SPMI diffs. Fix #61077

* Remove _validator field from ObjectCollection * Use more specific types for fields and args

Co-authored-by: Rainer Sigwald <raines@microsoft.com> Co-authored-by: Jeremy Koritzinsky <jekoritz@microsoft.com>

Add a new optimization to redundant branch opts that looks at prior statements in the same block for redundant relop trees. If one is found, we check to see if it can be forward-substituted down to the terminal jump tree. And if it can we duplicate the computation down at the jump. This removes many of the cases we see in our generated code where we materialize a boolean value in a register and then immediately test that register to see if it is true/false, and then use that second test to drive branching -- instead we now use the initial test logic to drive the jump and so the boolean value only exists in the flags.

…erop guidelines - part 2 Convert more of S.S.C.X509Certificates to use correct interop style.

* fixed nullability checking to be threadsafe * use volatile write for nullability cache Co-authored-by: Stephen Toub <stoub@microsoft.com> * made read of nullability cache volatile and fixed other PR comments * made non-lock-protected reads from SymbolicRegexBuilder._delta volatile * Apply suggestions from code review Co-authored-by: Stephen Toub <stoub@microsoft.com>

…61188)

Since inlined statements use the IL location of the inliner's statement all statements created while inlining devirtualized calls would get no debug info before, while normally these will have IL location pointing to the call statement in the root.

Co-authored-by: SingleAccretion <62474226+SingleAccretion@users.noreply.github.com>

…lfilled. (#61165)

…1020) * [interp] Fix GetType called on ptr constrained to Nullable` We were statically optimizing this call to return the actual constrained class type, which is incorrect for nullables, because boxing of a nullable (as part of the constrained call) actually creates an object with the type of the nullable's value (or null if there is no value). * Add test for GetType call on ptr constrained to nullable

…61142) * improved BDD Unicode table representation in NonBacktracking engine * remove line Co-authored-by: Dan Moseley <danmose@microsoft.com> * improved bounds-check elimination Co-authored-by: Stephen Toub <stoub@microsoft.com> * clearer notation of numbers Co-authored-by: Dan Moseley <danmose@microsoft.com> * fixed typo Co-authored-by: Dan Moseley <danmose@microsoft.com> Co-authored-by: Stephen Toub <stoub@microsoft.com>

… the marshalling generator for collection elements (#61219)

* Split the new library from just NetCoreAppCurrent to all of the varieties that S.S.C.Encoding had, plus browser. * Move src/libraries/System.Security.Cryptography.Encoding/src/System/... to src/libraries/System.Security.Cryptography/src/System/... * Move src/libraries/System.Security.Cryptography.Encoding/src/Internal/Cryptography/* to src/libraries/System.Security.Cryptography.Encoding/src/System/Security/Cryptography/* * Merge in the Strings.resx content that was needed. * Added two ObjectDisposedException.ThrowIf calls to eliminate a resource from the move. * Move src/libraries/System.Security.Cryptography.Encoding/tests/... to src/libraries/System.Security.Cryptography.Encoding/tests/..., with a couple files gaining the "Tests" suffix that their class had. * Fix the namespaces of the previously moved from-Primitives tests to match the new location. * Fix the namespaces of the newly moved from-Encoding tests to match the new location. * Make System.Security.Cryptography.Encoding ref & src just build as facades/forwarders.

)

…rt . (#61212) Revert back to eval in wasm InvokeJS with modularization support . Wrap code to evaluate in a function with MONO, BINDING, INTERNAL and module as local variables. Added tests for running js expressions (successful, null, undefined, global scope separation) Co-authored-by: Pavel Savara <pavelsavara@microsoft.com>

* Fix UnobservedTaskException from SemaphoreSlim.WaitAsync If a SemaphoreSlim.WaitAsync times out, it correctly returns false, but it also results in TaskScheduler.UnobservedTaskException being raised unexpectedly, due to internal use of a faulted task whose exception isn't observed. This fixes that by marking any such exceptions as having been observed. * Fix wasm build

Fixes #61190.

Just enable AVX, so that we can run the lib on any of our test machines.

* Generate dynamic serialization assembly in the appropriate ALC, and don't keep any hard refs to types that could prevent unloading.

) (#61199)

Co-authored-by: Larry Ewing <lewing@microsoft.com>

PartitionTree had a holdover from when it was tracking the predicates that made up each minterm, which caused it to use order n too much memory for some patterns. Nodes with the same predicate as the parent are no longer created. This also significantly simplified the algorithm. Remove the deduplication, since its now optional and the only call site already passes in a unique set of predicates. Make MintermGenerator take an IEnumerable.

…dDllImport (#61184) * Enable DllImport generator for all source projects. * Convert OleDb to only use GeneratedDllImport. * Update source generator to distinguish between "no marshalling info" which can cause a failure vs "missing support" which enables us to fallback to the marshalling forwarder. * Remove unused DllImports. * Reuse existing P/Invoke definitions.

* Move late arithmetic to its own section in gtlist * Implement AND_NOT for AAarch * Delete the unnecessary platfrom-specific methods They were a leftover from some previous work. * Mention the SIMD origins of AND_NOT

…rom GeneratedDllImportAttribute (#61182)

Co-authored-by: Geoffrey Kizer <geoffrek@windows.microsoft.com>

…rappers RCW. (#61267) * Don't create a COM weak reference if the object is an aggregated COMWrappers RCW. * Add test for weak reference + aggregation with native weak reference impl. * Apply suggestions from code review Co-authored-by: Aaron Robinson <arobins@microsoft.com> Co-authored-by: Aaron Robinson <arobins@microsoft.com>

…sserts (#61226) Asserts not provided by Xunit are provided by an AssertExtensions class. Tests that reference System.Private.Corelib directly will use a polyfill implementation based off the old CoreCLRTestLibrary asserts. All assert methods provided by CoreCLRTestLibrary have been changed to follow Xunit conventions.

* [interp] Fix opcode dump * [interp] Implement tailcalls Before this commit we supported tailcalls only for calls to the same method. Tailcalls are implemented by copying the call arguments to the start of the stack space, followed by replacing of the current executing method. * Re-enable test

* [wasm] Add console version of browser-bench sample Example run: > v8 --expose_wasm runtime.js -- --run Wasm.Console.Bench.Sample.dll -t Json:non,Exceptions:Inline console.info: Arguments: --run,Wasm.Console.Bench.Sample.dll,-t,Json:non,Exceptions:Inline console.debug: MONO_WASM: Initializing mono runtime console.debug: MONO_WASM: ICU data archive(s) loaded, disabling invariant mode console.debug: mono_wasm_runtime_ready fe00e07a-5519-4dfe-b35a-f867dbaf2e28 console.info: Initializing..... Benchmark started Json, non-ASCII text serialize count: 23, per call: 7.565217391304348ms, total: 0.174s ... Exceptions, TryCatchFilterInline count: 1766666, per call: 4.301888415806949E-05ms, total: 0.076s Summary Json, non-ASCII text serialize: 7.478260869565218ms Json, non-ASCII text deserialize: 10.789473684210526ms Exceptions, TryCatchFilterInline: 4.301888415806949E-05ms .md | measurement | time | |-:|-:| | Json, non-ASCII text serialize | 7.4783ms | | Json, non-ASCII text deserialize | 10.7895ms | | Exceptions, TryCatchFilterInline | 0.0430us | * Don't generate v8 script for browser project * Fix option set * Changes for linux/mac * Add link to bench sample README * Pass ARGS to the console version Add info about filtering and options to the README * Show how the table will looks like * Do not sign the assembly To avoid CSC : error CS8002: Referenced assembly 'Mono.Options, Version=6.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. * Add examples how to run it on windows * Fix interpolated string * Improve formatting * Remove unwanted space

These are leaf nodes without side effects, and we can sometimes evaluate them in VN, so it does not seem like there is any reason not to enable propagation for them.

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Premek Vysoky <premek.vysoky@microsoft.com>

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

When value numbering an assignment to a field, VNApplySelectorsAssign was called twice, first for the field sequence to find the exact value, then for the heap VN update. This is not correct as the method expects to be called only for sequences that will end up updating a map with exact values, as it type checks the store with the helper VNApplySelectorsAssignTypeCoerce. Usage of VNApplySelectorsAssign to update the heap ended up meaning that any stores to struct fields ended up with casts for maps, blocking traversal in VNForMapSelect. A handful of positive diffs for this commit resulting from more CSEs and redundant branch optimizations, due to more precise liberal VNs.

* remove m_hostAssemblyMap and ceremony around it * Removes SHash::AddPhases - support for callouts in hosted scenarios. * typos

* use HashTypeKey(), remove ComputeHash() * nonrecursive HashTypeHandle * remove instrumentation from pendingload.cpp * two more uses of `ComputeHash()` * Make GCC happy * couple tweaks * unnecessary full fences.

* Fix incorrect CCW decoding in GetObjectForCCW * Fix linux/mac build

Fixes #60454.

Adjust for dotnet/jitutils#343

Handle the case where the side-effecting redundant relop appears just before the jump tree relop. Also revamp how we search for related VNs so it can be done as a search loop.

* Add instructions for using Codespaces * Automate resetting the HEAD to the same commit ID used to build the pre-built container * - Add notes about testing changes. - Add 4-core machines * Revert 4-core change since prebuild doesn't support multiple sku_names.

Now that the Encoding and Primitive assemblies are unified, an implementation for PKCS#8 and SPKI keys can be provided in the abstract AsymmetricAlgorithm. Derived types will continue to override these members to provide support for additional PEM labels.

…Roslyn3.11.Tests` failing due to OOM (#61323) * [wasm] Disable tests failing due to OOM Microsoft.Extensions.Logging.Generators.Roslyn3.11.Tests #61339 * [wasm] Use special exitcodes when AOT test build fails, and when it gets oom-killed Co-authored-by: Ankit Jain <radical@gmail.com>

…60057) * use interpolation string * fix XmlSerializationReader * fix new line * fix new line for XmlSerializationReaderILGen * review feedback * review feedback * fix XmlTextWriter * fix Compiler * fix XmlSchemaExporter * fix XmlSchemas * fix XmlSerializationReader * fix XmlSerializationWriter * fix XmlSerializationWriter * fix XmlSerializationWriterILGen * fix QilValidationVisitor * fix XmlSerializationReaderILGen * fix XmlQueryOutput Co-authored-by: kronic <kronic@softland.ru>

It is not needed anymore, as the flags and stripping is handled by msbuild and cmake. The configuration dependent template parameters are not used anymore.

#61289) * Add System.Diagnostics.StackFrame.GetMethodInfoFromNativeIP API for VS4Mac Issue: #61186

Fix the misleading total bytes displayed when diffing with SPMI. If one of the JITs encountered missing data, the code bytes would not be included, while the other JIT could still have it included if it did not encounter missing data. Also add more information about missing SPMI data/successful replays printed. For replays that is useful to be able to gauge whether there is still ok coverage after a large change. For diffs, we print a warning for missing SPMI data that the diff summary may be misleading. Example for a successful replay: Clean SuperPMI replay (219868 contexts processed) Example for a replay with missing data: SuperPMI encountered missing data for 6 out of 27272 contexts Example warning printed: Warning: SuperPMI encountered missing data during the diff. The diff summary printed above may be misleading. Missing with base JIT: 0. Missing with diff JIT: 6. Total contexts: 27272.

* Number zero-initialized struct properly Previously, zero-initialized struct locals were given a special $VNForZeroMap, which had the nice property that VNForMapSelect($VNForZeroMap, Field) would always return zero of the appropriate type. However, the fact that this VN is not unique meant that it was dangerous to propagate it, e. g. through copies, which was hidden by the fact that ApplySelectorsTypeCheck's logic is very conservative. This change introduces a new VNFunc to represent zeroed objects - VNF_ZeroObj, that takes a struct handle as its argument and is thus free of the aforementioned problem and can be propagated freely and not treated specially. It also, of course, retains the ZeroMap's selection property. There are almost no diffs for this change because of some other deficiencies in assertion and copy propagation, they are being/will be fixed separately. * Fix VNZeroForType's handling of SIMD types Previously this codepath was reachable, but did not matter as the returned values were immediately discarded because of the conservative logic in VNApplySelectorsTypeCheck. This is still more or less the case, but let's just fix it properly.

* Refactor IL_OFFSETX into DebugInfo Remove IL_OFFSETX in favor of a DebugInfo structure. Previously we were packing extra information into the upper bits of IL_OFFSETX, which are now separate bit fields on a new ILLocation structure. DebugInfo contains an ILLocation and also an inline context, which will be used in the future when tracking debug info inside of inlinees. Another problem with IL_OFFSETX was that there were several sentinel values used to describe prologs, epilogs and no mappings. However these were only used in code-gen, so refactor codegen to track this separately instead of having to muddle it into IL_OFFSETX. This makes it clearer what we can expect from IL offsets during JIT. This change is no-diff and PIN also shows that TP is not negatively affected; in fact, there seems to be a small TP gain, maybe because we don't have to handle sentinel values anymore. * Track debug information in statements from inlinees Add support for tracking debug information in statements coming from inlinees. Changes: * Turn on compDbgInfo in inlinees. We use the implicit boundaries from the inline root, but we do not use any explicit boundaries. That is, we do not query the EE for explicit boundaries for the inlinee. * Create InlineContexts eagerly and use them during import. All DebugInfo created in the JIT is in a "consistent" state, meaning that we never see an IL location set without a corresponding inline context. This was difficult before as InlineContexts would be created quite late, after the importer for the inlinee had run. We now create it eagerly and attach it to debug info during importation. Later, when we figure out whether an inline succeeded or not, we mark it as succeeded or failed. * Stop carrying InlineContext around unconditionally in Statement. The inline context is now only part of the debug info, which may not be set. Inlining needs the inline context to create new inline contexts and to check for recursive inlines. Previously it retrieved it from the inline statement, but due to the above change we now have to get it from somewhere else. To do this we now keep it unconditionally together with InlineCandidateInfo so that we can retrieve it later. * Validate all created debug information when associated with a statement. This is done by creating a bitvector containing IL locations that mark the beginning of IL instructions, and validating that all IL offsets point to these when Statement::SetDebugInfo is called. * While we track debug info in statements from inlinees, the runtime side is still not hooked up. Currently we track the information until we get to rationalize, where we normalize all debug info back to the root inserted as GT_IL_OFFSET nodes. The change is free of any diffs due to this normalization. We also track IL offsets as part of basic blocks: these are also normalized to be in the root.

Fixes #51519

…ling manifests (#60699)

Fixes #61144.

)

Direct heap writes via Module.HEAPxx[y] = func(...) are incorrect because the left-hand side (according to spec) is evaluated before the right, so if evaluating func(...) causes the heap to grow, the assignment target becomes a detached buffer and the write goes nowhere, breaking your application. This PR introduces a new set of helper functions for memory reads and writes.

Add various tests for marshaling task to JavaScript promise. - Sync and async. - Task and ValueTask. - Generic and non-generic. - Successful and failed.

#61386) * [wasm][debugger[ Correct the endian swapping and string implementation

Fixes #61067 Fixes #59538

* Delete unused mono_unbox_int * Fix gcc step display name * Match command invocation order

…e to be blittable (#61071)

Create a new `runtime-coreclr superpmi-asmdiffs` pipeline that runs SuperPMI asmdiffs for every change in the JIT directory. The diffs are run on two platforms: Windows x64 and Windows x86. Linux, and Arm64 and Arm32, asm diffs are done using cross-compilers, as follows: | Platform | Asm diffs | | -- | -- | | Windows x64 | win-x64, win-arm64, linux-x64, linux-arm64 | | Windows x86 | win-x86, linux-arm | The resulting summary .md files are uploaded into the pipeline artifacts, one .md file per platform (so, one for the Windows x64 runs and one for the Windows x86 runs). The results are also displayed in "Extensions" page of the AzDO pipeline. The runs take about 50 minutes to complete (assuming not much waiting for machines). The asm diffs pipeline is similar to the "superpmi-replay" pipeline, except: 1. It determines what an appropriate baseline JIT would be based on the PR commit and how it merges with the `main` branch. Given this, it downloads the matching baseline JITs from the JIT rolling build artifacts in Azure Storage. 2. It clones the `jitutils` repo and builds the `jit-analyze` tool, needed to generate the summary .md file. 3. It downloads and adds to the Helix machine payload a "portable" `git` installation, as `git diff` is used by `jit-analyze` for analyzing the generated .dasm files of the diff. 4. It collects all the various summary.md files into one per platform on which the runs are done, and publishes that to the artifacts and the `Extensions` page. 5. It only does one replay (asmdiffs) run, not one for each of a set of multiple stress modes. As part of this implementation, a. The `azdo_pipelines_util.py` was renamed to `jitutil.py`, and a lot of utility functions from superpmi.py were moved over to it. This was mostly to share the code for downloading and uncompressing .zip files. (There is a slight change to the output from the `superpmi.py download` command as a result.) However, I also moved a bunch of simple, more general helpers, for possible future sharing. b. `jitrollingbuild.py download` can now take no arguments and download a baseline JIT (from the JIT rolling build Azure Storage location), for the current enlistment, to the default location. Previously, it required a specific git_hash and target directory. There is similar logic in superpmi.py, but not quite the same. c. The `superpmi.py --no_progress` option was made global, and applied in a few more places. This was necessary because `superpmi.py asmdiffs` will download a coredistools binary from the JIT Azure Storage if one isn't found in the Core_Root folder. Fixes #59445

… dotnet/hotreload-utils dotnet/llvm-project (#61365) Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Ankit Jain <radical@gmail.com>

Previously the Directory.Build.targets script used to automatically infer that OutputType=Library without a CLRTestKind implies SharedLibrary. This is however hard to consolidate with the planned test merging - as the SDK script set OutputType=Library by default, we need the combination Library+(implicit)BuildAndRun to indicate the "new-style" [Fact]-based tests. For this reason I propose to remove this automatic inference and manually fix the handful of tests that are missing an explicit CLRTestKind=SharedLibrary property. In light of this description we can theoretically remove the OutputType=Library specification from all test projects but even if we decide to do that, I believe it will be easier to do that as a separate mechanical change, not as part of this relatively small change that has a different purpose. Additionally in the one case of the GitHub_22583 regression test, I removed the explicit setting of GenerateRunScript=false because that's the default. Fix OutputType=exe in OpenDelegate.csproj I believe this was a pre-existing bug - previously, with the special clause regarding SharedLibrary, the test just got silently skipped because it was considered to be a shared library. Thanks Tomas

* coreclr/ Make sure integral types respect sign extension during widen operation for Invoke Array ctor. Remove always false IsStructRequiringStackAllocRetBuf(). * mono/ Create macro define for SPAN_T. * libraries/ Add tests for Reflection Binder type conversion support during Invoke.

Catch patterns like the one in #37904 where a trinary compare feeds a binary compare.

Fixes #61322 Fixes #61299

* We were not recording precise info in inlinees except for at IL offset 0 because most of the logic that handles determining when to attach debug info did not run for inlinees. There are no changes in what the EE sees since we were normalizing debug info back to the root anyway. * Propagate debug info even further than just until rationalization, to make it simpler to dump the precise debug info. This means we create some more GT_IL_OFFSET nodes, in particular when the inlinee debug info is valid but the root info is invalid. This is currently happening for newobj IL instructions when the constructor is inlined. We generate two statements: GT_ASG(GT_LCL_VAR(X), ALLOCOBJ(CLS)); GT_CALL(CTOR, GT_LCL_VAR(X)) and the first statement ends up "consuming" the debug info, meaning we end up with no debug info for the GT_CALL, which eventually propagates into the inline tree. I have held off on fixing this for now since it causes debug info diffs in the data reported back to the EE. The additional nodes in LIR result in 0.15% more memory use and 0.015% more instructions retired for SPMI over libraries. There is also a small fix in gtlist.h for GT_BFIZ when MEASURE_NODE_SIZES is defined. No SPMI diffs.

There are connectivity issues on some physical Android devices. We should disable the affected tests until the issue isn't resolved. Ref #61343

) We reference entry #0 in the pinned plug queue even if there are no pinned plugs at all and thus the pinned plug queue contains left-over data from the mark phase. The fix is to initialize saved_pinned_plug_index to a value that is invalid as a pinned plug queue index, and only use saved_pinned_plug_index as an index if is valid.

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

Don't run if the JIT-EE GUID has changed, since there won't be any collections to download, so the downloads will fail. Also update the superpmi-asmdiffs trigger comment.

#61449)

… annotate individual methods as requiring AVX (#61259) * Updating src/tests/Interop/PInvoke/Generics/GenericsNative.Vector* to annotate individual methods as requiring AVX * Always use __m256i on XARCH

* XARCH: Remove redudant tests for GT_LT/GT_GE relops. We can now optimize cases such as `(x + y < 0)` or `for (int x = v; x >= 0; x--)` using the flag tracking logic during the emit stage. Notably, cases that would generate... ``` add reg0, reg1 test reg0, reg0 jge LABEL ``` now transform to ``` add reg0, reg1 jns LABEL ``` This transform is valid for signed GE and signed LT only. * Add a few asserts related to flag reuse optimizations.

* Moving Narrow to implemented using SIMDAsHWIntrinsic * Moving Widen to implemented using SIMDAsHWIntrinsic * Fix some handling of Narrow/Widen hwintrinsics * Ensure that Vector.Widen is still treated as an intrinsic * Fixing NI_VectorT128_WidenUpper on ARM64 to actually call gtNewSimdWidenUpper

Measure browser app start times, 2 measurements implemented. First to measure till the JS window.pageshow event, second to measure time when we reach managed C# code. Example ouput: | measurement | time | |-:|-:| | AppStart, Page show | 108.1400ms | | AppStart, Reach managed | 240.2174ms |

* Environment.SystemPageSize returns cached value * we are no longer shipping MS.IO.Redist, so we can use Array.MaxLength directly * we are no longer shipping MS.IO.Redist, there is no need for File to be partial * we are no longer shipping MS.IO.Redist, there is no need for FileInfo to be partial * there is no need for .Win32.cs and .Windows.cs file anymore

…61479)

* Remove all DLLIMPORTGENERATOR_ENABLED usage. * Explicitly set EnableDllImportGenerator to true in test project.

… the receiver is a gshared type constrained to a primitive type/enum. (#61513)

…trained types. (#61514) These are used for example in Span<T>:.ctor ().

This was preventing block compaction with loop entry blocks in loops that had been previously optimized away (and thus removed from the loop table). There are a few cases where we now delete dead code that was previously left in the function. There are a number of spurious local weighting and IG textual asm diffs changes, possibly due to how PerfScore is implemented (there are some surprisingly large PerfScore changes in a few cases, despite no change in (most) generated code).

…ardless of what they point at (#61538)

Implement method name wildcard matching for method descriptions Globbing doesn't work because we don't have g_pattern_match_simple in eglib. But a plain '*' wildcard does work.

When both OSR and PGO are enabled: * Enable instrumenting OSR methods, so that the combined profile data from Tier0 plus any OSR variants provide a full picture for subsequent Tier1 optimization. * Use block profiles for both Tier0 methods that are likely to have patchpoints and OSR methods. * Fix phase ordering so partially jitted methods don't lose probes. * A few more fixes for partial compilation, because the number of things we think we might instrument and the number of things we end up instrumenting can differ. * Also improve the DumpJittedMethod output for OSR, and allow selective dumping of a particular OSR variant by specifying its IL offset. The updates on the runtime side are to pass BBINSTR to OSR methods, and to handle the (typical) case where the OSR method instrumentation schema is a subset of the Tier0 method schema. We are still allowing OSR methods to read the profile data. So they are both profile instrumented and profile optimized. Not clear if this is going to work well as the Tier0 data will be incomplete and optimization quality may be poor. Something to revisit down the road.

Emit it in the interpreter when a method is inlined or replaced with an intrinsic. This is needed so the AOT profiler can track these methods.

…traints. (#59437) If a generic argument is a primitive type, and it has an interface constraint that enums don't implement, then partial sharing for that instance is not useful, since only the specific primitive type instance will be able to use the shared version. Fixes #54910.

* Enable new analyzers in global configs * Address PR feedback

For now there's no functional change to the behavior of the tests, I have just copied the bits to inject from Jeremy's example in his pending PR. Thanks Tomas

* embedded size * next array * read consistency while rehashing * comments * remove CRST_UNSAFE_ANYMODE and COOP mode in the callers * typo * fix 32bit builds * couple changes from review. * Walk the buckets in resize. * remove a `REVIEW:` comment. * Update src/coreclr/vm/dacenumerablehash.inl PR review suggestion Co-authored-by: Jan Kotas <jkotas@microsoft.com> * remove use of `auto` * DAC stuff * Constructor and GrowTable are not called by DAC, no need for DPTR, added a comment about a cast. * SKIP_SPECIAL_SLOTS * More compact dac_cast in GetNext Co-authored-by: Jan Kotas <jkotas@microsoft.com>

For now there's no functional change to the behavior of the tests, I have just copied the bits to inject from Jeremy's example in his pending PR. Thanks Tomas Contributes to: #54512

For now there's no functional change to the behavior of the tests, I have just copied the bits to inject from Jeremy's example in his pending PR. I have spot-checked that some of the tests use Main with the command-line args argument. I'm not changing them in this PR, the signature only becomes important once we start actually merging the IL tests and I presume we'll clean that up at that point. Thanks Tomas Contributes to: #54512

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

* Add JsonSerializerOptions.Default * address feedback * address feedback

`__DoCrossgen2` appears to be redundant with `__TestBuildMode=crossgen2`.

Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

The issue was that VNApplySelectors uses the types of fields when selecting, but that's not valid for "the first field" maps. Mirror how the stores to fields are numbered.

For very large structs (> 64K in size) poisoning could end up generating instructions requiring larger local var offsets than we can handle. This hits IMPL_LIMIT that throws InvalidProgramException. Turn off poisoning for larger structs that require more than 16 movs to also avoid the significant code bloat by the singular movs. This is a less risky version of #61521 for backporting to .NET 6. Fix #60852

…tValueAtIndex (#60520) * SortedList<TKey, TValue> added GetKeyAtIndex, GetValueAtIndex, and SetValueAtIndex * Update src/libraries/System.Collections/src/System/Collections/Generic/SortedList.cs Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>

This optimization is only legal if: 1) "Anything" is a sufficiently small constant itself. 2) We are in a context where we know the address will in fact be used for an indirection. It is the second point that is problematic - one would like to use MorphAddrContext, but it is not suitable for this purpose, as an unknown context is counted as an indirecting one. Additionally, the value of this optimization is rather low. I am guessing it was meant to support the legacy nullchecks, before GT_NULLCHECK was introduced, and had higher impact then. So, just remove the optimization and leave the 5 small regressions across all of SPMI be.

* Apply suggestions from code review Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com> Co-authored-by: Jeremy Koritzinsky <jkoritzinsky@gmail.com>

…Process, System.Diagnostics.Process, System.Diagnostics.FileVersionInfo, System.Runtime.InteropServices.RuntimeInformation (#61532)

* FileSystem.Unix: improve CopyFile. Like the upcoming version of GNU coreutils 'cp' prefer a copy-on-write clone. This shares the physical storage between files, which means no data needs to copied. CoW-clones are supported by a number of Linux file systems, like Btrfs, XFS, and overlayfs. Eliminate a 'stat' call that is always performed for checking if the target is a directory by only performing the check when the 'open' syscall reports an error. Eliminate a 'stat' call for retrieving the file size of the source by passing through the length that was retrieved when checking the opened file is not a directory. Create the destination with file permissions that match the source. We still need to fchmod due to umask being applied to the open mode. When performing a manual copy, limit the allocated buffer for small files. And, avoid the last 'read' call by checking when we've copied the expected nr of bytes. * Don't FICLONE for zero sourceLength * PR feedback * When using sendfile, don't loop when source file gets truncated. * Fall through when FICLONE fails. * Don't stop CopyFile_ReadWrite until read returns zero. * Revert all changes to CopyFile_ReadWrite * Move comment a few lines up. * Fix unused error.

Since libproc is a private Apple API, it is not available on iOS/tvOS and should be excluded (see #61265 (comment) and above for more details). This PR excludes $(CommonPath)Interop\OSX\Interop.libproc.cs on the iOS/tvOS as well as makes some methods in Process, ProcessManager, and ProcessThread classes calling that API throw PNSE so that for iOS/tvOS it's possible to re-use the respective *.UnknownUnix.cs parts.

We've tried to consistently use global:: whenever referring to core library types in the regex generator emitted code. I'd missed these two. (That said, these make the code a lot harder to read, especially in places where we're unable to use extension methods as extensions, so we'll want to revisit this policy.)

…#52639) * Implement most of the fix for #38824 Reverts the changes in the seperate PR a617a01 to fix #38824. Does not re-enable the test because that relies on #49555, will add a seperate commit to whichever is merged last to enable the SettingUpdatesPropertiesOnSymlink test. * Most of the code for PR #52639 to fix #38824 • Deal with merge conflicts • Add FSOPT_NOFOLLOW for macOS and use it in setattrlist • Use AT_SYMLINK_NOFOLLOW with utimensat (note: there doesn't seem to be an equivalent for utimes) • Add SettingUpdatesPropertiesOnSymlink test to test if it actually sets it on the symlink itself (to the best that we can anyway ie. write + read the times) • Specify FILE_FLAG_OPEN_REPARSE_POINT for CreateFile in FileSystem.Windows.cs (is there any other CreateFile calls that need this?) * Remove additional FILE_FLAG_OPEN_REPARSE_POINT I added FILE_FLAG_OPEN_REPARSE_POINT before and it was then added in another PR, this removes the duplicate entry. * Add missing override keywords Add missing override keywords for abstract members in the tests. * Fix access modifiers Fix access modifiers for tests - were meant to be protected rather than public * Add more symlink tests, rearrange files • Move symlink creation api to better spot in non-base files • Add IsDirectory property for some of the new tests • Change abstract symlink api to CreateSymlink that accepts path and target • Create a CreateSymlinkToItem method that creates a symlink to an item that may be relative that uses the new/modified abstract CreateSymlink method • Add SettingUpdatesPropertiesCore to avoid code duplication • Add tests for the following variants: normal/symlink, target exists/doesn't exist, absolute/relative target • Exclude nonexistent symlink target tests on Unix for Directories since they are counted as files * Fix return type of CreateSymlink in File/GetSetTimes.cs * Remove browser from new symlink tests as it doesn't support creation of symlinks Remove browser from new symlink tests as it doesn't support creation of symlinks * Use lutimes, improve code readability, simplify tests • Use lutimes when it's available • Extract dwFlagsAndAttributes to a variable • Use same year for all tests • Checking to delete old symlink is unnecessary, so don't • Replace var with explicit type * Change year in test to 2014 to reduce diff * Rename symlink tests, use 1 core symlink times function, and check that target times don't change Rename symlink tests, use 1 core symlink times function, and check that target times don't change * Inline RunSymlinkTestPart 'function' Inline RunSymlinkTestPart 'function' so that the code can be reordered so the access time test can be valid. * Share CreateSymlinkToItem call in tests and update comment for clarity * Update symlink time tests • Make SettingUpdatesPropertiesOnSymlink a theory • Remove special case for Unix due to #52639 (comment) (will revert if fails) • Rename isRelative to targetIsRelative for clarity * Remove unnecessary Assert.All * Changes to SettingUpdatesPropertiesOnSymlink test • Rename item field to link field • Don't use if one-liner • Use all time functions since only using UTC isn't necessary • Remove the now-defunct IsDirectory property since we aren't checking it anymore * Remove unnecessary fsi.Refresh() • Remove unnecessary fsi.Refresh() since atime is only updated when reading a file * Updates to test and pal_time.c • Remove targetIsRelative cases • Multi-line if statement • Combine HAVE_LUTIMES and #else conditions to allow more code charing * Remove trailing space

by removing obvious warnings

* Delete regArgList Unused. * Delete CountSharedStaticHelper Unused. * Delete GTF_RELOP_QMARK Unused. * Delete lvaPromotedStructAssemblyScratchVar Unused. * Delete dead code from fgMorphSmpOp The conditions tested constitute invalid IR. * Delete gtCompareTree Unused. * Delete gtAssertColonCond Unused. * Delete IsSuperPMIException Unused.

- Remove unneeded FIRST concept in loop table; it was always equal to TOP. - Rename optMarkLoopsBlocks to optScaleLoopBlocks to more accurately describe what it does. More consistently report block scaling in the dump - Create optMarkLoopHeads. This was refactored out of fgRemoveUnreachableBlocks so it can be called in a more logical location (at the beginning of optFindLoops), and only does one thing. - fgMakeOutgoingStructArgCopy: remove unused `argIndex` argument; reorder calls to fgMightHaveLoop. - Update and write a bunch of comments; convert away from `/* */` style comments.

) This change allows devs to manually kick off full test runs on the configurations that only execute smoke tests per PR. /azp runtime-staging-manual will do the trick

* Adding support for X86Base.Pause() and ArmBase.Yield() * Applying formatting patch * Ensure NI_ArmBase_Yield actually gets through to codegen on arm64

The comment above the method mentions "many problems" with leaving null pointers around, but it is unclear what kind of problems. I can only speculate those were the problems in legacy codegen which "could not handle constant op1". It also mentions that "we cannot even fold (null+offset)", which is incorrect: "gtFoldExprConst" does in fact fold such expressions to zero byrefs. It is also the case that spilling the null into a local affects little as local assertion propagation happily propagates the null back into its original place. There was also a little bug associated with the method that got fixed: morph was trying to use it, and in the process created uses of a local that was not initialized, since the statement list used by the method is the importer's one, invalid in morph.

Currently, Actions in the dotnet/runtime repository have read/write access by default, unless their permissions have been explicitly declared. The markdownlint workflow can be restricted from all access except the repository contents. This limits what the 3rd party `markdownlint-cli` npm package can do which is installed as part of the workflow.

* Obsolete thumbtacked AssemblyName properties Fix #59061 * Ignore obsoletion * Fix pragma * Merge the AssemblyName member obsoletions into a single diagnostic id * Fix pragma to use updated diagnostic id * Suppress SYSLIB0037 in reflection tests * Suppress SYSLIB0037 warnings Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com> Co-authored-by: Jeff Handley <jeff.handley@microsoft.com>

* Trim the hot code size to the actual code length * Remove allocatedHotCodeSize field * Remove verbose logging * Perform trimming only for xarch * Include more comment about armarch * Fix the condition for arm64

* Provide locations for src-gen diagnostic output * Fix tests and address feedback * Add defensive check for context type location

The on-demand Azure module load was refactored to jitutil.py, but that requires some cross-module importing to work. Do the minimal required to make this work.

For very large structs poisoning could end up generating instructions requiring larger local var offsets than we can handle which would hit an IMPL_LIMIT that throws InvalidProgramException. Switch to using rep stosd (x86/x64)/memset helper (other platforms) when a local needs more than a certain number of mov instructions to poison. Also includes a register allocator change to mark killed registers as modified in addRefsForPhysRegMask instead of by the (usually) calling function, since this function is used directly in the change.

* Unify some code and load mono-config.json using our _fetch_asset

@lewing

…xpressions (#61470) * Fix memory consumption. * Fix debugger-tests * Fix compilation. * Addressing @lewing PR. * Address @lewing comment * Addressing @radical comment. * Addressing comments. * Addressing @radical comments. * missing return. * Addressing @radical comments * Adding test case Co-authored-by: Larry Ewing <lewing@microsoft.com> * Fixing tests. * Adding another test case. Thanks @lewing. * Reuse the script. Co-authored-by: Larry Ewing <lewing@microsoft.com>

@lewing

* It's working when debug from chrome but not when debug from VS, because it uses callFunctionOn * Remove unrelated change. * Working also on VS. * Working also on VS. * Addressing @lewing and @radical comments * Change ArrayDimensions to be a record and not a class as suggested by @radical. * Addressing @radical comments.

* Update WASM README.md

…on, System.IO.Compression.Brotli, System.Net.Http, System.Net.NameResolution (#61638)

… Build ID 1472678 (#61680) * Localized file check-in by OneLocBuild Task: Build definition ID 679: Build ID 1472242 * Localized file check-in by OneLocBuild Task: Build definition ID 679: Build ID 1472678

… convert to GeneratedDllImport (#61678)

…codespace with prebuilt (#61684)

Adjust partial comp not to import IL instead of importing it and then deleting the IR. Saves some time and also sometimes a bit of stack space as we won't create as many temps. Update both PC and OSR to check for blocks in handlers early, rather than pretending to support these and then backing out later.

* Update SPCL to use GeneratedDllImport where possible.

- no imports from outer scope - move dotnet to -extern-pre-js - re-enable JS minification with ES2018 - rename main javaScript files to main.js and test-main.js - sample and test script cleanup - rename set_exit_code method - rewrite test start as async method - improve script loading via script element on page in test - use BINDING.bind_static_method instead of INTERNAL where possible - better .d.ts exports - formatted html files - renamed modules.ts to imports.ts which makes more sense - improved error propagation - renamed __initializeImportsAndExports - delayed exit and stdout flush

…d AOT. (#61627) If a profile contained a method like List<AnInst>:.ctor () and the 'profile-only' option was used, the fully shared version of the method needs to be added to the aot image.

…rrent namespace for evaluation (#61252) * Using current namespace as the default place to serach for the resolved class. * Add tests for static class, static fields and pausing in async method. * Added tests for class evaluation. * Fixing support to the current namespace and adding tests for it * Assuing that we search within the current assembly first. Removed tests that fail in Consol App. * Remove a test-duplicate that was not testing static class or static fields. * Fixing indentation. * Refixing indentation. * Refix indentations again. * Applied the advice about adding new blank lines. * Changed the current assembly check. * Extracting the check from the loop. One time check is enough. * Simplifying multiple test cases into one call. * Using local function as per review suggestion. * Added test that was skipped by mistake. * Added looking for the namespace in all assemblies because there is a chance it will be located out of the current assembly. * Extracting value based on the current frame, not the top of stack location. * Test for classes evaluated from different frames. * Fixing indentation and spaces. * Applied review comments for values evaluation. * Compressed two tests into one with MemberData. * Added test case of type without namespace (failing). * Addressed Ankit advices from the review. Co-authored-by: DESKTOP-GEPIA6N\Thays <thaystg@gmail.com>

* switch from FileStream to RandomAccess * use Array.MaxLength as a limit for File.ReadAllBytes and fix an edge case bug for files: Array.MaxLength < Length < int.MaxValue * there is no gain of using FileOptions.SequentialScan on Unix, as it requires an additional sys call Co-authored-by: Dan Moseley <danmose@microsoft.com>

* Extend RegexCharClass.Canonicalize range inversion optimization There's a simple optimization in RegexCharClass.Canonicalize that was added in .NET 5, with the goal of taking a set that's made up of exactly two ranges and seeing whether those ranges were leaving out exactly one character. If they were, the set can instead be rewritten as that character negated, which is a normalized form used downstream and optimized. We can extend this normalization ever so slightly to be for two ranges separated not just be a single character but by more than that as well. * Update TODO comment * Add some more reduction tests

* Remove some unnecessary slicing from generated Regex code When we're outputting code to match a "multi" (a sequence of multiple characters), we're currently issuing a Slice for the known tracked offset even if that offset is 0. We can skip that nop. * Address PR feedback

* Ignore missing data result * Move result = false only if error Code is other than 2/3

Co-authored-by: kronic <kronic@softland.ru>

…61691) During code review of my latest PR Bruce raised the concern that hard-coding public key values and version ID for the xunit.core reference will cause enormous maintenance pain if we decide to upgrade to a newer version of the module in the future. As Jeremy verified that the metadata is not really needed, I'm deleting it from all tests I switched over last week. Thanks Tomas

…tem.Runtime.CompilerServices.Unsafe (#61704)

…tChar (#61490) * Factor out and improve the vectorization of RegexInterpreter.FindFirstChar This change started with the "simple" goal of factoring out the FindFirstChar logic from RegexInterpreter and consuming it in SymbolicRegexMatcher. The existing engines use FindFirstChar to quickly skip ahead to the next location that might possibly match, at which point they fall back to analyzing the whole pattern at that location. SymbolicRegexMatcher (used by RegexOptions.NonBacktracking) had its own implementation for this, which it used any time it entered a start state. This required non-trivial additional code to maintain, and there's no good reason it should be separate from the other engines. However, what started out as a simple change grew due to regressions that resulted from differences in the implementations. In particular, SymbolicRegexMatcher already works off of precomputed equivalence tables for casing, which gives it very different characteristics in this regard from the existing engines. For example, SymbolicRegexMatcher's existing "skip ahead to the next possible match start location" logic already evaluated all the characters that could possibly start a match, which included variations of the same character when using IgnoreCase, but the existing RegexInterpreter logic didn't. That discrepancy then results in a significant IgnoreCase regression for NonBacktracking due to losing the ability to use a vectorized search for the next starting location. We already plan to shift the existing engines over to a plan where all of these equivalences are computed at construction time rather than using ToLower at both construction time and match time, so this PR takes some steps in that direction, doing so for most of ASCII. This has added some temporary cruft, which we'll be able to delete once we fully shift the implementations over (which we should do in the near future). Another difference was SymbolicRegexMatcher was enabling use of IndexOfAny for up to 5 characters, whereas RegexOptions.Compiled was only doing up to 3 characters, and RegexInterpreter wasn't doing for any number. The PR now uses 5 everywhere. However, the more characters involved, the more overhead there is to IndexOfAny, and for some inputs, the higher the chances are that IndexOfAny will find a match sooner, which means its overhead compounds more. To help with that, we now not only compute the possible characters that might match at the beginning of the pattern, but also characters that might match at a fixed offset from the beginning of the pattern (e.g. in \d{3}-\d{2}-\d{4}, it will find the '-' at offset 3 and be able to vectorize a search for that and then back off by the relevant distance. That then also means we might end up with multiple sets to choose to search for, and this PR borrows an idea from Rust, which is to use some rough frequency analysis to determine which set should be targeted. It's not perfect, and we can update the texts use to seed the analysis (right now I based it primarily on *.cs files in dotnet/runtime and some Project Gutenberg texts), but it's good enough for these purposes for now. We'd previously switched to using IndexOf for a case-sensitive prefix string, but still were using Boyer-Moore for case-insensitive. Now that we're able to also vectorize a search for case-insensitive values (right now just ASCII letter, but that'll be fixed soon), we can just get rid of Boyer-Moore entirely. This saves all the costs to do with constructing the Boyer-Moore tables and also avoids having to generate the Boyer-Moore implementations in RegexOptions.Compiled and the source generator. The casing change also defeated some other optimizations already present. For example, in .NET 5 we added an optimization whereby an alternation like `abcef|abcgh` would be transformed into `abc(?:ef|gh)`, and that would apply whether case-sensitive or case-insensitive. But by transforming the expression at construction now for case-insensitive into `[Aa][Bb][Cc][Ee][Ff]|[Aa][Bb][Cc][Gg][Hh]`, that optimization was defeated. I've added a new optimization pass for alternations that will detect common prefixes even if they're sets. The casing change also revealed some cosmetic issues. As part of the change, when we encounter a "multi" (a multi-character string in the pattern), we convert that single case-insensitive RegexNode to instead be one case-sensitive RegexNode per character, with a set for all the equivalent characters that can match. This then defeats some of the nice formatting we had for multis in the source generator, so as part of this change, the source generator has been augmented to output nicer code for concatenations. And because sets like [Ee] are now way more common (since e.g. a case-insensitive 'e' will be transformed into such a set), we also special-case that in both the source generator and RegexOptions.Compiled, to spit out the equivalent of `(c | 0x20) == 'e'` rather than `(c == 'E'| c == 'e')`. Along the way, I cleaned up a few things as well, such as passing around a CultureInfo more rather than repeatedly calling CultureInfo.CurrentCulture, using CollectionsMarshal.GetValueRefOrAddDefault on a hot path to do with interning strings in a lookup table, tweaking SymbolicRegexRunnerFactory's Runner to itself be generic to avoid an extra layer of virtual dispatch per operation, and cleaning up code / comments in SymbolicRegexMatcher along the way. For the most part the purpose of the change wasn't to improve perf, and in fact I was willing to accept some regressions in the name of consolidation. There are a few regressions here, mostly small, and mostly for cases where we're simply paying an overhead for vectorization, e.g. where the current location is fine to match, or where the target character being searched for is very frequent. Overall, though, there are some substantial improvements. * Fix missing condition in RegexCompiler * Try to fix mono failures and address comment feedback * Delete more now dead code * Fix dead code emitting after refactoring Previously return statements while emitting anchors were short-circuiting the rest of the emitting code, but when I moved that code into a helper, the returns stopped having that impact, such that we'd end up emitting a return statement and then emit dead code after it. Fix it. * Remove some now dead code

* re-enable optimization when a token is replaced by corresponding type handle * compute "isNested" ony when we need it. * no need to make more than 2 attempts at reading

)

…tography

* Add support for custom container for Linux Arm64 Update platform_matrix.yml to allow custom containers for Arm64 Linux runs. This is required for our performance runs as we need a newer version of the python runtime. This change follows the same pattern as https://github.com/dotnet/runtime/pull/59202/files * Update perf_slow.yml with correct container values

Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com> Co-authored-by: Jan Kotas <jkotas@microsoft.com> Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com> Co-authored-by: Tomas <trylek@microsoft.com>

Global usings no longer means "global breaking" after RC1.

* Temp change to disable align loop * download specific artifacts to squash: * Upload .dasm files * fix the build id * Add ci_run and retainOnlyTopFiles * Rename ci_run to retainOnlyTop * Disable struct promo to test asmdiff * Revert "Disable struct promo to test asmdiff" This reverts commit 3ef7adb. * fix the parameter retainOnlyTopFiles * add missing - * Revert "Temp change to disable align loop" This reverts commit b1de5c4. * Revert "download specific artifacts" This reverts commit db3de57. * Review comments

…implify the api (#61392) * Make SMonoSdbHelper part of the execution context

* Hide align behind a jmp fix the alignBytesRemoved Some fixes and working model Some fixes and redesign Some more fixes more fixes fix Add the check for fgFirstBB misc changes code cleanup + JitHideAlignBehindJmp switch validatePadding only if align are before the loop IG More cleanup, remove commented code jit format * Fix a problem where curIG==0 and loop might be emitted in curIG, adjust the targetIG to prevIG Add IGF_REMOVED_ALIGN flag for special scenarios * Add stress mode to emit int3 for xarch * Add stress mode to emit bkpt for arm64 * Add a loop align instruction placement phase * review comments * Change from unsigned short to unsigned * review comments around cleanup * emitForceNewIG * Remove emitPrevIG * Revert change to forceNewIG for align instruction * Use loopAlignCandidates * Use loopHeadIG reference * jit format * Remove unneeded method * Misc changes * Review feedback * Do not include align behind Jmp in PerfScore calculation * jit format and fix a bug * fix the loopCandidates == 0 scenario * Add unmarkLoopAlign(), add check for fgFirstBB * merge conflict fix * Add missing } * Grammar nit Co-authored-by: Bruce Forstall <brucefo@microsoft.com> Co-authored-by: Bruce Forstall <brucefo@microsoft.com>

This change allows devs to manually kick off full test runs on the configurations that only execute smoke tests per PR. To kick things off, you can run /azp run runtime-manual

* Convert Crypto P/Invokes to GeneratedDllImport.

…erviceProcess.ServiceController (#61741)

- Do not printout exceptions from failing task tests. - Remove default MONO_LOG_MASK=gc from debug configuration.

…leted. (#60214) * FileSystemEntry.Unix: ensure attributes are available when file is deleted. When the file no longer exists, we create attributes based on what we know. The test for this was passing because it cached the attributes before the item was deleted due to enumerating with skipping FileAttributes.Hidden. * GetLength: fix reading from uninitialized cache.

* FileStatus.Unix/Process.Unix: align implementation. Process: remove the user identity caching and extend the logic to avoid retrieving the identity in most cases by checking if all x-bits are set or not set. FileStatus: use same group check as Process. FileStatus: cache the read only flag instead of caching the identity.

* In the JIT, add support for dumping the precise debug info out through an environment variable `DOTNET_JitDumpPreciseDebugInfoFile` in a simple JSON format. This is a stopgap until we expose the extra information through ETW events. * In dotnet-pgo, add an argument --precise-debug-info-file which can point to the file produced by the JIT. When used, dotnet-pgo will get native<->IL mappings from this file instead of through ETW events. * In dotnet-pgo, add support for attributing samples to inlinees when that information is present. This changes the attribution process a bit: previously, we would group all LBR data/samples and then construct the profile from all the data. We now do it in a more streaming way where there is a SampleCorrelator that can handle individual LBR records and individual samples. * In dotnet-pgo, add an argument --dump-worst-overlap-graphs-to which can be used in the compare-mibc command to dump out a .dot file containing the flow graph of the methods with the worst overlap measures, and showing the relative weight count on each basic block and edge for the two profiles being compared. This is particular useful to find out where we are producing incorrect debug mappings, by comparing spgo.mibc and instrumented.mibc files.

* Reduce atomic single char lazy loops * Add single char lazy loop support to simplified code gen

These checks appears to be redundant.

This is a follow up PR for #61590. It includes: - additional UnsupportedOSPlatform annotations for some System.Diagnostics.Process APIs throwing PNSE on iOS/tvOS (they started doing so after excluding some managed logic around librpoc ) - fixing a bit ugly workaround for CS0649 (see https://github.com/dotnet/runtime/pull/61590/files#r749525127) - used a local pragma in the ThreadInfo class. - skipping the respective S.D.P. tests ( it will address [iOS/tvOS] System.Diagnostics.Tests.ProcessTests.TestGetProcesses fails on devices #60588 as well)

…guration.Binder` (#57418) * Annotate * Update Microsoft.Extensions.Configuration.Binder.csproj * DisableImplicitAssemblyReferences * Nullablity fix * configureOptions are non-nullable * Revert "configureOptions are non-nullable" This reverts commit 5dac613. * NetCoreAppMinimum TFM * Revert non-nullable changes * Remove not needed project references * Refactor while * Revert non-nullable changes * Fix up csproj files to build for necessary TFMs Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>

…61799)

…1750) I believe that despite its singular nature this is worth a separate review to make sure we're in agreement regarding the conversion principles. There are several things worth noting here: 1) According to the current plan we're continuously documenting in #54512 we want to remove command-line parameters from all test entrypoints. Variant tests driven by command-line parameters should be turned into multiple test cases marked with separate [Fact] attributes. 2) For ilproj tests, to facilitate local debugging, our current plan is to keep them runnable as standalone executables. This implies that ilproj tests comprising several [Fact] test entrypoints require a new entrypoint that just calls into the individual test cases. 3) The Roslyn-generated merged wrapper for the tests won't care about the "composite" main (that is for local debugging only), it will directly identify and use the individual test cases marked with [Fact] attributes. 4) In accordance with this scheme, such composite ILPROJ tests are specific in not having their entrypoint method itself marked with the [Fact] attribute. 5) Funnily enough this example nicely demonstrates the implied cleanup - the entire command-line machinery is only used for a handwritten switch to choose one of the three variants; moreover we only exercised two out of the three variants, possibly due to an authoring bug when creating the variant test, potentially caused by previous complexity of such endeavor. Thanks Tomas

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge upstream #2

Merge upstream #2

Commits on Nov 2, 2021

Commits on Nov 3, 2021

Commits on Nov 4, 2021

Commits on Nov 5, 2021

Commits on Nov 6, 2021

Commits on Nov 7, 2021

Commits on Nov 8, 2021

Commits on Nov 9, 2021

Commits on Nov 10, 2021

Commits on Nov 11, 2021

Commits on Nov 12, 2021

Commits on Nov 13, 2021

Commits on Nov 14, 2021

Commits on Nov 15, 2021

Commits on Nov 16, 2021

Commits on Nov 17, 2021

Commits on Nov 18, 2021