Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge upstream #2

Merged
merged 637 commits into from
Nov 18, 2021
Merged

Merge upstream #2

merged 637 commits into from
Nov 18, 2021

Conversation

deeprobin
Copy link
Owner

No description provided.

LoopedBard3 and others added 30 commits November 1, 2021 21:48
* Fix artifact match string.

* Completed first attempt for a complete Maui SOD run. Added ios_scenarios tests, setup file copies, uploads, and downloads; and setup perf branch to use as partner testing branch.

* Reset performance-setup.ps1 file to use main perf branch.

* Reenable yml jobs disabled for testing.
…pecified (#60988)

* diable sending NT Authority if specific trust was specified

* feedback from review

* add missing file

* Apply suggestions from code review

Co-authored-by: Cory Nelson <phrosty@gmail.com>

* one more style update

Co-authored-by: Cory Nelson <phrosty@gmail.com>
* All of System.Security.Cryptography.Primitives has been moved
* AsymmericKeyExchangeFormatter moved from S.S.C.Algorithms
* This just starts the process to keep the infrastructure pieces separate from code layout pieces.
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
* Remove "block" from VNApplySelectorsAssign

It was unused and all the callers passed the current block anyway.

Also some renaming to make Assign and AssignTypeCoerce methods consistent.

* Improve the readability of VNForMapStore

By naming parameters properly.

* Improve the readability of VNForMapSelect

By naming parameters properly.

* Pull ASG numbering out of fgValueNumberTree

In my upcoming substantive changes to this code the unreasonable level
of nesting started to significantly degrade the readability of code.

So, move it out of the main numbering function. The result is still
pretty huge and could be split up further, but this will do for now.

* Refactor VNApplySelectors

Make it use a loop instead of recursion.

Rename parameters to match the "Assign" variants.

Add standard header comments.
This change enables device runs on CI by building each test app on the helix instance it was deployed to. In past attempts, we looked at enhancing what takes place on the build machine via build tricks, compressing / cleaning up files early, etc and we could not overcome the need for excessively long timeouts and far more disk space.

The change also adopts the patterns established in the wasm test build, giving us the opportunity to support different scenario runs as well as workloads testing in the future.
* Remove quasi passwords

* oops
* Fix gcc warnings during mono linux-x64 build

main with Debug configuration: 822 warnings - http://sprunge.us/2GzrDE
PR with Debug configuration: 3 warnings related to deprecated sys/sysctl.h includes - http://sprunge.us/JuyA3K

after fixing Debug warnings, there were 13 additional warnings in Release configuration:
http://sprunge.us/PJCivP

PR with Release configuration: (same) 3 warnings - http://sprunge.us/NwKHNE

* Address CR feedback
…code sent (#59313)

HTTP2 loopback server WaitForCancellationAsync should validate error code sent
…xtensions.DependencyModel` is empy (#60842)

* Setting framework name as nullable.

* Setting path to nullable.

* Setting runtime to nullable.

* Setting path to nullable.

* Setting Target Name, RuntimeTargetEntrySub Path and TargetLibrary Name as nullable (string) properties.

* Adding missing line of code.

* Adding new string resource and throwing argument exception on null/empty values.
Adding unit tests to check for specific null/empty value cases

* Removing Argument from string resource.
Updating PR according to comments

* Updating unit tests to use FormatException

* Rollback of ArgumentExceptions in DependencyModels

* Added missing usings

* Updating all exceptions to FormatException

* Moving FormatException while reading target name

* Updating PR with new comments.

* Update src/libraries/Microsoft.Extensions.DependencyModel/src/DependencyContextJsonReader.cs
These packages are unused as described in #60794

Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>
This adds support for setting per-project optimization flags, and sets them for these projects so they don't OOM.

Microsoft.Extensions.Logging.Generators.Roslyn3.11.Tests
Microsoft.Extensions.Logging.Generators.Roslyn4.0.Tests
System.Text.Json.SourceGeneration.Roslyn3.11.Unit.Tests
System.Text.Json.SourceGeneration.Roslyn4.0.Unit.Tests
System.Text.RegularExpressions.Generators.Tests
Some individual ones are disabled due to
#58226, and #60899 .

Additionally, if a AOT build fails, then on linux it dumps the last few lines from dmesg, to help identify it was an oom-kill.

Fixes #51961 .

Co-authored-by: Larry Ewing <lewing@microsoft.com>
* Disable couple of randomly failing PerformanceCounter tests

* fix: disabled the wrong test
…GetField` method (#53645)

* Reorder checks for TypeBuilder.GetConstructor()

* Reorder checks for TypeBuilder.GetField()

* Adjust unit test to reordered TypeBuilder checks

* Use 'not' keyword for pattern matching

* Apply the same Type checks for Mono

GetConstructor and GetField had the same incorrect checks order
comparing to GetMethod as in the CoreCLR.
To keep the consistent solution, Mono has also been adjusted.

* Check whether a declaring type isn't type's generic definition

* Make generic type if contains generic parameters

* Suppres UnrecognizedReflectionPattern for GetField and GetConstructor

* Move GetMethodType validation method at original order

* Test GetConstructor against throwing an exception

* Fix: copy-paste error  'type' -> 'constructor'

* Align Mono and CoreCLR with nameof for GetConstructor

* Use type for argument exception and test against type parameter

* Get tests passing on both coreclr and mono.

Ensure checks are consistent across GetConstructor, GetField, and GetMethod for both coreclr and mono.

* Fix mono exception messages to use SR resources.

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
Co-authored-by: Elinor Fung <elfung@microsoft.com>
* Fix refcount when a composite r2r image is loaded from a bundle

* PR feedback - better comments.
* Add TFM and version support query to marshaller interface.

* If any marshaller fails to support the current tfm version, fallback
to using the built-in DllImport attribute.

* Add tests for fallback generation on downlevel or generation if marshallers
  are supported on the target framework.

* Convert CreateCompilationWithReferenceAssemblies to CreateCompilation overload.

Co-authored-by: Elinor Fung <elfung@microsoft.com>
* added configSrc - path to MonoConfig json file
* added Module.onConfigLoaded callback
* added Module.onDotNetReady replacing MonoConfig.loaded_cb
* removed loaded_cb, assembly_list, runtime_assets, runtime_asset_sources from MonoConfig
* simplified all sample and test projects
* renamed no_global_exports to disableDotNet6Compatibility
* implement default Module.preInit and Module.onRuntimeInitialized which could be overriden by user code (like Blazor)

* re-enable the interp lane
Fixes three warnings:
```sh
  [ 56%] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/utils/mono-mmap.c.o
  In file included from /runtime/src/mono/mono/utils/mono-mmap.c:23:
  /usr/include/x86_64-linux-gnu/sys/sysctl.h:21:2: warning: #warning "The <sys/sysctl.h> header is deprecated and will be removed." [-Wcpp]
     21 | #warning "The <sys/sysctl.h> header is deprecated and will be removed."
        |  ^~~~~~~
# snip
  [ 59%] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/utils/mono-time.c.o
  In file included from /runtime/src/mono/mono/utils/mono-proclib.c:42:
  /usr/include/x86_64-linux-gnu/sys/sysctl.h:21:2: warning: #warning "The <sys/sysctl.h> header is deprecated and will be removed." [-Wcpp]
     21 | #warning "The <sys/sysctl.h> header is deprecated and will be removed."
        |  ^~~~~~~
  [ 59%] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/utils/mono-uri.c.o
  [ 59%] Building C object mono/mini/CMakeFiles/monosgen-objects.dir/__/utils/mono-poll.c.o
  In file included from /runtime/src/mono/mono/utils/mono-time.c:107:
  /usr/include/x86_64-linux-gnu/sys/sysctl.h:21:2: warning: #warning "The <sys/sysctl.h> header is deprecated and will be removed." [-Wcpp]
     21 | #warning "The <sys/sysctl.h> header is deprecated and will be removed."
        |  ^~~~~~~
```

With this PR: 0 warnings - http://sprunge.us/96kyq9

Like in src/libraries/Native, skip linux from sys/sysctl.h check. For other systems, `check_include_files` (instead of `check_include_file` (singular)) when a header depends on another one.
 
Also, simplified few conditions: "is linux" check supersedes "is android".
agocke and others added 28 commits November 17, 2021 12:43
* Add support for custom container for Linux Arm64

Update platform_matrix.yml to allow custom containers for Arm64 Linux runs.
This is required for our performance runs as we need a newer version of the
python runtime. This change follows the same pattern as https://github.com/dotnet/runtime/pull/59202/files

* Update perf_slow.yml with correct container values
Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>
Co-authored-by: Tomas <trylek@microsoft.com>
Global usings no longer means "global breaking" after RC1.
* Temp change to disable align loop

* download specific artifacts

to squash:

* Upload .dasm files

* fix the build id

* Add ci_run and retainOnlyTopFiles

* Rename ci_run to retainOnlyTop

* Disable struct promo to test asmdiff

* Revert "Disable struct promo to test asmdiff"

This reverts commit 3ef7adb.

* fix the parameter retainOnlyTopFiles

* add missing -

* Revert "Temp change to disable align loop"

This reverts commit b1de5c4.

* Revert "download specific artifacts"

This reverts commit db3de57.

* Review comments
…implify the api (#61392)

* Make SMonoSdbHelper part of the execution context
* Hide align behind a jmp

fix the alignBytesRemoved

Some fixes and working model

Some fixes and redesign

Some more fixes

more fixes

fix

Add the check  for fgFirstBB

misc changes

code cleanup + JitHideAlignBehindJmp switch

validatePadding only if align are before the loop IG

More cleanup, remove commented code

jit format

* Fix a problem where curIG==0 and loop might be emitted in curIG, adjust the targetIG to prevIG

Add IGF_REMOVED_ALIGN flag for special scenarios

* Add stress mode to emit int3 for xarch

* Add stress mode to emit bkpt for arm64

* Add a loop align instruction placement phase

* review comments

* Change from unsigned short to unsigned

* review comments around cleanup

* emitForceNewIG

* Remove emitPrevIG

* Revert change to forceNewIG for align instruction

* Use loopAlignCandidates

* Use loopHeadIG reference

* jit format

* Remove unneeded method

* Misc changes

* Review feedback

* Do not include align behind Jmp in PerfScore calculation

* jit format and fix a bug

* fix the loopCandidates == 0 scenario

* Add unmarkLoopAlign(), add check for fgFirstBB

* merge conflict fix

* Add missing }

* Grammar nit

Co-authored-by: Bruce Forstall <brucefo@microsoft.com>

Co-authored-by: Bruce Forstall <brucefo@microsoft.com>
This change allows devs to manually kick off full test runs on the configurations that only execute smoke tests per PR. 

To kick things off, you can run /azp run runtime-manual
* Convert Crypto P/Invokes to GeneratedDllImport.
- Do not printout exceptions from failing task tests.
- Remove default MONO_LOG_MASK=gc from debug configuration.
…leted. (#60214)

* FileSystemEntry.Unix: ensure attributes are available when file is deleted.

When the file no longer exists, we create attributes based on what we know.

The test for this was passing because it cached the attributes before the
item was deleted due to enumerating with skipping FileAttributes.Hidden.

* GetLength: fix reading from uninitialized cache.
* FileStatus.Unix/Process.Unix: align implementation.

Process: remove the user identity caching and extend the logic
to avoid retrieving the identity in most cases by checking
if all x-bits are set or not set.

FileStatus: use same group check as Process.

FileStatus: cache the read only flag instead of caching the
identity.
* In the JIT, add support for dumping the precise debug info out through
  an environment variable `DOTNET_JitDumpPreciseDebugInfoFile` in a
  simple JSON format. This is a stopgap until we expose the extra
  information through ETW events.

* In dotnet-pgo, add an argument --precise-debug-info-file which can
  point to the file produced by the JIT. When used, dotnet-pgo will get
  native<->IL mappings from this file instead of through ETW events.

* In dotnet-pgo, add support for attributing samples to inlinees when
  that information is present. This changes the attribution process a
  bit: previously, we would group all LBR data/samples and then
  construct the profile from all the data. We now do it in a more
  streaming way where there is a SampleCorrelator that can handle
  individual LBR records and individual samples.

* In dotnet-pgo, add an argument --dump-worst-overlap-graphs-to which
  can be used in the compare-mibc command to dump out a .dot file
  containing the flow graph of the methods with the worst overlap
  measures, and showing the relative weight count on each basic block
  and edge for the two profiles being compared. This is particular
  useful to find out where we are producing incorrect debug mappings, by
  comparing spgo.mibc and instrumented.mibc files.
* Reduce atomic single char lazy loops

* Add single char lazy loop support to simplified code gen
These checks appears to be redundant.
This is a follow up PR for #61590.

It includes:

 - additional UnsupportedOSPlatform annotations for some System.Diagnostics.Process APIs throwing PNSE on iOS/tvOS (they started doing so after excluding some managed logic around librpoc )

 - fixing a bit ugly workaround for CS0649 (see https://github.com/dotnet/runtime/pull/61590/files#r749525127) - used a local pragma in the ThreadInfo class.

 - skipping the respective S.D.P. tests ( it will address [iOS/tvOS] System.Diagnostics.Tests.ProcessTests.TestGetProcesses fails on devices #60588 as well)
…guration.Binder` (#57418)

* Annotate

* Update Microsoft.Extensions.Configuration.Binder.csproj

* DisableImplicitAssemblyReferences

* Nullablity fix

* configureOptions are non-nullable

* Revert "configureOptions are non-nullable"

This reverts commit 5dac613.

* NetCoreAppMinimum TFM

* Revert non-nullable changes

* Remove not needed project references

* Refactor while

* Revert non-nullable changes

* Fix up csproj files to build for necessary TFMs

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
…1750)

I believe that despite its singular nature this is worth a separate
review to make sure we're in agreement regarding the conversion
principles. There are several things worth noting here:

1) According to the current plan we're continuously documenting in
#54512 we want to remove
command-line parameters from all test entrypoints. Variant tests
driven by command-line parameters should be turned into multiple
test cases marked with separate [Fact] attributes.

2) For ilproj tests, to facilitate local debugging, our current plan
is to keep them runnable as standalone executables. This implies that
ilproj tests comprising several [Fact] test entrypoints require a
new entrypoint that just calls into the individual test cases.

3) The Roslyn-generated merged wrapper for the tests won't care about
the "composite" main (that is for local debugging only), it will
directly identify and use the individual test cases marked with
[Fact] attributes.

4) In accordance with this scheme, such composite ILPROJ tests are
specific in not having their entrypoint method itself marked with
the [Fact] attribute.

5) Funnily enough this example nicely demonstrates the implied
cleanup - the entire command-line machinery is only used for a
handwritten switch to choose one of the three variants; moreover
we only exercised two out of the three variants, possibly due to
an authoring bug when creating the variant test, potentially caused
by previous complexity of such endeavor.

Thanks

Tomas
@deeprobin deeprobin merged commit 4625a20 into deeprobin:issue-20234 Nov 18, 2021
deeprobin pushed a commit that referenced this pull request Jun 23, 2022
* Initial implementation for contract customization

fix build errors

Move converter rooting to DefaultJsonTypeInfoResolver so that it can be used standalone

Fix ConfigurationList.IsReadOnly

Minor refactorings (#1)

* Makes the following changes:

* Move singleton initialization for DefaultTypeInfoResolver behind a static property.
* Consolidate JsonSerializerContext & IJsonTypeInfoResolver values to a single field.
* Move reflection fallback logic away from JsonSerializerContext and into JsonSerializerOptions

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializerOptions.cs

* remove testing of removed field

Simplify the JsonTypeInfo.CreateObject implemenetation (#2)

* Simplify the JsonTypeInfo.CreateObject implemenetation

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs

* Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs

Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>

Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>

Tests and fixes for JsonTypeInfoKind.None

TypeInfo type mismatch tests

Allow setting NumberHandling on JsonTypeInfoKind.None

test resolver returning wrong type of options

JsonTypeInfo/JsonPropertyInfo mutability tests

rename test file

Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver (#3)

* Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver

* address feedback

Add simple test for using JsonTypeInfo<T> with APIs directly taking it

fix and tests for untyped/typed CreateObject

uncomment test cases, remove todo

More tests and tiny fixes

Add a JsonTypeInfoResolver.Combine test for JsonSerializerContext (#4)

* Fix JsonTypeInfoResolver.Combine for JsonSerializerContext

* Break up failing test

Fix simple scenarios for combining contexts (#6)

* Fix simple scenarios for combining contexts

* feedback

JsonSerializerContext combine test with different camel casing

Remove unneeded virtual calls & branching when accessing Get & Set delegates (#7)

JsonPropertyInfo tests everything minus ShouldSerialize & NumberHandling

Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs

Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs

throw InvalidOperationException rather than ArgumentNullException for source gen when PropertyInfo.Name is assigned through JsonPropertyInfoValues

tests for duplicated property names and JsonPropertyInfo.NumberHandling

Add tests for NumberHandling and failing tests for ShouldSerialize

disable the failing test and add extra checks

disable remainder of the failing ShouldSerialize tests, fix working one

Fix ShouldSerialize and IgnoreCondition interop

Add failing tests for CreateObject + parametrized constructors

Fix CreateObject support for JsonConstructor types (#10)

* Fix CreateObject support for JsonConstructor types

* address feedback

Make contexts more combinator friendly (#9)

* Make contexts more combinator friendly

* remove converter cache

* redesign test to account for JsonConstructorAttribute

* Combine unit tests

* address feedback

* Add acceptance tests for DataContract attributes & Specified pattern (#11)

* Add private field serialization acceptance test (#13)

* tests, PR feedback (#14)

* PR feedback and extra tests

* Shorten class name, remove incorrect check (not true for polimorphic cases)

* Make parameter matching for custom properties map property Name with parameter (#16)

* Test static initialization with JsonTypeInfo (#17)

* Fix test failures and proper fix this time (#18)

* Fix test failures and proper fix this time

* reinstate ActiveIssueAttribute

* PR feedback and adjust couple of tests which don't set TypeInfoResolver

* fix IAsyncEnumerable tests

* Lock JsonSerializerOptions in JsonTypeInfo.EnsureConfigured()

Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>

Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.