Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runtime ingestion #73

Merged
merged 756 commits into from
Aug 28, 2020
Merged

Runtime ingestion #73

merged 756 commits into from
Aug 28, 2020

Conversation

joperezr
Copy link
Member

FYI: @jkotas merging recent changes gone into dotnet/runtime in order to set up runtimelab official builds and pushing packages to the experimental feed.

bbartels and others added 30 commits August 14, 2020 11:53
There are 2 flaky HTTP/2 tests verifying multiple connections feature which are randomly failing on CI, but not locally.
- Http2_MultipleConnectionsEnabled_ConnectionLimitNotReached_ConcurrentRequestsSuccessfullyHandled
- Http2_MultipleConnectionsEnabled_IdleConnectionTimeoutExpired_ConnectionRemovedAndNewCreated

It seems the failure is caused by not reading the request body. In current implementation, `Http2LoopbackServer` read only HEADERS frame via `ReadRequestHeaderAsync` and then immediately sends response. However, the client firstly completely sends headers and body and only then starting reading a response. Thus, it seems to get blocked sometimes if the server didn't read the full body. This PR fixes this by calling `ReadAndParseRequestHeaderAsync` instead of `ReadRequestHeaderAsync`.

Fixes #40436
Fixes #40115
…are retrieved the same way (#40641)

* Ensure FileStatus and FileSystemEntry IsHidden attribute is retrieved the same way

* Add missing check in public property FileSystemEntry.IsHidden. Address PR suggestions.

* Add tests for Hidden and ReadOnly attribute check for each platform. Split existing Skip attribute test into different platforms.

* Use _initialAttributes instead of Attributes for IsHidden. Add comment on top.

Co-authored-by: carlossanlop <carlossanlop@users.noreply.github.com>
* Move content type verification to a later stage

Fix #38713

* Rename ValidateContent to ValidateContentType

Fix #38713

* Get rid of content type check

* Add newline at end of the file

* Rename TestValidMediaTypes to TestVariousMediaTypes
…TINGS frame (#40779)

* Set MaxConcurrentStreams = 100 on new Http2Connection until first SETTINGS frame

* Update src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http2Connection.cs

Co-authored-by: Chris Ross <Tratcher@Outlook.com>

* - Setting infinite MaxConcurrentStreams fixed
- Test verifying new connection request queueing added

* More tests

* - const case fixed
- Outerloop attributed added

* ExtraStreams case fixed

Co-authored-by: Chris Ross <Tratcher@Outlook.com>
…1 SDK (#40796)

* Disable net analyzers from the SDK and use them from package reference instead

* Fix build with new SDK and bump net analyzer version

* PR Feedback
* Fix ilasm, ildasm and crossgen launch from PATH

When trying to launch ilasm, ildasm, crossgen or any tool using
PAL_Initialize that are on the current PATH just by their file names,
they were crashing with SIGSEGV.
The problem was that the current exe path was extracted incorrectly
in that case.

This change fixes the problem using the same methods that corerun
uses to figure out the current exe path.

* Add missing FreeBSD headers

* Reflect PR feedback

* Update src/coreclr/src/pal/src/init/pal.cpp

* Fix typo in Solaris version of the code

Co-authored-by: Jan Kotas <jkotas@microsoft.com>
* Fix a deadlock in CounterGroup and EventSource

* add more comments

* Add some more comments

* PR feedback

* Add comments
* fixing all config build

* address feedback
* Porting changes from dotnet/extensions#536

* Correcting reference

* Avoiding wrapping exception as there may be other reasons why closing the generic type fails other than constraint failure
…ullable structs (#40824)

* Wrap non-nullable struct converters in NullableOfT converter when
they explicitly handle nullable structs

* Throw exception instead of being nice

* Improve exception message
Improve validation of tail calls that are not tail-prefixed in the IL
but are marked as such because of TailCallStress. We now do the same
correctness validation in morph for such tail calls as we do for
implicit tail calls. That blocks tail calls when we have address-taken
locals, struct promoted params, and pinned vars.

Fixes #39398.
Fixes #39309.
Fixes #38892.
Fixes #38889.
Fixes #38887.
Fixes #37117.
Fixes #8017.
…d apply Unit and Name converter (#40825)

* Move Font, Image, and Margins converters to System.Drawing.Common

* Add FontUnit converter and FontName converter to Font properties

* Fix Unix tests

* Sort Compile items
* Nullable: System.Xml, part 6 (XSLT minus XSLT.Runtime)

* change Evaluate to take non-nullable

* apply pr feedback

* fix nullability merge conflicts

* apply feedback
When running the new nullable constructor analysis on runtime we
discovered what appears to be a missing annotation in
`DataTableReaderListener`.

Note: the second `if` check in the constructor seems like dead code.
It's checking for the field to be non-null before ever setting the field
to a value. Probably could be deleted.
…e #37583") (#40753)

* Revert " Use clonefile for CopyFile, if available (dotnet/corefx#37583)"

This reverts commit 02ba75a.

* Add unit test

* Disable tests failing in browser-wasm
* Change Assembly.GetFile(s) to throw when in-memory

This is technically a breaking change, although a minor one, as the
previous behavior also threw an exception. The previous exception was
an IOException and this is an InvalidOperationException, so if users
were catching the previous exception they would not succeed now.

Fixes #40154

* Don't specify exception for GetFile when passed null

* Skip ImplementedInterfaces test on Mono

* Fix tests for running in memory

Co-authored-by: Dan Moseley <danmose@microsoft.com>
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
Add two build tasks:
1. DownloadTimeZoneData 
2. CreateWasmBundle (given directory bundle files into a data archive)
* Limit Debugger.Log() calls in EventSource

Fixing #40799 

Previously we would send a message via Debugger.Log() for event call to ReportOutOfBandMessage().
Now we only do it the first 16 times, then all errors after that are silent. This should alleviate performance
overhead from a scenarios with a large number of errors.

I made this change quickly via github to help out Bing. I have never compiled or tested it prior to the PR
but fingers crossed it appears simple enough that it has a decent chance to work : )

* Remove trailing whitespace

Co-authored-by: David Mason <davmason@microsoft.com>
* Workaround for WSL1 process IO redirection bug

* Update src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs

Co-authored-by: Adeel Mujahid <adeelbm@outlook.com>

Co-authored-by: Adeel Mujahid <adeelbm@outlook.com>
* Make suspend the default port configuration
* ignore empty port addresses

* Appease MSVC
maryamariyan and others added 24 commits August 27, 2020 10:33
Ensure signal handlers are setup to invalidate cached values
for window size.
…otations (#41382)

* Fix null check in S.Diagnostics.Contract regressed by nullability annotations

* fix product per Jan's recommendation
…gurecompiler.cmake (#41435)

The latter still specified 10.12 but .NET Core 3+ actually only supports macOS 10.13 or later.
Move the handling of allocMethodBlockCounts to the method context.
While this data is both an input and an output, we're more interested
in the input side.

We could also treat it as an output, if we say wanted to verify that
the replay jit wrote the same sequence of IL offsets into the BlockCount
records, but that doesn't seem crucial as changes here would also lead
to code diffs.

Fix the code that checks the AddressMap for reloc hints so it doesn't
fail if the exact lookup fails, but instead falls back to a search.
This is needed because the base of the replay count buffer is recorded
as the map key, but the instrumentation probes in jitted code refer
to offsets from this value.

Fixes #37270.
…083)

* Port nullability annotations to System.Xml.ReaderWriter contract assembly

* Port annotations of recently updated members in Xml/Serialization

* Fix anontations on XmlElement discovered while updating System.Data.Common Xml related annotations

* Fix Xml related annotations on System.Data.Common

* address feedback

Co-authored-by: Krzysztof Wicher <kwicher@microsoft.com>
* add tracing to Decryp to complement tracing in Encrypt

* feedback from review

* revert encrypt
* Fixup JSON equality checks in polymorphic converter tests

* Limit deserialization value validation to custom _polymorphic_ converters
* remove semicolon from upload token

* will revert - test private commands

* remove condition

* add debug code

* add dump task

* comment out internal param

* rename helixprecommands to sharedhelixprecommands

* update and revert for check-in
…tributes (#41484)

* Remove if-def from JSON ref file previously used to support linker attributes

* Run GenerateReferenceSource target on ref project
…elements (#41488)

* Switch Markup property to XmlNode?[]? to reflect it can contain null elements

* Update ref
…th (#41469)

.. it's pdb. Running debugger-tests, I saw lot of messages like:

`CWL: : /Users/radical/dev/runtime/src/mono/mono/metadata/debug-mono-ppdb.c:230 <disabled>`

.. which is trying to print:
`CWL: Symbol file data-0x1c38008 doesn't match image System.Private.CoreLib.dll`

And indeed the dll is out of sync with it's pdb. We need to copy the pdb
too.
…41474)

* Port nullability annotations to refs XmlDocument and XmlSerializer

* Fix new System.Data.Common nullability related errors

* Switch nullability of parameter in SoapElementAttribute ctor
…s (#41086)

* Enable nullability on System.Xml.XPath.XDocument src

* Port nullability annotations to System.Xml.XPath.XDocument

* Port nullability annotations to System.Xml.XDocument

* Add ? to XmlAttribute argument in int? explicit operator
Noticed we were missing this while looking at another PR.
Updates the emscripten Docker container we use to build in CI. We no longer need to explicitly set `EMSDK_PATH` since the container already does it.
Also remove `--embedded` flag from `./emsdk activate` since it was deprecated and is now the default.
Copy link
Member

@jkotas jkotas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@MichalStrehovsky
Copy link
Member

What would it take to set up official builds and packages from the NativeAOT branch as well?

@joperezr
Copy link
Member Author

What would it take to set up official builds and packages from the NativeAOT branch as well?

I'll know for sure once I enable that for this branch but theoretically all changes will have to go into runtimelab.yml which will be added into dotnet/runtime repo, so once this branch is publishing official packages all NativeAOT branch would have to do in order to start getting official packages on the experimental feed would be to take a new ingestion from dotnet/runtime.

@joperezr joperezr merged commit 1741472 into dotnet:Utf8String Aug 28, 2020
@joperezr joperezr deleted the runtime-ingestion branch August 28, 2020 20:20
scalablecory pushed a commit that referenced this pull request Sep 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.