Update SDK to 7.0.100-alpha.1.21480.20 #37192

Tratcher · 2021-10-01T17:56:45Z

Weekly SDK update.

wtgodbe · 2021-10-01T18:27:37Z

@Tratcher you'll probably need all the changes from #36783

Tratcher · 2021-10-01T22:04:18Z

Tratcher · 2021-10-01T22:27:16Z

Running tests: dotnet run --no-restore --project RunTests/RunTests.csproj -- --target Microsoft.AspNetCore.WebSockets.Tests.dll --runtime 7.0.0-ci --queue Ubuntu.1804.Amd64.Open --arch x64 --quarantined false --ef 7.0.0-alpha.1.21501.1 --helixTimeout 00:30:00
It was not possible to find any compatible framework version
The framework 'Microsoft.NETCore.App', version '6.0.0-rc.2.21452.2' (x64) was not found.
  - The following frameworks were found:
      7.0.0-alpha.1.21480.1 at [/datadisks/disk1/work/9AA508EB/p/dotnet-cli/shared/Microsoft.NETCore.App]
      7.0.0-alpha.1.21501.1 at [/datadisks/disk1/work/9AA508EB/p/dotnet-cli/shared/Microsoft.NETCore.App]

Helix seems to be stuck on RC2? Any idea where that's specified?

Nevermind, I think it's here:

aspnetcore/eng/helix/content/RunTests/RunTests.csproj

Line 5 in 3b7f281

eng/helix/content/RunTests/RunTests.csproj

Tratcher · 2021-10-01T22:53:57Z

Linux build failure:

Cannot get required symbol SSL_set_alpn_protos from libssl
/mnt/vss/_work/1/s/eng/common/tools.sh: line 475:   139 Aborted                 (core dumped) "$_InitializeBuildTool" "$@"

dotnet/runtime#59431 (comment)

Are our linux agents old?

eng/docker/rhel.Dockerfile

dougbu · 2021-10-04T20:34:39Z

New image seems to require a change to our commands:

  /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/xpath_parser.rb:12: warning: Refinements are experimental, and the behavior may change in future versions of Ruby!
  /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/document.rb:12:in `require_relative': /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/parsers/baseparser.rb:515: syntax error, unexpected ',' (SyntaxError)
  /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/parsers/baseparser.rb:542: syntax error, unexpected ','
  /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/parsers/baseparser.rb:673: syntax error, unexpected keyword_end, expecting end-of-input
  	from /usr/local/share/gems/gems/rexml-3.2.5/lib/rexml/document.rb:12:in `<top (required)>'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/local/share/gems/gems/fpm-1.13.1/lib/fpm/package/osxpkg.rb:7:in `<top (required)>'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/local/share/gems/gems/fpm-1.13.1/lib/fpm.rb:15:in `<top (required)>'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
  	from /usr/local/share/gems/gems/fpm-1.13.1/bin/fpm:4:in `<top (required)>'
  	from /usr/local/bin/fpm:23:in `load'
  	from /usr/local/bin/fpm:23:in `<main>'
/mnt/vss/_work/2/s/src/Installers/Rpm/Directory.Build.targets(80,5): error MSB3073: The command "fpm --verbose --input-type=dir --output-type=rpm --name=aspnetcore-runtime-7.0 --package=/mnt/vss/_work/2/s/artifacts/installers/Release/aspnetcore-runtime-7.0.0-ci-x64.rpm --version=7.0.0 --iteration=0.1.ci --architecture=amd64 --depends="dotnet-runtime-7.0 >= 7.0.0" --rpm-changelog="/mnt/vss/_work/2/s/artifacts/obj/Rpm.Runtime/Release/linux-x64/changelog" --rpm-summary="Microsoft ASP.NET Core 7.0.0-ci Shared Framework" --description="Shared Framework for hosting of Microsoft ASP.NET Core applications. It is open source, cross-platform and is supported by Microsoft. We hope you enjoy using it! If you do, please consider joining the active community of developers that are contributing to the project on GitHub (https://github.com/dotnet/aspnetcore). We happily accept issues and PRs." --maintainer="Microsoft <nugetaspnet@microsoft.com>" --vendor="Microsoft Corporation" --license=MIT --url=https://asp.net --directories="/usr/share/dotnet/shared/Microsoft.AspNetCore.App" "/mnt/vss/_work/2/s/artifacts/obj/SharedFx.Layout/Release/linux-x64/=/usr/share/dotnet/"" exited with code 1. [/mnt/vss/_work/2/s/src/Installers/Rpm/Runtime/Rpm.Runtime.rpmproj]

Tratcher · 2021-10-04T21:04:34Z

Hmm, the only commas are in the description.
--description="Shared Framework for hosting of Microsoft ASP.NET Core applications. It is open source, cross-platform and is supported by Microsoft. We hope you enjoy using it! If you do, please consider joining the active community of developers that are contributing to the project on GitHub (https://github.com/dotnet/aspnetcore). We happily accept issues and PRs."

We could always try a run-on sentence 😁.

dougbu · 2021-10-04T21:16:44Z

We could always try a run-on sentence 😁.

Maybe the command supports escaping the comma in some way❔ I'm not sure how to find the Ruby sources but I'm hoping it's more clear when looking at baseparser.rb:673, baseparser.rb:542, or baseparser.rb:515 what it's attempting to parse.

Tratcher · 2021-10-04T21:26:19Z

Hmm
https://github.com/ruby/rexml/blob/v3.2.5/lib/rexml/parsers/baseparser.rb#L515

I'm not sure it's the command, it looks more like an issue loading the ruby code. Does this image have the right version?

Tratcher · 2021-10-04T21:32:31Z

ruby/rexml#69
https://www.server-world.info/en/note?os=CentOS_7&p=ruby25

janvorli · 2021-10-06T16:14:14Z

It is possible to install ruby 2.5 on centos 7 though, from the centos-sclo-rh repo. You's need to execute anything that uses ruby using scl enable rh-ruby25 or run the bash shell that you run your stuff in using that command (scl enable rh-ruby25 bash) or add /etc/profile.d/rh-ruby25.sh script like described here: https://www.server-world.info/en/note?os=CentOS_7&p=ruby25
Would that be feasible?

TanayParikh · 2021-10-06T18:43:54Z

It is possible to install ruby 2.5 on centos 7 though, from the centos-sclo-rh repo. You's need to execute anything that uses ruby using scl enable rh-ruby25 or run the bash shell that you run your stuff in using that command (scl enable rh-ruby25 bash) or add /etc/profile.d/rh-ruby25.sh script like described here: https://www.server-world.info/en/note?os=CentOS_7&p=ruby25 Would that be feasible?

I believe that'd require updating the dockerfile here with the ruby25 script / instantiation you suggest. This would likely involve creating and publishing a new image with those changes. Would that be possible? cc/ @dougbu ?

dougbu · 2021-10-06T18:48:08Z

@janvorli

As for using CentOS 8, is this image used to build some native components too?

We use the image to build only our two RPM projects: src/Installers/Rpm/Runtime/Rpm.Runtime.rpmproj and src/Installers/Rpm/TargetingPack/Rpm.TargetingPack.rpmproj. Most of the real work is done in src/Installers/Rpm/Directory.Build.targets. From what I can see, fpm gets run but not much else. See a recent log for example.

If we can move to CentOS 8 without breaking our RHEL 7 compatibility, I guess I'm 🆗 with it. I continue to have nagging doubts about whether the fpm version there produces the right RPMs.

Hmm, I've just loaded the rhel 7 image mcr.microsoft.com/dotnet-buildtools/prereqs:rhel-7-rpmpkg-e1b4a89-20175311035359 and the ruby -v shows ruby 2.0.0p648 (2015-12-16) [x86_64-linux].

That's not a big surprise because that image is for RHEL 7.5 and is 3 or 4 years old. We were using it until it stopped working with the OpenSSL expectations in the SDK. That's why we've been messing w/ the image in the first place.

An updated RHEL 7 or CentOS 7 image containing compatible rexml and ruby versions would be ideal.

scl enable rh-ruby25 bash

I guess we could add this to eng/docker/rhel.Dockerfile. But, wouldn't a working and modern rpmpkg image on mcr be the right way forward❔

dougbu · 2021-10-06T18:51:49Z

This would likely involve creating and publishing a new image with those changes.

No, that wouldn't be necessary @TanayParikh. the Dockerfile is completely additive; it creates layers on top of the image we download from mcr.

pranavkm · 2021-10-07T16:08:50Z

@dougbu could you summarize what the next steps are to get this merged?

dougbu · 2021-10-07T16:35:26Z

@dougbu could you summarize what the next steps are to get this merged?

Sure @pranavkm. We're basically waiting for more information from people like @janvorli @mthalman @NikolaMilosavljevic. The options are

Leave this PR as-is and merge it. That would move us from an old RHEL 7 image to a current CentOS 8 image when we execute fpm. A potential downside is changing how (or if) the RPM works on RHEL 7.
Change eng/docker/rhel.Dockerfile to include scl enable rh-ruby25 bash and use mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-rpmpkg-20210714125435-9b5bbc2. This avoids the above downside but hacks around an incompatibility w/in the image (which is intended for fpm execution).
Wait for a compatible CentOS or RHEL 7 image on mcr i.e. one w/ aligned rexml and ruby versions.

Looking back at this list, I lean toward (2) unless someone tells us my concerns about creating the RPM on a CentOS 8 image and running it on RHEL 7 are unfounded.

dougbu · 2021-10-07T16:38:03Z

Nice to have: Figure out why the new SDK appears to be incompatible w/ the OpenSSL version on RHEL 7.5. It may not matter because 7.5 is out of support (right @Tratcher❔) but we tend not to break things this way.

Tratcher · 2021-10-07T17:21:53Z

The original runtime change is discussed in dotnet/runtime#59431 (comment), which requires features from updated versions of OpenSSL. This is a harder requirement than has been enforced in the past.

pranavkm · 2021-10-07T18:44:32Z

How about we do option (b) and file an issue to mop this up based on a resolution? PR is out for a week and it'd be good to pick up SDK changes because waiting in the past has come to bite us.

eng/docker/rhel.Dockerfile

dougbu · 2021-10-07T21:59:07Z

Perhaps a missing /etc/scl/conf folder @pranavkm❔

Step 14/14 : RUN echo -e "#!/bin/bash\nsource /opt/rh/rh-ruby25/enable\nexport X_SCLS=\"`scl enable rh-ruby25 'echo $X_SCLS'`\"\n" >> /etc/profile.d/rh-ruby25.sh
 ---> Running in b2b9d5da7dbc
Unable to open /etc/scl/conf/rh-ruby25!
/bin/sh: /etc/profile.d/rh-ruby25.sh: Permission denied
The command '/bin/sh -c echo -e "#!/bin/bash\nsource /opt/rh/rh-ruby25/enable\nexport X_SCLS=\"`scl enable rh-ruby25 'echo $X_SCLS'`\"\n" >> /etc/profile.d/rh-ruby25.sh' returned a non-zero code: 1
##[error]Bash exited with code '1'.

/fyi I tend to fire up docker in WSL, install an image I'm interested in, then run docker run -it --rm 8455aeb53704 /bin/bash to play around a bit before trying things on the CI. (WSL is a bit of a pain for this because it doesn't use systemd or similar. I always forget to run sudo dockerd, authenticating, then putting the process in the background before doing anything else.)

janvorli · 2021-10-07T22:15:30Z

You have accidentally removed the

RUN yum --enablerepo=centos-sclo-rh -y install rh-ruby25

in the last iteration

eng/docker/rhel.Dockerfile

dougbu · 2021-10-08T19:01:14Z

eng/docker/rhel.Dockerfile

+RUN gem uninstall fpm
+RUN yum remove -y rubygems
+RUN yum remove -y ruby-devel
+RUN yum --enablerepo=centos-sclo-rh -y install rh-ruby25
+RUN yum --enablerepo=centos-sclo-rh -y install rh-ruby25-ruby-devel
+RUN yum --enablerepo=centos-sclo-rh -y install rh-ruby25-rubygems
+RUN scl enable rh-ruby25 'gem install --no-document fpm'


This is a lot and all the more reason to move to a Docker image with matching fpm and ruby installations soon

Yup. @janvorli helped with it. The problem was that fpm uses the ruby in the /usr/bin dir and we had to reset enough state to be have it use the new version. Again, I'd much rather we use this as a stop-gap and merge it (assuming it works this time), than hold out for a compatible image.

I completely agree on this being a stop gap. My comment amounted to "Ick, let's burn this w/ fire ASAP" 😃

#37406 tracks undoing these changes.

dougbu · 2021-10-08T21:26:30Z

Wahoo❕

Update SDK to 7.0.100-alpha.1.21480.20

d703ca2

Tratcher added this to the 17.1-Preview1 milestone Oct 1, 2021

Tratcher self-assigned this Oct 1, 2021

Tratcher requested a review from a team as a code owner October 1, 2021 17:56

ghost added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Oct 1, 2021

wtgodbe approved these changes Oct 1, 2021

View reviewed changes

Tratcher enabled auto-merge (squash) October 1, 2021 17:59

EnablePreviewFeatures in Transport.Quic

41c5262

Tratcher requested review from BrennanConroy and halter73 as code owners October 1, 2021 18:22

Remove RequiresPreviewFeatures

37f9ce1

Tratcher disabled auto-merge October 1, 2021 21:02

Tratcher modified the milestones: 17.1-Preview1, 7.0-preview1 Oct 1, 2021

Update RunTests.csproj

3b7f281

wtgodbe reviewed Oct 1, 2021

View reviewed changes

eng/helix/content/RunTests/RunTests.csproj Show resolved Hide resolved

Tratcher added 4 commits October 4, 2021 10:19

Use DefaultNetCoreTargetFramework

e045062

More DefaultNetCoreTargetFramework

ab087e1

centos7

5982858

Less DefaultNetCoreTargetFramework

9671ae3

dougbu reviewed Oct 4, 2021

View reviewed changes

eng/docker/rhel.Dockerfile Show resolved Hide resolved

dougbu mentioned this pull request Oct 6, 2021

Merge bootstrap4 sxs support from release/6.0 #37294

Closed

Use CentOS 7 image

a0d5593

pranavkm force-pushed the tratcher/SDK branch from 2403711 to a0d5593 Compare October 7, 2021 20:47

pranavkm reviewed Oct 7, 2021

View reviewed changes

eng/docker/rhel.Dockerfile Outdated Show resolved Hide resolved

Update eng/docker/rhel.Dockerfile

273a50b

pranavkm reviewed Oct 7, 2021

View reviewed changes

eng/docker/rhel.Dockerfile Outdated Show resolved Hide resolved

Update eng/docker/rhel.Dockerfile

5cebe3e

pranavkm reviewed Oct 7, 2021

View reviewed changes

eng/docker/rhel.Dockerfile Outdated Show resolved Hide resolved

pranavkm added 4 commits October 7, 2021 15:28

Update eng/docker/rhel.Dockerfile

979089b

Run yum while root

c54faee

Cool surgery

685b125

WIP

d826016

Tratcher requested a review from Pilchie as a code owner October 8, 2021 18:53

dougbu reviewed Oct 8, 2021

View reviewed changes

pranavkm mentioned this pull request Oct 8, 2021

Undo workarounds added to dockerfile while updating to 7.0 SDK #37406

Closed

pranavkm merged commit a06b39e into main Oct 8, 2021

pranavkm deleted the tratcher/SDK branch October 8, 2021 20:30

wtgodbe mentioned this pull request Apr 14, 2022

Try new docker image #41194

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update SDK to 7.0.100-alpha.1.21480.20 #37192

Update SDK to 7.0.100-alpha.1.21480.20 #37192

Tratcher commented Oct 1, 2021

wtgodbe commented Oct 1, 2021

Tratcher commented Oct 1, 2021

Tratcher commented Oct 1, 2021 •

edited

Loading

Tratcher commented Oct 1, 2021 •

edited

Loading

dougbu commented Oct 4, 2021

Tratcher commented Oct 4, 2021

dougbu commented Oct 4, 2021

Tratcher commented Oct 4, 2021

Tratcher commented Oct 4, 2021 •

edited

Loading

janvorli commented Oct 6, 2021

TanayParikh commented Oct 6, 2021

dougbu commented Oct 6, 2021 •

edited

Loading

dougbu commented Oct 6, 2021

pranavkm commented Oct 7, 2021

dougbu commented Oct 7, 2021

dougbu commented Oct 7, 2021 •

edited

Loading

Tratcher commented Oct 7, 2021

pranavkm commented Oct 7, 2021

dougbu commented Oct 7, 2021

janvorli commented Oct 7, 2021

dougbu Oct 8, 2021

pranavkm Oct 8, 2021

dougbu Oct 8, 2021

pranavkm Oct 8, 2021

dougbu commented Oct 8, 2021

Update SDK to 7.0.100-alpha.1.21480.20 #37192

Update SDK to 7.0.100-alpha.1.21480.20 #37192

Conversation

Tratcher commented Oct 1, 2021

wtgodbe commented Oct 1, 2021

Tratcher commented Oct 1, 2021

Tratcher commented Oct 1, 2021 • edited Loading

Tratcher commented Oct 1, 2021 • edited Loading

dougbu commented Oct 4, 2021

Tratcher commented Oct 4, 2021

dougbu commented Oct 4, 2021

Tratcher commented Oct 4, 2021

Tratcher commented Oct 4, 2021 • edited Loading

janvorli commented Oct 6, 2021

TanayParikh commented Oct 6, 2021

dougbu commented Oct 6, 2021 • edited Loading

dougbu commented Oct 6, 2021

pranavkm commented Oct 7, 2021

dougbu commented Oct 7, 2021

dougbu commented Oct 7, 2021 • edited Loading

Tratcher commented Oct 7, 2021

pranavkm commented Oct 7, 2021

dougbu commented Oct 7, 2021

janvorli commented Oct 7, 2021

dougbu Oct 8, 2021

Choose a reason for hiding this comment

pranavkm Oct 8, 2021

Choose a reason for hiding this comment

dougbu Oct 8, 2021

Choose a reason for hiding this comment

pranavkm Oct 8, 2021

Choose a reason for hiding this comment

dougbu commented Oct 8, 2021

Tratcher commented Oct 1, 2021 •

edited

Loading

Tratcher commented Oct 1, 2021 •

edited

Loading

Tratcher commented Oct 4, 2021 •

edited

Loading

dougbu commented Oct 6, 2021 •

edited

Loading

dougbu commented Oct 7, 2021 •

edited

Loading