`global.json` updates #71

KathleenDollard · 2019-05-13T00:22:00Z

Add features for

Preview
Exact
Latest within feature band
Latest with lower bound

tannergooding · 2019-05-13T14:26:17Z

I think one of the scenarios that is missing here, but that could easily be addressed, is allowing the user to specify a local install directory to check.

That is, for CI, some repos will always download a local SDK, regardless of what is available on box. And, while we have the ability to include additional search paths by modifying the current environment, there is no way to configure that to "just work" for a clean enlistment where the user is directly running dotnet build.

I would propose that you should be able to specify an additional directory to check for dotnet installs as part of the global.json. For example, specifying { "sdk": { "localDirectory": "./artifacts/dotnet" } } (or a better name) would cause dotnet to first try to resolve an SDK/runtime installed under localDirectory.

As extensions, you could also have an option that blocks searching in the default locations (i.e. only look under localDirectory and not on the path) and an option that indicates whether or not downloading a missing sdk should be allowed (currently this requires CI environments to pull down the dotnet-install script and pass in much of the information that already exists in the global.json).

This functionality, combined with the new functionality proposed in the OP, would allow a repository to ensure that an appropriate SDK version is used and also have that integrate with their repo infrastructure such that they don't require users to use build.cmd/sh scripts to work successfully.

KathleenDollard · 2019-05-13T17:28:08Z

We have this in mind for a future update, but need to keep this one simple.

In other words: this is the first step of a journey.

accepted/global-json-updates.md

jaredpar · 2019-05-13T23:00:20Z

accepted/global-json-updates.md

+
+When a `global.json` is encountered and it specifies an SDK version, the SDK resolvers attempt to locate that specific SDK patch version:
+
+1. When the resolver finds no `global.json` or finds one that does not specify SDK version, the latest SDK is used, regardless of whether it is preview or stable.


This doesn't seem to reflect the current behavior. Consider that when building from VS the preview SDK is only used from a preview VS.

Or is this correct in that an RTM VS will pick a preview SDK if there is no global.json but refuse to use a preview SDK if there is a global.json that specifies it?

Correct. This reflects the behavior of dotnet, but not VS. VS passes in a flag to the same underlying to control when previews are used or not. This flag defaults based on whether VS is a preview and can be overridden by a checkbox in VS. Having a bool in the global.json essentially makes this flag available without the checkbox.

So, dotnet can respect it too, and you can set up previews to be used in VS irrespective of checkboxes with this.

jaredpar · 2019-05-13T23:04:14Z

accepted/global-json-updates.md

+    * Prior to .NET Core 2.1
+      * The highest patch version for the SDK feature band is used _regardless of whether it is higher than what is specified_. For example, if `global.json` specifies 2.2.103 is specified and is not available, and 2.2.101 is the highest SDK on the machine, it will be used.
+    * In .NET Core 2.1 and higher
+      * The highest patch version for the SDK feature band is used _assuming it is higher than what is specified_. For example, if `global.json` specifies 2.2.103 is specified, and 2.2.101 is the highest SDK on the machine, an error will occur.


If I'm reading this correct we get the "greater than or equal to SDK version" feature by just specifying the version up to the feature brand level? That seems to be how the document reads but at the same time the behavior I'm describing has always seemed to be an ask vs. something that was already there.

The behavior today is exact version if it exists, else latest from feature band.

I'm assuming @nguerrera comment resolves @jaredpar 's. This is not the same as greater or equal, which is a feature added later in this proposal.

jaredpar · 2019-05-13T23:06:03Z

accepted/global-json-updates.md

+  * `ignorePreview`
+    * `true` or `false`
+    * Default: `false` to match previous behavior.
+    * Alternate names: `releaseOnly` or `useReleaseOnly`


Consider also usePreview. Feel like a positive declarative property about the behavior desired here is more natural.

I like usePreview.

jaredpar · 2019-05-13T23:06:47Z

accepted/global-json-updates.md

+    * Default: `false` to match previous behavior.
+    * Alternate names: `releaseOnly` or `useReleaseOnly`
+    * This is new behavior.
+  * `rollForward`


What would be the impact of declaring rollForward but not specifying sdk.version? It seems this property is only meaningful if sdk.version is specified correct?

nguerrera · 2019-05-14T20:56:21Z

accepted/global-json-updates.md

+  "sdk": {
+    "version": "3.0.100-Pre",
+    "ignorePreview": true,
+    "rollForward": "highest"


Should this be:

Suggested change

"rollForward": "highest"

"rollForward": "latest"

jaredpar · 2019-05-14T22:33:45Z

accepted/global-json-updates.md

+    * This is new behavior.
+  * `rollForward`
+    * Lower version numbers are always ignored. 
+    * `patch`


Is there a functional difference between the following:

Option 1:

"sdk": { "version": "3.0.1" }

Option 2

"sdk": { "version": "3.0.100", "rollForward": "highest" }

I'm confused. There is no 3.0.1 patch of the SDK. Is there a typo in your comment?

Co-Authored-By: Jared Parsons <jaredpparsons@gmail.com>

accepted/global-json-updates.md

… into global-json-updates

KathleenDollard

@peterhuene Can you review for correctness. I let this public proposal get out of date.

Thanks!

KathleenDollard · 2019-06-23T21:08:38Z

accepted/global-json-updates.md

+    * Prior to .NET Core 2.1
+      * The highest patch version for the SDK feature band is used _regardless of whether it is higher than what is specified_. For example, if `global.json` specifies 2.2.103 is specified and is not available, and 2.2.101 is the highest SDK on the machine, it will be used.
+    * In .NET Core 2.1 and higher
+      * The highest patch version for the SDK feature band is used _assuming it is higher than what is specified_. For example, if `global.json` specifies 2.2.103 is specified, and 2.2.101 is the highest SDK on the machine, an error will occur.


I'm assuming @nguerrera comment resolves @jaredpar 's. This is not the same as greater or equal, which is a feature added later in this proposal.

KathleenDollard · 2019-06-23T21:50:22Z

accepted/global-json-updates.md

+    * This is new behavior.
+  * `rollForward`
+    * Lower version numbers are always ignored. 
+    * `patch`


I'm confused. There is no 3.0.1 patch of the SDK. Is there a typo in your comment?

accepted/global-json-updates.md

nguerrera · 2019-06-27T01:01:25Z

accepted/global-json-updates.md

+    * Default: empty
+  * `usePreview`
+    * `true` or `false`
+    * Default: `true` to match today's behavior.


We should state that for VS, default is to honor what VS does for previews as described above. And Global.json should win over VS on this.

nguerrera · 2019-06-27T01:02:10Z

accepted/global-json-updates.md

+## Proposal
+
+This proposal parallels [runtime roll-forward behavior as initially described](https://github.com/dotnet/core-setup/blob/master/Documentation/design-docs/roll-forward-on-no-candidate-fx.md)} and [recent improvements](https://github.com/dotnet/designs/blob/master/accepted/runtime-binding.md). All of the features proposed here may not be included in Phase 1. It will include at least `usePreview` and `"rollforward": "disable"`
+


We're doing the full set below now, right. Do we need the disclaimer about phase 1?

nguerrera · 2019-06-27T01:05:46Z

accepted/global-json-updates.md

+      * `feature`, `minor`, `major`: highest patch of the nearest version within specified limits.
+      * `latestPatch`, `latestFeature`, `latestMinor`, `latestMajor`: highest patch of the highest available version within specified limits.
+      * `disable`: use exactly the version specified.
+    * `patch` **[[(I've gone back and forth on "patch" or "legacy" here)]]**


I like patch. It's not exactly analogous to the other non-latest as it fallback to latest instead of falling back to nearest, but close enough. We can document the difference.

nguerrera · 2019-06-27T01:08:04Z

accepted/global-json-updates.md

+
+The new behavior is that the highest patch within the feature band will be used even if the requested version is available, unless the `rollForward` is set to `disable`.
+
+Also, the nearest higher SDK will be used if there is no version that matches the requested major, minor and feature. This behavior can be changed by specifying other options for the `rollForward` property. 


Is it nearest higher SDK or is it basically RollForward=major by default? The difference I see is in the case where there are multiple patches.

I think the default should map to some well defined roll forward setting - it should not be yet another way of doing things - that's super confusing. I don't know which is the best default... currently it is basically patch. Changing it to major feels like a rather big change. But I guess we make different guarantees about SDK backward compatibility across major versions than we do for the product...

VS will be cleaning up after itself moving forward and will only have a current SDK with optional runtimes, templates, etc. for older version. As such, using recent SDK with projects targeting older runtimes is mainline scenario. We should not break it as a general rule. There will be exceptions and those can be handled with an explicit strengthening in global.json IMHO. The current strictness by default causes a mess of having to keep a bazillion sdks to open up different projects. And we're not going to be leaving SDKs behind anymore.

There will be exceptions and those can be handled with an explicit strengthening in global.json IMHO.

Or you can install older SDK and major will still prefer it. In the case where you hit a break.

I do think that the sdk compatibility is different than runtime compatibility. We have always (meaning before .NET Core even) given customers latest tools (Visual Studio et al) to use on their existing projects. I think this should always work by default.

accepted/global-json-updates.md

Co-Authored-By: Nick Guerrera <nicholg@microsoft.com>

vitek-karas · 2019-06-27T16:31:55Z

accepted/global-json-updates.md

+  * `usePreview`
+    * `true` or `false`
+    * Default: `true` to match today's behavior.
+  * `rollForward` (The `version` must be specified if `rollForward` is specified, except `latestMajor`.)


The version should be specified always if rollForward is specified. Having the latestMajor as an exception feels really weird (even though I understand that it would effectively give us the default behavior).

I disagree only because there would be no way to represent the current default behavior if global.json were not present (or present and not have sdk settings) in global.json without allowing a version-less latestMajor. But it's not a strongly-held disagreement since I don't know how important that is.

I think it is essential that we be able to represent the default behavior with an explicit entry (latestMajor with no version). This is the only way to override a setting higher in the directory structure.

Does version=0.0.0, rollForward=latestMajor not work? I think allowing version to be omitted for latestMajor is fine, but wondering if it's true that it's impossible. Certainly using 0.0.0 is not obvious.

That's a good point. I could see a scenario where someone might want some subtree to not respect the SDK settings from further up the source tree and just use the default "without a global.json" behavior.

You're right, version 0.0.0 would work and I agree that it would be non-obvious.

Given that works, perhaps we should update the design to express the default "no global.json present" behavior with a version of 0.0.0 and treat a missing version when rollForward is specified as an error for consistency? I'm happy either way, personally.

I do agree that 0.0.0 is not discoverable... I guess the original is OK then.

vitek-karas · 2019-06-27T16:33:11Z

accepted/global-json-updates.md

+    * `true` or `false`
+    * Default: `true` to match today's behavior.
+  * `rollForward` (The `version` must be specified if `rollForward` is specified, except `latestMajor`.)
+    * There are four basic intents for roll-forward:


To match the framework roll forward setting - we should explicitly state that these values are case insensitive - so I can use Patch, LatestMajor and so on.

Currently the implementation PR is case-sensitive, but if the runtime does it this way, the SDK should to (personally not a big fan of case-insensitive enums).

I'll fix the implementation PR.

We should match the runtime here. Thanks @vitek-karas for the catch.

Context: We allowed case-insensitive for framework roll forward because that setting can be specified in several different ways. For JSON it makes sense to use the same casing you have here. For command that is less obvious but might be OKish, but for environment variables it just didn't look right.

vitek-karas · 2019-06-27T17:16:05Z

accepted/global-json-updates.md

+
+The new behavior is that the highest patch within the feature band will be used even if the requested version is available, unless the `rollForward` is set to `disable`.
+
+Also, the nearest higher SDK will be used if there is no version that matches the requested major, minor and feature. This behavior can be changed by specifying other options for the `rollForward` property. 


I think the default should map to some well defined roll forward setting - it should not be yet another way of doing things - that's super confusing. I don't know which is the best default... currently it is basically patch. Changing it to major feels like a rather big change. But I guess we make different guarantees about SDK backward compatibility across major versions than we do for the product...

vitek-karas · 2019-06-27T17:25:14Z

accepted/global-json-updates.md

+    * The lowest SDK the resolver would select - any SDKs with lower versions are ignored.
+    * Identical to today - wildcards are not supported.
+    * Default: empty
+  * `usePreview`


In SemVer the correct term is pre-release. In the framework roll forward we even have an environment variable which uses the term "pre-release" for this.

I think this needs a much more detailed description of the desired behavior. Again in framework roll forward we actually spent a lot of time on this: https://github.com/dotnet/core-setup/blob/master/Documentation/design-docs/framework-version-resolution.md#pre-release-versions

In short:

If the version is a release version, should we try to fulfill the request with release first and only then fallback to pre-release? (even is usePreview is true).

What happens if version is pre-release but usePreview=false

If usePreview=false and we can't find any version and are about to fail - should we consider pre-release versions in this case to avoid failing?

In the below text you always state that we pick the latest patch in the given range (except for disable). We should be explicit what it means for pre-release versions. Do we always pick latest pre-release for a given patch? We actually went back and forth on this for frameworks and ended up having a somewhat in-between solution:

If the version is pre-release, and there is an exact match, use it - don't roll to latest patch/pre-release - otherwise it's really hard to test several different pre-release versions side-by-side. Also we tend to treat pre-release versions as breaking (P4 versus P8 - there might be breaking changes there), so it's better to pick "closest" in this case, to avoid breaking people. I know that we mostly mitigate this by uninstalling other previews, but still - even for internal testing this is important.

The setting is allowPrerelease. This part is out of date.

To answer questions:

If the version is a release version, should we try to fulfill the request with release first and only then fallback to pre-release? (even is usePreview is true).

If I specify version = 2.2.300, rollForward = latestMajor, allowPrerelease = true and have 2.2.505 and 3.0.100-preview7 installed, I would be quite surprised if it prefers 2.2.505 over 3.0.100-preview7. I don't think the version should have semantic meaning regarding whether or not prereleases are matched against (see next answer).

What happens if version is pre-release but usePreview=false

Currently it silently allows prerelease versions. My changes make this a warning saying we're ignoring your allowPrerelease setting and proceed to allow prereleases. Personally, I don't like either of these behaviors as to me version is a minimum bound for the roll-forward policy and shouldn't otherwise have semantic meaning.

If usePreview=false and we can't find any version and are about to fail - should we consider pre-release versions in this case to avoid failing?

I don't think so. If I instruct my tools to not consider prerelease versions (which is not the default) then it should do so.

Do we always pick latest pre-release for a given patch?

As this design is currently going to be implemented and assuming allowPrerelease = true, yes. Users should be able to explicitly use a specific prerelease patch level by using either the patch or disable policies. Generally users don't install preview servicing releases and our testers don't test multiple preview servicing releases side by side, so I think the impact here would be low.

OK - makes sense, I just think this should be explicitly stated - if nothing else so that there's a doc which can be compared against tests/actual behavior.

accepted/global-json-updates.md

sdmaclea · 2019-06-28T19:10:07Z

accepted/global-json-updates.md

+Prior to arriving at this proposal, two others were considered. Neither of these align well with roll-forward on the .NET Core Runtime.
+
+* An independent versioning scheme. It's largest downside is that it would be _another_ way users would have to think about specifying versioning behavior. 
+* An attempt to parallel NuGet. Definining [NuGet version ranges](https://docs.microsoft.com/en-us/nuget/reference/package-versioning) is relatively complex, and it didn't feel like a good fit. 


We may also want to consider a version blacklist. A specific version is incompatible with a given project. Triggers a compile exception, generate bad code ...

@sdmaclea Is there a specific scenario, or is this for completeness/in case we screw up.

Since we're human, this makes sense to consider for a backlog item, but I don't want to make the current round of changes any more complex unless necessary.

This is for completeness or in case we screw up.

We don't have a way to work around specific broken SDK releases. It is especially problematic when the latest release is broken. Presumably the preferred solution here would be to use disable with a specific version.

Co-Authored-By: Steve MacLean <stmaclea@microsoft.com>

vitek-karas · 2019-10-11T11:41:45Z

@KathleenDollard any objections to merge this so that we have a doc we can point customers at?

KathleenDollard added 2 commits May 5, 2019 19:35

Add proposal for global.json updates

0e683b8

Simplified proposal

2775347

jaredpar reviewed May 13, 2019

View reviewed changes

nguerrera reviewed May 14, 2019

View reviewed changes

jaredpar reviewed May 14, 2019

View reviewed changes

Update accepted/global-json-updates.md

fe947bf

Co-Authored-By: Jared Parsons <jaredpparsons@gmail.com>

nguerrera reviewed Jun 21, 2019

View reviewed changes

accepted/global-json-updates.md Show resolved Hide resolved

nguerrera reviewed Jun 21, 2019

View reviewed changes

accepted/global-json-updates.md Outdated Show resolved Hide resolved

KathleenDollard added 5 commits June 23, 2019 14:26

Updated strategy based on runtime roll forward

14d8ce6

Merge branch 'global-json-updates' of https://github.com/dotnet/designs…

10d7dca

… into global-json-updates

Removed old proposals, updated usePreview

8704917

Updated more usePreview

04fc796

Respond to some comments

da302f7

KathleenDollard commented Jun 23, 2019

View reviewed changes

Changed to high patch, changed default

c79e219

peterhuene reviewed Jun 24, 2019

View reviewed changes

accepted/global-json-updates.md Outdated Show resolved Hide resolved

Updated roll-forward options

8d8eb31

peterhuene approved these changes Jun 26, 2019

View reviewed changes

nguerrera approved these changes Jun 27, 2019

View reviewed changes

peterhuene mentioned this pull request Jun 27, 2019

Implement roll-forward policies for .NET Core SDK resolution. dotnet/core-setup#6953

Merged

KathleenDollard and others added 3 commits June 27, 2019 09:07

Update accepted/global-json-updates.md

117d7eb

Co-Authored-By: Nick Guerrera <nicholg@microsoft.com>

Update accepted/global-json-updates.md

eb73096

Co-Authored-By: Nick Guerrera <nicholg@microsoft.com>

Update based on comments

95aacbb

vitek-karas reviewed Jun 27, 2019

View reviewed changes

Updated usePreview to allowPrerelease

3de53e3

sdmaclea approved these changes Jun 28, 2019

View reviewed changes

Update accepted/global-json-updates.md

56b7fcd

Co-Authored-By: Steve MacLean <stmaclea@microsoft.com>

jnm2 mentioned this pull request Sep 12, 2019

Global.json is causing projects to fail to load in VS unless I install an older SDK DapperLib/Dapper#1294

Closed

This was referenced Sep 17, 2019

Add documentation for the new global.json options in 3.0 dotnet/docs#14479

Closed

Add information about new global.json options in 3.0 dotnet/docs#14480

Closed

dagood mentioned this pull request Sep 18, 2019

Incorrect SDK version used when multiple SDKs are installed dotnet/arcade#3764

Open

ViktorHofer mentioned this pull request Oct 7, 2019

Updating global.json to pick preview SDKs dotnet/corefx#41589

Merged

KathleenDollard merged commit 02237ac into master Oct 11, 2019

jeromelaban mentioned this pull request Jan 12, 2020

502.5 Bad gateway error running basic wasm app unoplatform/uno#2463

Closed

12 tasks

jkotas deleted the global-json-updates branch April 29, 2020 15:56

TheXenocide mentioned this pull request Apr 29, 2020

Make global.json "sdk/version" support min/max, or wildcards dotnet/runtime#2541

Closed


		When a `global.json` is encountered and it specifies an SDK version, the SDK resolvers attempt to locate that specific SDK patch version:

		1. When the resolver finds no `global.json` or finds one that does not specify SDK version, the latest SDK is used, regardless of whether it is preview or stable.

		## Proposal

		This proposal parallels [runtime roll-forward behavior as initially described](https://github.com/dotnet/core-setup/blob/master/Documentation/design-docs/roll-forward-on-no-candidate-fx.md)} and [recent improvements](https://github.com/dotnet/designs/blob/master/accepted/runtime-binding.md). All of the features proposed here may not be included in Phase 1. It will include at least `usePreview` and `"rollforward": "disable"`


		The new behavior is that the highest patch within the feature band will be used even if the requested version is available, unless the `rollForward` is set to `disable`.

		Also, the nearest higher SDK will be used if there is no version that matches the requested major, minor and feature. This behavior can be changed by specifying other options for the `rollForward` property.

global.json updates #71

global.json updates #71

Uh oh!

Conversation

KathleenDollard commented May 13, 2019

Uh oh!

tannergooding commented May 13, 2019

Uh oh!

KathleenDollard commented May 13, 2019

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

KathleenDollard left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peterhuene Jun 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peterhuene Jun 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

`global.json` updates #71

`global.json` updates #71

peterhuene Jun 27, 2019 •

edited

Loading

peterhuene Jun 27, 2019 •

edited

Loading

peterhuene Jun 27, 2019 •

edited

Loading