-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generate shim with PackAsTool #2239
Generate shim with PackAsTool #2239
Conversation
<data name="CannotFindApphostForRid" xml:space="preserve"> | ||
<value>Cannot find Apphost for {0}. {0} could be an invalid RuntimeIdentifier</value> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mairaw could you help us review this error message while @KathleenDollard is in Europe?
This is the error message shown when (most likely) users used a wrong runtime identifier like win-x63. Or it is a rid that is not supported
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My comments @wli3:
- should Apphost really be one word?
- RuntimeIdentifier -> runtime identifier (RID)
- missing punctuation in the last sentence
- should we add a fwlink to the RID topic for more info? https://docs.microsoft.com/dotnet/core/rid-catalog
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will, is it practical to search our error/warning messages for Apphost to see how we usually do it. I would like consistency on that. Apphost, AppHost, or application host all make sense to me, although app host not as much.
Same with RID vs RuntimeIdentifier. Do you have thoughts on the best place to keep notes that will help us be consistent on stuff like this?
A link to valid RIDs would be great here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apphost
according to https://docs.microsoft.com/en-us/dotnet/core/, it should be "app host". As in The 'dotnet' app host, which is used to launch .NET Core apps. It selects the runtime and hosts the runtime, provides an assembly loading policy and launches the app. The same host is also used to launch SDK tools in much the same way.
should we add a fwlink to the RID topic for more info?
created https://aka.ms/rid-catalog
is it practical to search our error/warning messages for Apphost to see how we usually do it. I would like consistency on that. Apphost, AppHost, or application host all make sense to me, although app host not as much.
should be "app host". That doc is kind of a "front page" of dotnet core
Same with RID vs RuntimeIdentifier. Do you have thoughts on the best place to keep notes that will help us be consistent on stuff like this?
"runtime identifier (RID)" should be the standard. As in docs.microsoft.com
so what about that
Cannot find app host for {0}. {0} could be an invalid runtime identifier (RID). For more information about RID https://aka.ms/rid-catalog.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can change the wording on docs if it's preferred.
Only minor suggestion:
For more information about RID, see https://aka.ms/rid-catalog.
And can you add me as a co-owner of that aka.ms link?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I prefer your version. And I added you to the aka.ms link
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed, changed to discussed
@KathleenDollard or @richlander could you review the property name it is used in csproj like <Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.1</TargetFramework>
<PackageToolShimRuntimeIdentifiers>win-x64;ubuntu-x64</PackageToolShimRuntimeIdentifiers>
<PackAsTool>true</PackAsTool>
</PropertyGroup>
</Project>
|
Regarding PackageToolShimRuntimeIdentifiers, I am not crazy about that name. Since this is parallel to
Yes it is for the package, but that's true on |
if I see Also the second question. Do you think it is a good idea to imply PackAsTool=true when above field is set? |
OK. What about
|
Talked to Kathleen in the morning. The current thinking is We don't want to imply PackAsTool = true. If people look at As a result, if PackAsTool = false and PackAsToolShimRuntimeIdentifiers has something, we will error out. |
<NETSdkError Condition=" '$(_TargetFrameworkVersionWithoutV)' < '2.1' " | ||
ResourceName="DotnetToolDoesNotSupportTFMLowerThanNetcoreapp21" /> | ||
|
||
<ItemGroup> | ||
<_GeneratedFiles Include="$(PublishDepsFilePath)"/> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
format change... sorry about that. However that's not too much and this happens several times.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok.
b3eb1fb
to
f5714e1
Compare
I find PackAsToolShimRuntimeIdentifiers a little long, but it is clear, so 👍. |
/// The RuntimeIdentifiers that shims will be generated for. | ||
/// </summary> | ||
[Required] | ||
public ITaskItem[] PackageToolShimRuntimeIdentifiers { get; set; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PackageToolShimRuntimeIdentifiers [](start = 27, length = 33)
I think we should rename this to match the property, or shorten it and decide there's enough context here to just "ShimRuntimeIdentifiers" or even "RuntimeIdentifiers". Right now, the unnecessary subtle difference between "PackAsToolShimRuntimeIdentifiers" and "PackageToolShimRuntimeIdentifiers" is confusing me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed, changed to ShimRuntimeIdentifiers
private static string NormalizeRelativePath(string relativePath) | ||
{ | ||
return relativePath.Replace('/', Path.DirectorySeparatorChar); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed, removed all these and just call nugetpackageresolver
<ItemGroup> | ||
<_PackageToolShimRuntimeIdentifiers Include="$(PackageToolShimRuntimeIdentifiers)"> | ||
|
||
</_PackageToolShimRuntimeIdentifiers> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: use self-closing tag.
<_PackageToolShimRuntimeIdentifiers Include="$(PackageToolShimRuntimeIdentifiers)" />
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
|
||
<GenerateShims | ||
DotNetAppHostExecutableNameWithoutExtension="$(_DotNetAppHostExecutableNameWithoutExtension)" | ||
PackagedShimOutputDirectory="$(IntermediateOutputPath)/$(Configuration)/shims/$(targetframework)" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(targetframework [](start = 85, length = 16)
Nit: casing: $(TargetFramework)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
else | ||
{ | ||
sdkCommandSpec.Environment.Add("DOTNET_ROOT(x86)", dotnetRoot); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we should do this for all tests here:
public void AddTestEnvironmentVariables(SdkCommandSpec command) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed moved to AddTestEnvironmentVariables
@@ -49,6 +49,7 @@ Copyright (c) .NET Foundation. All rights reserved. | |||
<PropertyGroup Condition="'$(NuGetBuildTasksPackTargets)' == '' AND '$(ImportNuGetBuildTasksPackTargetsFromSdk)' != 'false'"> | |||
<IncludeBuildOutput Condition=" '$(PackAsTool)' == 'true' ">false</IncludeBuildOutput> | |||
<PackageType Condition=" '$(PackAsTool)' == 'true' ">DotnetTool</PackageType> | |||
<RuntimeIdentifiers Condition=" '$(PackAsTool)' == 'true' ">$(RuntimeIdentifiers);$(PackageToolShimRuntimeIdentifiers)</RuntimeIdentifiers> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks wrong that this is in this property group. Why does the condition apply to it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It need to be set early enough before nuget is loaded. Only when PackAsTool=true aka use dotnet pack to create global tools this is useful. Also I want to add error when PackAsTool!=true and PackageToolShimRuntimeIdentifiers is set. According to what I talked to Kathleen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note I did not move this logic due to the reason above. It is out of date due to rename change
</ItemGroup> | ||
</Target> | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: extra blank line
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
public string ToolEntryPoint { get; set; } | ||
|
||
/// <summary> | ||
/// The output Directroy path of generated shims. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: directory. (Also, nit, I wouldn't capitalize it.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
Path.Combine(packagedShimOutputDirectoryAndRid, | ||
$"{ToolCommandName}{Path.GetExtension(resolvedApphostAssetPath)}"); | ||
var appBinaryFilePath = | ||
$".store/{PackageId.ToLowerInvariant()}/{PackageVersion}/{PackageId.ToLowerInvariant()}/{PackageVersion}/tools/{targetFramework.GetShortFolderName()}/any/{ToolEntryPoint}"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Strange to see this next to other Path.Combine calls. I think Path.Combine with each portion of path as an arg on its own line would be better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the embedded string. If we use Path.Combine, the embedded file will be different according to which machine builds it. \ build on win, / build on Linux. Although technically apphost is fine with that.
I will add comments and move it away from other Path.Combine
// This is the embedded string. We should normalize it on forward slash, so the result apphost won't be different according to
// build machine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will this look better?
// This is the embedded string. We should normalize it on forward slash, so the file won't be different according to
// build machine.
var appBinaryFilePath = string.Join("/",
new[] {
".store",
PackageId.ToLowerInvariant(),
PackageVersion,
PackageId.ToLowerInvariant(),
PackageVersion,
"tools",
targetFramework.GetShortFolderName(),
"any",
ToolEntryPoint});
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed as above
var packagedShimOutputDirectoryAndRid = Path.Combine(PackagedShimOutputDirectory, runtimeIdentifier); | ||
var appHostDestinationFilePath = | ||
Path.Combine(packagedShimOutputDirectoryAndRid, | ||
$"{ToolCommandName}{Path.GetExtension(resolvedApphostAssetPath)}"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I find ToolCommandName + Path.GetExtension(resolvedApphostAssetPath)
easier to read. It's also faster than string.Format. but perf isn't really my concern here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed, changed as suggested
if (File.Exists(appHostDestinationFilePath)) | ||
{ | ||
File.Delete(appHostDestinationFilePath); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is because for other contexts, Apphost.Create assumes if host is already written, it doesn't need to be regenerated?
I think it would be better to have a bool overwriteExisting
parameter on AppHost.Create.
Just so I understand, is the issue that the args to Create can change without the path changing, hence we need to recreate it every time?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct. I had a bug during my test. So I have to enforce recreate all the time. I will add bool overwriteExisting
param to AppHost.Create
@mairaw @KathleenDollard
|
/// The output Directroy path of generated shims. | ||
/// </summary> | ||
[Required] | ||
public string PackagedShimOutputDirectory { get; set; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Sorry in advance, I think this came up in the meeting, but I forget the answer.)
What's the story for signing the shims put in this directory? Are we relying on folks to repack nupkgs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PackagedShimOutputDirectory currently set to temporary folder under BaseIntermediateOutputPath. The producer (code gen in this case) need to unpack, sign and repack.
Their ask is to generate shim during publish instead of pack. We have not promised any further feature yet.
I maybe able to just move this task execution from pack to publish. However, there are some package context needed like packageid, packageversion which is used in part of the embedded path. Maybe that's fine due to they are properties. But we do need some design on that interface. #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Like I said on the overall review comment, we can talk about this separately and (maybe) do it as a follow-up. Can you file an issue to track it? #Resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
created https://github.com/dotnet/cli/issues/9288
In reply to: 188433576 [](ancestors = 188433576)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good overall. Lots of comments but they're actually pretty minor.
The signing thing is something I think might end up needing iteration, but I am fine leaving that discussion for a future change.
@@ -69,7 +69,8 @@ Copyright (c) .NET Foundation. All rights reserved. | |||
<_NativeExecutableExtension Condition="'$(_NativeExecutableExtension)' == '' and $(RuntimeIdentifier.StartsWith('win'))">.exe</_NativeExecutableExtension> | |||
|
|||
<_DotNetHostExecutableName>dotnet$(_NativeExecutableExtension)</_DotNetHostExecutableName> | |||
<_DotNetAppHostExecutableName>apphost$(_NativeExecutableExtension)</_DotNetAppHostExecutableName> | |||
<_DotNetAppHostExecutableNameWithoutExtension>apphost</_DotNetAppHostExecutableNameWithoutExtension> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm trying to understand this change. It seems the only place we use _DotNetAppHostExecutableNameWithoutExtension
is where there's a check to append .exe
if on Windows anyways (the same check that's done to set _NativeExecutableExtension
above).
Is there a reason we couldn't use _DotNetAppHostExecutableName
for GenerateShims
?
Is this just so that users can't override the extension in the task?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change it is to get a clean string "apphost" and it is only in one place. The problem I cannot use "_DotNetAppHostExecutableName" is even on Windows, GenerateShims can create Linux/mac apphost. But at that point, it is apphost.exe passed in.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I see. It's using a different RID list to check against for adding the extension. Makes sense 👍.
Actually... So no additional change on existing code. |
4f50eb2
to
a75c716
Compare
Great point, 👍 to doing nothing in this case. |
OK, Do nothing when it isn't sensible |
@@ -49,6 +49,7 @@ Copyright (c) .NET Foundation. All rights reserved. | |||
<PropertyGroup Condition="'$(NuGetBuildTasksPackTargets)' == '' AND '$(ImportNuGetBuildTasksPackTargetsFromSdk)' != 'false'"> | |||
<IncludeBuildOutput Condition=" '$(PackAsTool)' == 'true' ">false</IncludeBuildOutput> | |||
<PackageType Condition=" '$(PackAsTool)' == 'true' ">DotnetTool</PackageType> | |||
<RuntimeIdentifiers Condition=" '$(PackAsTool)' == 'true' ">$(RuntimeIdentifiers);$(PackAsToolShimRuntimeIdentifiers)</RuntimeIdentifiers> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe this and the 2 options above it are out of place in the property group. I believe the intention here is that there is a NuGet package for the build tasks that can override the pack targets that come from the SDK. As is, if that is used, then it will break packing tools. I think these should be set unconditionally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(By "unconditionally", I really mean conditioned only on PackAsTool == true.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean like that?
<!-- Import targets from NuGet.Build.Tasks.Pack package/Sdk -->
<PropertyGroup Condition="'$(NuGetBuildTasksPackTargets)' == '' AND '$(ImportNuGetBuildTasksPackTargetsFromSdk)' != 'false'">
<NuGetBuildTasksPackTargets Condition="'$(IsCrossTargetingBuild)' == 'true'">$(MSBuildThisFileDirectory)..\..\NuGet.Build.Tasks.Pack\buildCrossTargeting\NuGet.Build.Tasks.Pack.targets</NuGetBuildTasksPackTargets>
<NuGetBuildTasksPackTargets Condition="'$(IsCrossTargetingBuild)' != 'true'">$(MSBuildThisFileDirectory)..\..\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tasks.Pack.targets</NuGetBuildTasksPackTargets>
<ImportNuGetBuildTasksPackTargetsFromSdk>true</ImportNuGetBuildTasksPackTargetsFromSdk>
</PropertyGroup>
<PropertyGroup>
<IncludeBuildOutput Condition=" '$(PackAsTool)' == 'true' ">false</IncludeBuildOutput>
<PackageType Condition=" '$(PackAsTool)' == 'true' ">DotnetTool</PackageType>
<RuntimeIdentifiers Condition=" '$(PackAsTool)' == 'true' ">$(RuntimeIdentifiers);$(PackAsToolShimRuntimeIdentifiers)</RuntimeIdentifiers>
</PropertyGroup>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I read this condition as"if there is no override pack target use Pack target from nuget's pack", which is most of the case. PackAsTool rely on nuget's Pack target, so it needs to be set only when use nuget's pack version.
However, if we think TfmSpecificPackageFile
is a public interface that custom pack target should implement, We can remove the condition on ImportNuGetBuildTasksPackTargetsFromSdk
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The whole point of that condition is to allow you to get nuget pack targets out of band from SDK. See #888.
Suppose the user wants to try a version with a bug fix over what's in the SDK. This should not prevent them from packing a tool with that configuration.
Also, this file, sdk/sdk.targets (main entry point) is also the wrong place to put code to handle specific scenarios. This should be closer to where PackAsTool is implemented.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exactly as what you said. If I move these props to toolpack, it will fail on muti targeting. Microsoft.NET.Sdk.Common.targets is imported by both. So it is the right place.
@@ -43,6 +43,12 @@ Copyright (c) .NET Foundation. All rights reserved. | |||
<DotnetCliToolTargetFramework>netcoreapp$(BundledNETCoreAppTargetFrameworkVersion)</DotnetCliToolTargetFramework> | |||
</PropertyGroup> | |||
|
|||
<PropertyGroup> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nguerrera so I moved them here
I think all issues are resolved by now |
Add PackageToolShimRuntimeIdentifiers property to the csproj. And the result nupkg will have shims which can be signed.