why restore folder has different structure between nuget global packages folder and nuget install

[Perry004]

When I used VS IDE install a nuget package and enter the global nuget cache, its structure is like C:\Users\xxx\.nuget\packages\castle.core\4.4.0.

However, when I used command line like nuget install castle.core -Version 4.4.0 -Source C:\LocalNuGetPackages -OutputDirectory C:\packages.

The unpack structure is like C:\packages\castle.core.4.4.0.

$(PackageId)\$(PackageVersion)\ and $(PackageId).$(PackageVersion)\.

Why the structure is different and can it be the same? The same structure could help us including anyone who faces the same situation in the future have a good user experience.

[Rand-Random]

@Perry004
Thanks for opening this issues.

Just want to add the Stackoverflow Post, where I describe why I need this.
https://stackoverflow.com/questions/66119194

[zivkan]

The answer to "why" is because the folder layout you see when doing "nuget.exe install" was the only folder layout when NuGet was originally created. I think it was when PackageReference was added, the global package folder layout was changed because for performance reasons, and the global packages folder is considered a location that NuGet "owns", so is acceptable to modify in incompatible ways. I wasn't in the team at the time, but I think I remember others telling me that some customers did find this impactful and complained about the change. But since this is a location that only Nuget is intended to use, it was considered acceptable, given the benefits it provided.

However, customers use "nuget.exe install" in build scripts, so changing the layout would break their scripts if the layout changed between different nuget versions. Note that non-SDK style projects using packages.config (rather than PackageReference) still use the old "v2" folder layout for the solution packages folder.

nuget.exe has add and init commands which will expand a nupkg in NuGet's "v3" folder layout. Unfortunately these commands do not have equivalents in the dotnet cli, but there might be issues already asking for it to be ported. However, you need the nupkgs downloaded locally already. I can't think of a one-line command to do this, other than nuget install into a temporary location, then use your favorite scripting language to find all the *.nupkg files under the temporary install location, then script nuget.exe add on them (or copy them all to a single directory, and then use nuget.exe init.

The NuGet.Client repo creates a "bootstrap" project that downloads and extracts packages to a specific location, and it avoids target framework compatibility checks, just as nuget install does, using PackageDownload, rather than PackageReference. However, since PackageDownload ignores target frameworks, it doesn't download dependencies since a package supporting multiple target frameworks might have different dependencies depending on framework. I'm actually not sure what nuget.exe does in this scenario.

Since the question doesn't explain the problem trying to be solved, I don't know what the best suggestion I can provide is, or if I should suggest upvoting an existing issue and closing this as a duplicate, or if we should open a new feature request.

[Rand-Random]

@zivkan
Thanks for your in-depth explanation.

This was the useful info, I needed to solve my issue.

nuget.exe has add and init commands which will expand a nupkg in NuGet's "v3" folder layout. Unfortunately these commands do not have equivalents in the dotnet cli, but there might be issues already asking for it to be ported. However, you need the nupkgs downloaded locally already. I can't think of a one-line command to do this, other than nuget install into a temporary location, then use your favorite scripting language to find all the *.nupkg files under the temporary install location, then script nuget.exe add on them (or copy them all to a single directory, and then use nuget.exe init.

It seems like I was looking for this command
(I am using castle.core here just like @Perry004 did, though in my problem that wouldn't be the case)

nuget add "C:\LocalNuGetPackages\castle.core.4.4.0.nupkg" -Source C:\packages -expand

Instead of

nuget install castle.core -Version 4.4.0 -Source C:\LocalNuGetPackages -OutputDirectory C:\packages

As the -Source parameter implies my *.nupkg is already locally, so I didn't need to have a one-line command to install and add.

Still untested, but on the first glance it looks promising

---

I will try to explain why I am looking for this command at the first place to clear up the XY problem at hand.
It is going to be long so bear with me, sorry.

Everything I am about to explain runs locally, no build-server, git action or what not, just plain old Visual Studio.
All NuGet packages involved are our personal packages, and are used internally only and never would be published to NuGet.

I have currently 3 solutions, 4 and 5 are in the making, all those solutions share a common solution.
All my projects are targeting multiple frameworks, which introduced issues with referencing the correct *.dlls to my project.

In the good old days where I didn't have multi-target projects, I just simply referenced the dlls directly.
Now with the swith to multi-target I was faced with the "challenge" which dll needs to be referenced depending on the target the projects have.

eg. Project A is targeting .net47 and .netstandard2.1 and Project B is targeting .net5.0, I would like to have Project B getting the dll for the target .netstandard2.1.

I of course could have edited the path to contain the desired target-framework, like the following.

Prior to multi-target, my reference in Project B would look like this

  <ItemGroup>
    <Reference Include="ProjectA">
      <HintPath>C:\repo\Solution 1\Project A\bin\Debug\ProjectA.dll</HintPath>
    </Reference>
  </ItemGroup>

Now I could have changed it to this

  <ItemGroup>
    <Reference Include="ProjectA">
      <HintPath>C:\repo\Solution 1\Project A\bin\Debug\netstandard2.1\ProjectA.dll</HintPath>
    </Reference>
  </ItemGroup>

But I refused to do this manually and wanted "magic" to happen which picked the ideal target it self.

At first I simply added the projects I was referencing into the solution as existing projects and than used project references instead of the old way to reference the dlls.

This seemed to work at first but I was running into more and more problems each day eg.

1. The referenced projects where part of the build process, which increased the build time
2. The referenced projects clustered the error lists with warnings, not all projects belong to me so I couldn't just fix those warning
3. The projects where used in multiple solutions and Visual Studio again and again blocked files and which were open in different instances of Visual Studio

So all in all a very unpleasant working environment.

My next idea was to use the magic of NuGet to pick the right target-framework.

So I enabled GeneratePackageOnBuild for my projects by adding this to my Directory.Build.props

<PropertyGroup>
  <GeneratePackageOnBuild>true</GeneratePackageOnBuild> 
  <PackageOutputPath>C:\LocalNuGetPackages</PackageOutputPath>
</PropertyGroup>

Every branch/trunk would now ouput NuGet packages with their respecting Version Number, each branch/trunk has a unique Version Number in our enviorment, but I didn't want to be notified about updates of branches with higher Version Numbers if I am currently working on a lower version.

So I added the following, notice the [] to specify a fixed version

<PropertyGroup>
   <MyLocalNugetVersion>[$(VersionPrefix)]</MyLocalNugetVersion>
</PropertyGroup>

Which declares MyLocalNugetVersion to be identically as the Version I am currently working at.

In a NuGet.config I have added my LocalNuGetPackages as a packageSource, addtionally I specified where to unpack the packages.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
	<packageSources>
		<add key="LocalSources" value="C:\LocalNuGetPackages"/>
	</packageSources>
	<config>
                <!-- according to google search, one is used for packages.config and the other for PackageReferences -->
		<add key="globalPackagesFolder" value="C:\packages" />
		<add key="repositoryPath" value="C:\packages" />
	</config>
</configuration>

Finally I could install the package into Project B from my local package source, after the install I have to manually change the version number to the information I have added in Directory.Build.props

<ItemGroup>
  <PackageReference Include="Solution1.ProjectA" Version="$(MyLocalNugetVersion)" />
</ItemGroup>

Everything seemed to work nicely, for the first days at least.
NuGet did a great job to solve all the issues I previously had with the project references.

But there was 1 problem, atleast for now, that I still needed to solve.

While building Project A the NuGet packages where re-created because of GeneratePackageOnBuild but sadly Project B which referenced this NuGet package failed to pick up that the NuGet package has changed and still worked with the old source code.

While testing we figured out we could delete the files from C:\packages\ProjectB\$(MyLocalNugetVersion) and upon Rebuilding Project B the NuGet package was unpacked once again with the new source code.

To not manually delete the files after each build, we added a <Target AfterTargets="Build"> task into our Directory.Build.props which cleans the C:\packages\$(PackageId)\$(PackageVersion) directory.

I want to replace the cleaning to the actually unpacking of the NuGet package.

So there were we are today, where I tried to unpack my package with

nuget install $(PackageId) -Version $(PackageVersion) -Source C:\LocalNuGetPackages -OutputDirectory C:\packages

Which has the goal of "updating" the unpacked stuff everytime a build happens.

Temporarly I tried to work with auto-incrementing the version number with each build so that NuGet can again make magic happen to always get the lastest version with the new source code from the latest build.

So I changed my Directory.Build.props to this

<PropertyGroup>
	<VersionPrefix>1.0.0.0</VersionPrefix>
	<VersionSuffix>pre-$([System.DateTime]::UtcNow.Ticks.ToString())</VersionSuffix>
	<MyLocalNugetVersion>$(VersionPrefix)-*</MyLocalNugetVersion>
</PropertyGroup>

But sadly this doesn't work at all.
The gerenated NuGet packages contain dependencies of NuGet packages which simply don't exist because it seems it evaluates the UtcNow.Ticks for every dependency individually so it is different every time, I could change the Ticks to a more "constant" value but what should I pick?

Every minute? - but the build would need to finish in the same minute
Every hour? - so I could only build my project every hour for the magic to work

Both those aren't ideal.

To overcome this I thought about calling dotnet pack for the whole soltution and pass the version suffix with the argument --version-suffix pre-$([System.DateTime]::UtcNow.Ticks.ToString()), which I guess would solve the problem, but I don't want to pack the whole solution after each projects build, because <Target AfterTargets="Build"> runs on each individual project that gets build and I couldn't find a way to only do it once per build process.

Imagine a solution with 50+ projects and I would hit rebuild soltution now dotnet pack would run atleast 50 times, but I would only want to run it once after everything has been successfully build.

Which I am currently asking here if that is possible:
https://stackoverflow.com/questions/66137360

Hope I explained my XY Problem.

[Rand-Random]

@zkat @zivkan
Should the request that the intall command should give an option to install in the v3 folder layout be re-created in a new issue?
I believe it is still a valid request for issues.

I understand that my plee for help got moved to discusssion, but the request about install should be treated differently IMHO.

[zivkan]

yes. I actually don't remember that there was a request to make nuget install use the v3 folder, so a short, clear feature request is good, making it more clear for others who want the same feature to upvote it.

[Rand-Random]

@zivkan
Well lets say it like this, the question "why isn't it producing v3 layout" imho implied pls give an option to do it in v3 style.

[zivkan]

We get many questions from customers who are unaware on how to use the product, so I don't think it's fair to assume that every question has an implicit feature request. Some times when customers find out the product doesn't work the way they were originally attempting to use it, they find another way to achieve their goal and are no longer interested in the product working the way they originally asked about.

Also as a software developer yourself, I hope you appreciate that any feature request that is not clear makes it hard to know when an implementation is finished and the request can be closed. Also known as "definition of done".

Also consider from other customers point of view. If they want the same feature, but neither the title of the issue, nor the body, make it clear what is being requested, it's hard for them to know to upvote the issue.

I don't think it's fair to say that every question should be implicitly a feature request that is only closed once implemented.

[Rand-Random]

Just for reference: #10593