error NU1403: The package Microsoft.CSharp.4.0.1 sha512 validation failed.

[ctaggart]

I'm trying to add nuget 4.9 lock files, but I get the error:

/src/Serialization.Test/Froto.Serialization.Test.fsproj : error NU1403: The package Microsoft.CSharp.4.0.1 sha512 validation failed. The package is different than the last restore. [/src/Froto.sln]

Steps to reproduce:
Checkout the lock branch from ctaggart/froto#104, then restore using docker:

docker run -it --rm -v ${PWD}:/src -w /src mcr.microsoft.com/dotnet/core/sdk:2.2.105 bash
dotnet restore Froto.sln

My hunch is that the nuget is NuGetFallbackFolder is causing problems. It includes nuget packages that don't match NuGet.org. WHY DOES IT HAVE PACKAGES THAT DON'T MATCH NUGET.ORG?

The Froto.Serialization.Test.fsproj has a lock to:

      "Microsoft.CSharp": {
        "type": "Transitive",
        "resolved": "4.0.1",
        "contentHash": "eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg==",

It matches what is downloaded from nuget.org:

C:\Users\taggac\.nuget\packages\microsoft.csharp\4.0.1\.nupkg.metadata
{
  "version": 1,
  "contentHash": "eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg=="
}

The docker image NuGetFallbackFolder contentHash does not match:

root@0366cb41b8b4:/src# cat /usr/share/dotnet/sdk/NuGetFallbackFolder/microsoft.csharp/4.0.1/.nupkg.metadata
{
  "version": 1,
  "contentHash": "17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A=="
}

At least the NuGetFallbackFolder on my Windows PC matches the docker image:

C:\Program Files\dotnet\sdk\NuGetFallbackFolder\microsoft.csharp\4.0.1\.nupkg.metadata
{
  "version": 1,
  "contentHash": "17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A=="
}

Is there a way to disable NuGetFallbackFolder from the lock check? I tried this without luck:

<DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>

If you are going to continue shipping NuGet packages in NuGetFallbackFolder, can you make sure they match NuGet.org?

[ctaggart]

It looks like it may not be the NuGetFallbackFolder, but the contentHash in docker is not producing the same thing as on Windows for all packages. It is different on these packages:

ctaggart/froto@83f9bd3

I think contentHash implementation has a bug.

https://github.com/NuGet/Home/wiki/Nupkg-Metadata-File
Ashish Jain @jainaashish

[nkolev92]

The fallback folder issues are due to #7414.

It's a bug on Dotnet CLI side with the LZMA tool. The bug summarizes the findings the actionable. Unfortunately there will be no changes to the LZMA tool as it's being completely done away with in .NET Core 3.0.

What you are articulating sounds like the bug in #7682, which should have been fixed in 2.2.104. Are you using the same version everywhere?

The property you are using to disable the fallback folder seems right:

<DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>

But you need to make sure the lock file is generated with that property.

Also, are you using the same version of the dotnet sdk on all instances?

[ctaggart]

No matter what I do the lock file that is generated while in a docker sdk container contains 17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A== as the contentHash when it should be eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg==.

I created a little app to troubleshoot this:
https://github.com/ctaggart/NuGet.Client/blob/PrintContentHash/PrintContentHash/Program.cs

PS C:\Users\taggac\github\NuGet.Client> docker run -it --rm -v ${PWD}:/src -w /src mcr.microsoft.com/dotnet/core/sdk:2.2.105 bash
root@fd158f92f5d6:/src# dotnet run -p PrintContentHash -- testpackages Microsoft.CSharp 4.0.1
eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg==
eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg==

[nkolev92]

@ctaggart
If possible can you provide the docker file you used for testing, so we don't have to go through the same exercise?

Thanks

[ctaggart]

@nkolev92, steps to reproduce:

git clone git@github.com:ctaggart/froto.git
git checkout lock
docker run -it --rm -v ${PWD}:/src -w /src mcr.microsoft.com/dotnet/core/sdk:2.2.105 bash
git rm **/packages.lock.json
dotnet restore Froto.sln

Various other troubleshooting:

git clean -xdf

dotnet nuget locals all --clear

export NUGET_FALLBACK_PACKAGES=$HOME/.nuget/fallbackpackages/
mkdir -p $NUGET_FALLBACK_PACKAGES

export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1
rm -rf /usr/share/dotnet/sdk/NuGetFallbackFolder/

Nothing has worked. I've also tried sdk:3.0.100-preview3.

Worth noting that the Directory.Build.props contains:

    <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
    <RestoreLockedMode>true</RestoreLockedMode>

[ctaggart]

I found this pretty strange. I was expecting these to hashes to match.

root@fd158f92f5d6:/src# dotnet run -p PrintContentHash -- /usr/share/dotnet/sdk/NuGetFallbackFolder Microsoft.CSharp 4.0.1
17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A==
lIwCKwiI4K/KZHVAMki2zuq9MpwOP/HP2vcbpe1pKo6KHKC8R8zXfA2Z8VNcqXs0DDPMJfrk7JBcMNXLK3nJLg==

root@fd158f92f5d6:/src# cat /usr/share/dotnet/sdk/NuGetFallbackFolder/microsoft.csharp/4.0.1/.nupkg.metadata
{
  "version": 1,
  "contentHash": "17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A=="
}

On Windows:

PS C:\users\taggac\github\NuGet.Client> dotnet run -p PrintContentHash -- 'C:\Program Files\dotnet\sdk\NuGetFallbackFolder' Microsoft.CSharp 4.0.1
17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A==
b9YcH7cOd8y0zsL8UqGh633QqxHf8A4Ehlia9HXvVBKvpURwF8iCYCllucq07uDQEYiSR95gZStAszIXYoO43Q==

[jainaashish]

@ctaggart it will also be worth checking what's the value in *.nupkg.sha512 file on the system.

And btw what's this value lIwCKwiI4K/KZHVAMki2zuq9MpwOP/HP2vcbpe1pKo6KHKC8R8zXfA2Z8VNcqXs0DDPMJfrk7JBcMNXLK3nJLg==?? and b9YcH7cOd8y0zsL8UqGh633QqxHf8A4Ehlia9HXvVBKvpURwF8iCYCllucq07uDQEYiSR95gZStAszIXYoO43Q==

[nkolev92]

@ctaggart
Those wouldn't match because of the bug in the LZMA.

The first hash value is the one that's written in the fallback folder prior to the LZMA is run.
The 2nd hash value is different because the package itself is different due to the LZMA scrambling it around.

[nkolev92]

The way that the GenerateNupkgMetadataFile method is the following.

public static void GenerateNupkgMetadataFile(string nupkgPath, string installPath, string hashPath, string nupkgMetadataPath)

1. If the package is signed, generate a content hash.
2. If it's not, fallback & use the value from the hashPath.

Now prior to the LZMA zips them, the package is signed.
The contentHash and the sha512 will be different.

[nkolev92]

It seems to me like the .nupkg.metadata file in the fallback folder has the same value as the sha512.

Do you see any issues if you completely disable the fallback folder?

[slang25]

I'm seeing this for Microsoft.NETCore.Targets.1.1.0 (and other packages), I've disabled the fallback folder, and still have the issue. I've also pinned the dotnet sdk to 2.2.105 in global.json.

Locally I'm running macOS (and generating the lock files), it's building and failing on a docker container.

[ctaggart]

The expectation is that I am using that same dotnet sdk version, such as 2.2.105, I can lock the depdendencies on one system (Windows) and then restore the depdendencies on another system (Docker). That doesn't work and I haven't found a work-a-round.

dotnet sdk 2.2.105 on Windows produces this:
ctaggart/froto@2c6837c#diff-ce2cbfcdedae8d0b839697df83de2102R64
Serialization.Test/packages.lock.json

      "Microsoft.CSharp": {
        "type": "Transitive",
        "resolved": "4.0.1",
        "contentHash": "eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg=="

That contentHash ends up matching the sha512sum of the file. I go restore on a docker sdk 2.2.105 docker image using the locked files and it fails:

PS C:\Users\taggac\github\froto> docker run -it --rm -v ${PWD}:/src -w /src mcr.microsoft.com/dotnet/core/sdk:2.2.105 bash
root@d731321ca5cf:/src# dotnet restore Froto.sln /p:DisableImplicitNuGetFallbackFolder=true
  Restoring packages for /src/Parser.Test/Froto.Parser.Test.fsproj...
  Restoring packages for /src/Compiler/Froto.Compiler.fsproj...
  Installing Microsoft.NETCore.DotNetAppHost 2.1.0.
  Installing Microsoft.NETCore.DotNetHostResolver 2.1.0.
  Installing Microsoft.NETCore.DotNetHostPolicy 2.1.0.
  Installing NETStandard.Library 2.0.3.
  Installing Microsoft.NETCore.Targets 2.1.0.
  Installing Microsoft.NETCore.Platforms 2.1.0.
  Installing Microsoft.Build.Tasks.Git 1.0.0-beta-63127-02.
  Installing Microsoft.SourceLink.Common 1.0.0-beta-63127-02.
  Installing FParsec 1.0.3.
  Installing Microsoft.NETCore.App 2.1.0.
  Installing Microsoft.SourceLink.GitHub 1.0.0-beta-63127-02.
  Installing Nerdbank.GitVersioning 2.3.138.
  Installing FSharp.Core 4.5.2.
  Installing Argu 4.0.0.
/src/Compiler/Froto.Compiler.fsproj : error NU1403: The package Argu.4.0.0 sha512 validation failed. The package is different than the last restore. [/src/Froto.sln]
  Generating MSBuild file /src/Compiler/obj/Froto.Compiler.fsproj.nuget.g.props.
  Generating MSBuild file /src/Compiler/obj/Froto.Compiler.fsproj.nuget.g.targets.
  Restore failed in 5.64 sec for /src/Compiler/Froto.Compiler.fsproj.
  Restoring packages for /src/Parser/Froto.Parser.fsproj...
  Installing Microsoft.NETCore.Platforms 1.1.0.
/src/Parser/Froto.Parser.fsproj : error NU1403: The package Microsoft.NETCore.Platforms.1.1.0 sha512 validation failed. The package is different than the last restore. [/src/Froto.sln]
  Generating MSBuild file /src/Parser/obj/Froto.Parser.fsproj.nuget.g.props.
  Generating MSBuild file /src/Parser/obj/Froto.Parser.fsproj.nuget.g.targets.
  Restore failed in 641.24 ms for /src/Parser/Froto.Parser.fsproj.
  Restoring packages for /src/Serialization.Test/Froto.Serialization.Test.fsproj...
  Installing System.Xml.XmlSerializer 4.0.11.
  Installing System.Collections.Concurrent 4.0.12.
  Installing System.Globalization.Extensions 4.0.1.
  Installing System.Private.DataContractSerialization 4.1.1.
  Installing System.Security.Cryptography.Encoding 4.0.0.
  Installing System.Runtime.Numerics 4.0.1.
  Installing System.Security.Cryptography.Primitives 4.0.0.
  Installing runtime.native.System.Security.Cryptography 4.0.0.
  Installing System.Collections.Specialized 4.0.1.
  Installing System.Collections.NonGeneric 4.0.1.
  Installing System.ComponentModel.Primitives 4.1.0.
  Installing System.ComponentModel 4.0.1.
  Installing System.Xml.XmlDocument 4.0.1.
  Installing System.Xml.XPath 4.0.1.
  Installing Microsoft.Win32.Primitives 4.0.1.
  Installing Microsoft.Win32.Registry 4.0.0.
  Installing Unquote 4.0.0.
  Installing FsUnit.xUnit 3.2.0.
  Installing System.Threading.ThreadPool 4.0.10.
  Installing Microsoft.NETCore.App 2.0.0.
  Installing System.Runtime.Handles 4.0.1.
  Installing runtime.native.System 4.0.0.
  Installing System.Collections.Immutable 1.2.0.
  Installing System.Threading.Tasks.Extensions 4.0.0.
  Installing System.IO.FileSystem.Primitives 4.0.1.
  Installing System.Reflection.Primitives 4.0.1.
  Installing System.Reflection.Emit.ILGeneration 4.0.1.
  Installing System.Reflection.Emit.Lightweight 4.0.1.
  Installing System.Reflection.Emit 4.0.1.
  Installing System.Diagnostics.Tools 4.0.1.
  Installing Microsoft.NETCore.Targets 1.0.1.
  Installing System.Runtime.Serialization.Json 4.0.2.
  Installing System.Security.Cryptography.Algorithms 4.2.0.
  Installing xunit 2.4.0.
  Installing xunit.runner.visualstudio 2.4.0.
  Installing System.ComponentModel.TypeConverter 4.1.0.
  Installing System.Xml.XPath.XmlDocument 4.0.1.
  Installing NHamcrest 2.0.1.
  Installing Microsoft.NETCore.Platforms 2.0.0.
  Installing System.IO.FileSystem 4.0.1.
  Installing Microsoft.NETCore.DotNetHostPolicy 2.0.0.
  Installing xunit.analyzers 0.10.0.
  Installing xunit.assert 2.4.0.
  Installing System.Diagnostics.Process 4.1.0.
  Installing xunit.core 2.4.0.
  Installing Microsoft.NET.Test.Sdk 15.0.0.
  Installing Microsoft.NETCore.DotNetHostResolver 2.0.0.
  Installing xunit.extensibility.execution 2.4.0.
  Installing System.Runtime.InteropServices 4.1.0.
  Installing xunit.extensibility.core 2.4.0.
  Installing Microsoft.TestPlatform.TestHost 15.0.0.
  Installing System.Reflection.TypeExtensions 4.1.0.
  Installing Microsoft.NETCore.DotNetAppHost 2.0.0.
  Installing xunit.abstractions 2.0.2.
  Installing Newtonsoft.Json 9.0.1.
  Installing System.Runtime.InteropServices.RuntimeInformation 4.0.0.
  Installing System.Diagnostics.TraceSource 4.0.0.
  Installing System.ComponentModel.EventBasedAsync 4.0.11.
  Installing System.Diagnostics.TextWriterTraceListener 4.0.0.
  Installing Microsoft.TestPlatform.ObjectModel 15.0.0.
  Installing System.Runtime.Loader 4.0.0.
  Installing System.Reflection.Metadata 1.3.0.
  Installing System.Diagnostics.Tracing 4.1.0.
  Installing System.ObjectModel 4.0.12.
  Installing System.Reflection.Extensions 4.0.1.
  Installing System.Threading.Thread 4.0.0.
  Installing System.Resources.ResourceManager 4.0.1.
  Installing System.Xml.ReaderWriter 4.0.11.
  Installing System.Diagnostics.Debug 4.0.11.
  Installing System.Collections 4.0.11.
  Installing Microsoft.CSharp 4.0.1.
  Installing System.Linq 4.1.0.
  Installing System.Reflection 4.1.0.
  Installing System.Runtime.Serialization.Primitives 4.1.1.
  Installing System.Globalization 4.0.11.
  Installing System.Dynamic.Runtime 4.0.11.
  Installing System.Linq.Expressions 4.1.0.
  Installing System.Text.RegularExpressions 4.1.0.
  Installing System.Text.Encoding.Extensions 4.0.11.
  Installing System.Text.Encoding 4.0.11.
  Installing System.Runtime 4.1.0.
  Installing System.IO 4.1.0.
  Installing System.Threading 4.0.11.
  Installing System.Threading.Tasks 4.0.11.
  Installing System.Runtime.Extensions 4.1.0.
  Installing System.Xml.XDocument 4.0.11.
/src/Parser.Test/Froto.Parser.Test.fsproj : error NU1403: The package Microsoft.CSharp.4.0.1 sha512 validation failed. The package is different than the last restore. [/src/Froto.sln]
/src/Serialization.Test/Froto.Serialization.Test.fsproj : error NU1403: The package Microsoft.CSharp.4.0.1 sha512 validation failed. The package is different than the last restore. [/src/Froto.sln]
  Generating MSBuild file /src/Parser.Test/obj/Froto.Parser.Test.fsproj.nuget.g.props.
  Generating MSBuild file /src/Serialization.Test/obj/Froto.Serialization.Test.fsproj.nuget.g.props.
  Generating MSBuild file /src/Parser.Test/obj/Froto.Parser.Test.fsproj.nuget.g.targets.
  Generating MSBuild file /src/Serialization.Test/obj/Froto.Serialization.Test.fsproj.nuget.g.targets.
  Restore failed in 10.84 sec for /src/Parser.Test/Froto.Parser.Test.fsproj.
  Restore failed in 4.3 sec for /src/Serialization.Test/Froto.Serialization.Test.fsproj.
  Restoring packages for /src/Serialization/Froto.Serialization.fsproj...
/src/Serialization/Froto.Serialization.fsproj : error NU1403: The package Microsoft.NETCore.Platforms.1.1.0 sha512 validation failed. The package is different than the last restore. [/src/Froto.sln]
  Generating MSBuild file /src/Serialization/obj/Froto.Serialization.fsproj.nuget.g.props.
  Generating MSBuild file /src/Serialization/obj/Froto.Serialization.fsproj.nuget.g.targets.
  Restore failed in 13.86 ms for /src/Serialization/Froto.Serialization.fsproj.

I begin troubleshooting by recreating the lock files on the docker image and it says the contentHash is:

ctaggart/froto@83f9bd3#diff-ce2cbfcdedae8d0b839697df83de2102R67

      "Microsoft.CSharp": {
        "type": "Transitive",
        "resolved": "4.0.1",
        "contentHash": "17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A==",

Global Packages in Windows

NUPKG=/mnt/c/Users/taggac/.nuget/packages/microsoft.csharp/4.0.1/microsoft.csharp.4.0.1.nupkg
SHA512=`cat $NUPKG.sha512`
echo $SHA512
eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg==

SHA512=`sha512sum "$NUPKG" | cut -d " " -f 1 | xxd -r -p | base64 -w 0`
echo $SHA512
eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg==

dotnet run -p PrintContentHash -- 'C:\Users\taggac\.nuget\packages' Microsoft.CSharp 4.0.1
nupkgMetadata.ContentHash
eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg==
packageReader.GetContentHash
eEgt/ktABPBa/1xBex3M6b7C4dYsGQ0HWuwe9Xhflgu4cvn4a5VvFSLj5vyTKkp+hKrVpTRdqTXDTAxsIL1Reg==

NuGetFallbackFolder in Windows

NUPKG=/mnt/c/Program\ Files/dotnet/sdk/NuGetFallbackFolder/microsoft.csharp/4.0.1/microsoft.csharp.4.0.1.nupkg
SHA512=`cat $NUPKG.sha512`
echo $SHA512
17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A==

SHA512=`sha512sum "$NUPKG" | cut -d " " -f 1 | xxd -r -p | base64 -w 0`
echo $SHA512
b9YcH7cOd8y0zsL8UqGh633QqxHf8A4Ehlia9HXvVBKvpURwF8iCYCllucq07uDQEYiSR95gZStAszIXYoO43Q==

dotnet run -p PrintContentHash -- 'C:\Program Files\dotnet\sdk\NuGetFallbackFolder' Microsoft.CSharp 4.0.1
nupkgMetadata.ContentHash
17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A==
packageReader.GetContentHash
b9YcH7cOd8y0zsL8UqGh633QqxHf8A4Ehlia9HXvVBKvpURwF8iCYCllucq07uDQEYiSR95gZStAszIXYoO43Q==

NuGetFallbackFolder in Docker

NUPKG=/usr/share/dotnet/sdk/NuGetFallbackFolder/microsoft.csharp/4.0.1/microsoft.csharp.4.0.1.nupkg
SHA512=`cat $NUPKG.sha512`
echo $SHA512
17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A==

SHA512=`sha512sum "$NUPKG" | cut -d " " -f 1 | xxd -r -p | base64 -w 0`
echo $SHA512
lIwCKwiI4K/KZHVAMki2zuq9MpwOP/HP2vcbpe1pKo6KHKC8R8zXfA2Z8VNcqXs0DDPMJfrk7JBcMNXLK3nJLg==

dotnet run -p PrintContentHash -- '/usr/share/dotnet/sdk/NuGetFallbackFolder' Microsoft.CSharp 4.0.1
nupkgMetadata.ContentHash
17h8b5mXa87XYKrrVqdgZ38JefSUqLChUQpXgSnpzsM0nDOhE40FTeNWOJ/YmySGV6tG6T8+hjz6vxbknHJr6A==
packageReader.GetContentHash
lIwCKwiI4K/KZHVAMki2zuq9MpwOP/HP2vcbpe1pKo6KHKC8R8zXfA2Z8VNcqXs0DDPMJfrk7JBcMNXLK3nJLg==

How do I get the contentHash?

I was expecting packageReader.GetContentHash to return the contentHash value that see in the nupkgMetadata, but looks like it is the hash of the entire nuget package. Any idea how I modify this to get the value I'm looking for?

https://github.com/ctaggart/NuGet.Client/blob/PrintContentHash/PrintContentHash/Program.cs#L46-L50

            using (var stream = File.OpenRead(zipPath))
            using (var packageReader = new PackageArchiveReader(stream, leaveStreamOpen: true))
            {
                var contentHash = packageReader.GetContentHash(CancellationToken.None);
Console.WriteLine(contentHash);

[slang25]

Update from me:

I generate a lock file that then goes on to work in CI (docker for me) after deleting the package cache folder.

[ctaggart]

My mistake was not locking down my NuGet sources with a NuGet.Config, which had this one enabled: Microsoft Visual Studio Offline Packages, but probably better to lock down your the sources with NuGet.Config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>
</configuration>

The DisableImplicitNuGetFallbackFolder of true is required for the restore to work correctly in docker. Here is my working Directory.Build.props:

<Project>
  <PropertyGroup>
    <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
    <RestoreLockedMode>true</RestoreLockedMode>
    <NoWarn>NU1603</NoWarn>
    <DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>
  </PropertyGroup>
</Project>

I then best to clear all the local nuget caches and regenerate the lock files.

dotnet nuget locals all --clear
git clean -xfd
git rm **/packages.lock.json -f
dotnet restore Froto.sln

It is working now for me. Some of the hash's don't make sense to me still, but at least it is consistent across platforms when you take the NuGetFallbackFolder's completely out of the picture.

[nkolev92]

Thanks for following up @ctaggart

We are sorry that we could not make this easier.

Why did the Microsoft Visual Studio Offline Packages cause issues for you?

[ctaggart]

@nkolev92 I just tested with it enabled and it doesn't appear to have been a park of the issue. Thanks for clearing that up.

[nkolev92]

Oh, perfect. That's good then.
Just wanted to make sure there are not any lingering issues that we might miss.

[RehanSaeed]

If I'm using the latest .NET Core 3.0.100 SDK, should this all work as advertised with just this in my Directory.Build.props?

<Project>
  <PropertyGroup>
    <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
  </PropertyGroup>
</Project>

[slang25]

It should, but you may need to clear out any packages from you cache that would have been restored with an old nuget. Going forward, you won't get into this state.

[RehanSaeed]

This does not seem to be the case. I get different hashes when I build with:

• .NET Core 3.0.100 CLI
• Visual Studio 16.3.8

It seems you still require:

    <DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>

[mattwoberts]

Hi,

I'm using dotnet SDK 3.0, and I'm introducing a lock file so that I can cache the nuget restore on azure pipelines. I've added <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile> to my project files, and checked the resulting package.lock.json files into source control. All works locally.

When I push this, the azure pipelines restore task fails, and I see lots of issues e.g.:

error NU1403: Package content hash validation failed for runtime.native.System.IO.Compression.4.3.0.

This was running previously.

My nuget restore task is configured to use feeds in my nuget.config file, which I do have in the root of the project:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
	<activePackageSource>
	  <add key="All" value="(Aggregate source)" />
	</activePackageSource>
	<packageSources>
	  <add key="nuget.org" value="https://www.nuget.org/api/v2/" />
	  <add key="myget" value="https://www.myget.org/F/[redacted]/index.json" />
	</packageSources>
	<disabledPackageSources />
</configuration>

Can anyone advise?

[NicolasDorier]

Had this problem with 3.1.100.

I deleted my nuget cache dotnet nuget locals all --clear, then deleted the lock file, and regenerated it.

This fixed the issue.

[NicolasDorier]

Reporting back:
Had this problem again building my app on 3.1.100 on appveyor where setting DisableImplicitNuGetFallbackFolder to true fixed. I thought this problem was fixed for good in 3.x...

DisableImplicitNuGetFallbackFolder was not needed when building inside docker container. Only on appveyor. (win64)

[shantanu1singh]

The fix suggested by @ctaggart worked for me too but doesn't deleting the packages.lock.json file while building the docker image defeat the purpose of trying to enforce the same package versions everywhere?

Wouldn't NuGet pull new versions of packages (if they were available) once the lock file is deleted?

[dazinator]

I've tried everything in this thread, Yet when my build runs on azure pipelines I get these errors:

Package content hash validation failed for runtime.native.System.Security.Cryptography.Apple.4.3.0. Expected: jwjwlEL0Elv6gwoyaokRn12nv/JE+UW/DXJEbzhjCPvGbef36StnHKc9XaZD/rGWqYicrphZ7eumR/jdmNcjRg== Actual: DloMk88juo0OuOWr56QG7MNchmafTLYWvABy36izkrLI5VledI0rq28KGs1i9wbpeT9NPQrx/wTf8U2vazqQ3Q==
Package content hash validation failed for runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple.4.3.0. Expected: Kh9W4agE0r/hK8AX1LvyQI2NrKHBL8pO0gRoDTdDb0LL6Ta1Z2OtFx3lOaAE0ZpCUc/dt9Wzs3rA7a3IsKdOVA== Actual: kVXCuMTrTlxq4XOOMAysuNwsXWpYeboGddNGpIgNSZmv1b6r/s/DPk0fYMB7Q5Qo4bY68o48jt4T4y5BVecbCQ==
Package content hash validation failed for System.Data.Common.4.3.0. Expected: lVldU0nX3NJ81QgIJuudcD0L2nDL099iovEA1dazVzhlxEHoKoid0wCDl7h5+SyZpMs1PbjWN+SXNOI70fp5xA== Actual: lm6E3T5u7BOuEH0u18JpbJHxBfOJPuCyl4Kg1RH10ktYLp5uEEE1xKrHW56/We4SnZpGAuCc9N0MJpSDhTHZGQ==
Package content hash validation failed for System.Threading.ThreadPool.4.3.0. Expected: SrCOmTvvOOCmJT4BBxdZcwhz6XEMsjBGZvGSVorOxdCznaUNeVotEjDfXGIZ8gcGo00qgPbTV6puHDgcFYl6Iw== Actual: k/+g4b7vjdd4aix83sTgC9VG6oXYKAktSfNIJUNGxPEj7ryEOfzHHhfnmsZvjxawwcD9HyWXKCXmPjX8U4zeSw==
Package content hash validation failed for System.Xml.XmlSerializer.4.3.0. Expected: VShQJhOxgD/5M2Z1IWm1vMaSqlbjo1zdFf8H7Ahte6bTvSUhUko/gDpAVVhGgGgTDeue4QyNg1fu1Zz2GKSEuQ== Actual: MYoTCP7EZ98RrANESW05J5ZwskKDoN0AuZ06ZflnowE50LTpbR5yRg3tHckTVm5j/m47stuGgCrCHWePyHS70Q==
1>D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.native.System.Security.Cryptography.Apple.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]

Some info about my setup:

1. using sdk version 3.1.201 in global.json
2. My pipelines yml has these tasks to do a restore using nuget which causes the failure:

- task: NuGetToolInstaller@1
    displayName: 'Install nuget'
    inputs:
      versionSpec: '5.2.0'

  - task: NuGetCommand@2   
    displayName: 'Authenticate with NuGet feed'
    inputs:
      command: custom
      arguments: sources update -Name "Libraries" -Username "this_value_could_anything" -Password "$(System.AccessToken)" -StorePasswordInClearText -ConfigFile src/NuGet.Config
      
  - task: DotNetCoreCLI@2   
    displayName: Restore
    inputs:
      command: restore
      projects: '$(solution)'
      selectOrConfig: config
      nugetConfigPath: $(Build.SourcesDirectory)/src/NuGet.config     

Note the additional authenticate step is necessary before doing the dotnet restore because my nuget.config file has an azure artifacts nuget feed in which requires authentication. If I don't add this step then the restore fails with a 401 error accessing the feed.

My NuGet.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="Libraries" value="https://pkgs.dev.azure.com/foo/_packaging/foo/nuget/v3/index.json" />   
    <add key="aspnetcore-dev" value="https://dotnet.myget.org/F/aspnetcore-dev/api/v3/index.json" protocolVersion="3" />       
  </packageSources>
</configuration>

My Directory.Build.props

    <!--See https://devblogs.microsoft.com/nuget/enable-repeatable-package-restores-using-a-lock-file/-->
    <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
    <RestoreLockedMode>true</RestoreLockedMode>
    <NoWarn>NU1603</NoWarn>
    <DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>

• I tried @ctaggart steps to no avail:

dotnet nuget locals all --clear
git clean -xfd
git rm **/packages.lock.json -f
dotnet restore src --configfile src/nuget.config

This regenerated all the lock files clean. I committed them, pushed up the changes to trigger a new build on azure pipelines and I see this:

      D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.native.System.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.native.System.IO.Compression.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.native.System.Net.Http.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.native.System.Net.Security.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.native.System.Security.Cryptography.Apple.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.native.System.Security.Cryptography.OpenSsl.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for System.Net.NetworkInformation.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for System.Private.DataContractSerialization.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for System.Security.Cryptography.Csp.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]
       D:\a\1\s\src\Foo.Server\Foo.Server.csproj : error NU1403: Package content hash validation failed for System.Security.Cryptography.OpenSsl.4.3.0. The package is different than the last restore. [D:\a\1\s\src\Foo.sln]

I have no ideas how to solve this issue. Any help much appreciated as I have lost so much time on this now.

I have checked that on my PATH I am using latest NuGet:

>nuget ?
NuGet Version: 5.2.0.6090

This is the same version of NuGet that I am installing in the pipelines job - you can see that above in my pipelines yml.

I have a global.json in my /src directory where my sln, and nuget.config is:

{
  "sdk": {
    "version": "3.1.201"
  }
}

My pipelines job uses the following agent:

pool:
  vmImage: 'windows-latest'

I have checked this agent - it has sdk version 3.1.201 already installed.

[dazinator]

I've got to the bottom of my issue. @ctaggart 's steps do work, but not if you accidentally use the latest preview sdk (3.1.300-preview-015095) when generating the lock files locally.

I was doing the following to regenerate the lock files locally:

dotnet nuget locals all --clear
git clean -xfd
git rm **/packages.lock.json -f
dotnet restore src --configfile src/nuget.config

dotnet --version
3.1.300-preview-015095

The azure pipelines hosted agent is on dotnet sdk 3.1.201

This caused all the nuget package hash mismatches from what I can tell. Using a global.json correctly so that 3.1.201 is also used locally fixed it.

However that leaves me wondering if this is a breaking change in 3.1.300-preview-015095?

[ghost]

Honestly, there are multiple msbuild settings that should be changed from defaults if you're hoping for reproducible builds. We're centralizing them into the Dotnet.ReproducibleBuilds.Isolated package. It's still in the early phases, but attempts to change msbuild defaults to avoid dependencies on anything machine/install specific (e.g. the nuget fallback folder that might have who-knows-what variants of real packages in it, and can be effected by installs of different dotnetsdk versions)

[Kurt-von-Laven]

Not sure where the best place to ask this question would be, but we are encountering the same error, NU1403, so I am curious what the status of the reproducible builds project is?

[nkolev92]

@Kurt-von-Laven

Please refer to #7921 (comment) for addressing the specific NU1403 problems you may have.

[Kurt-von-Laven]

Thank you for the quick reply, @nkolev92. I did see that comment, and tried clearing my NuGet cache to no avail, but my (admittedly limited) understanding of the other steps is that any of them would subsequently require recreating the lock file. Clearly this will work (even if none of those steps are taken and no configuration is changed), but it also removes one of the key security guarantees lock files offer, preventing us from confirming that the package hasn't been tampered with on the NuGet server (or en route from the NuGet server). It would be somewhat like having invested in the construction of a gate only to then open it anytime anyone knocks. Please correct me if I am missing something as I realize that my analogy may be greatly oversimplifying. (I am also happy to move this discussion to a more appropriate venue; I just don't know what that would be yet.)

[nkolev92]

You only recreate the lock file once, once you've excluded all the local packages that are basically seen as "tampered" by nuget.
It still retains the guarantee of repeatability, after all the failing is exactly that, NuGet saying the build isn't repeatable.

The tldr of this whole is that there are packages in the SDK fallback fodlers that are effectively seen as tampered because the compression algorithm changed the zip.
Our guidance is basically to exclude the SDK fallback folders and allow NuGet to download the packages from one of your sources.