Fake.DotNet.NuGet.NuGet.getLatestPackage fails with NullReferenceException

[UnoSD]

Description

I have a build script that has been working well for a few months, I recently re-run it and the error from the title occurred.

This is the stack trace:

Script reported an error:
-> BuildFailedException: Target 'Pack' failed.
   StackTrace:
        at Fake.Core.TargetModule.raiseIfError(OptionalTargetContext context) in D:\a\1\s\src\app\Fake.Core.Target\Target.fs:line 991
        at Fake.Core.TargetModule.runOrDefaultWithArguments(String defaultTarget) in D:\a\1\s\src\app\Fake.Core.Target\Target.fs:line 1177
        at <StartupCode$build_90647E5AC9A5A971800432B6C150AF059CFDF2ACF555E8D293E8D7B334C1E24C>.$Build$fsx.main@() in /home/uno/savs/sourcecode/dotNET/Pulumi.FSharp.Extensions/build.fsx:line 263
-> One or more errors occurred. (Object reference not set to an instance of an object.)
-> NullReferenceException: Object reference not set to an instance of an object.
   StackTrace:
        at Fake.DotNet.NuGet.NuGet.getFeedPackagesFromUrl@814.GenerateNext(IEnumerable`1& next) in D:\a\1\s\src\app\Fake.DotNet.NuGet\NuGet.fs:line 814
        at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 371
        at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System-Collections-IEnumerator-MoveNext() in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seqcore.fs:line 403
        at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)
        at Fake.DotNet.NuGet.NuGet.getLatestPackage(String repoUrl, String packageName) in D:\a\1\s\src\app\Fake.DotNet.NuGet\NuGet.fs:line 818
        at Build.clo@181-13.Invoke(String provider) in /home/uno/savs/sourcecode/dotNET/Pulumi.FSharp.Extensions/build.fsx:line 184
        at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source) in F:\workspace\_work\1\s\src\fsharp\FSharp.Core\seq.fs:line 497
        at Build.clo@178-12.Invoke(TargetParameter _arg6) in /home/uno/savs/sourcecode/dotNET/Pulumi.FSharp.Extensions/build.fsx:line 179
        at Fake.Core.TargetModule.runSimpleInternal(TargetContext context, Target target) in D:\a\1\s\src\app\Fake.Core.Target\Target.fs:line 299

The code to reproduce it:

Trace.traceImportant <| NuGet.getRepoUrl()
Trace.traceImportantfn "%A" <| NuGet.getLatestPackage "https://packages.nuget.org/v1/FeedService.svc/" "Bogus"
Trace.traceImportantfn "%A" <| (NuGet.getLatestPackage (NuGet.getRepoUrl()) ("Pulumi.FSharp.Kubernetes"))

The first line works and returns the (I presume) correct feed URL: https://packages.nuget.org/v1/FeedService.svc

I tried running with Bogus as another similar issue is present in the repository, but in that case the second line seems to work (but not for me).

Repro steps

Please provide the steps required to reproduce the problem

1. Add the code above to a build script

2. Run FAKE

Expected behavior

To retrieve the NuGet package details

Actual behavior

NullReferenceException

Known workarounds

No known workaround

Related information

• Operating system
  Arch Linux
• Branch
  Installed as dotnet tool, dotnet fake --version:

FAKE 5 - F# Make (5.20.3) (this line is written to standard error, see https://github.com/fsharp/FAKE/issues/2066)
FakePath: /home/user/.nuget/packages/fake-cli/5.20.3/tools/netcoreapp2.1/any/Fake.Runtime.dll
Paket.Core: 5.249.0

• .NET Runtime, CoreCLR or Mono Version

dotnet --list-sdks
3.1.406 [/home/uno/.dotnet/sdk]
5.0.103 [/home/uno/.dotnet/sdk]

• Indications of severity
  Medium?
• Version of FAKE (4.X, 5.X)
  5.20.3

I fetched the https://packages.nuget.org/v1/FeedService.svc/ URL and does not seem to return any package (which may be the issue), is there an active URL I can hardcode instead of calling getFeedUrl to get it to work?

Thank you

[github-actions]

Welcome to the FAKE community! Thank you so much for creating your first issue and therefore improving the project!

[yazeedobaid]

I have run your code in the following test case:

testCase "Check NuGet Feed" <| fun _ ->
  let bogusPackage: NuGet.NugetPackageInfo = NuGet.getLatestPackage (NuGet.getRepoUrl()) "Bogus"
  let PulumiPackage: NuGet.NugetPackageInfo = NuGet.getLatestPackage (NuGet.getRepoUrl()) "Pulumi.FSharp.Kubernetes"
  Trace.traceImportant <| NuGet.getRepoUrl()
  Trace.traceImportantfn "%A" <| bogusPackage
  Trace.traceImportantfn "%A" <| PulumiPackage
  Expect.equal bogusPackage.Id "Bogus" "Id filled"

And the test passes and the follow is the output from tracing statments:

https://azuresearch-usnc.nuget.org/query
{ Id = "Bogus"g... 132/194 -
  Version = "33.0.2"
  Description =
               "A simple and sane data generator for populating objects that supports different locales. A delightful port of the famed faker.js and inspired by FluentValidation. Use Bogus to create UIs with fake data or seed databases. Get started by using Faker class or a DataSet directly."
  Summary = ""
  IsLatestVersion = true
  Authors = "Brian Chavez"
  Owners = "Brian Chavez"
  Tags =
        "faker,fake,bogus,poco,data,generator,database,seed,values,test-data,test,tdd,testing,.net,EF"
  ProjectUrl = "https://github.com/bchavez/Bogus"
  LicenseUrl = "https://raw.githubusercontent.com/bchavez/Bogus/master/LICENSE"
  Title = "Bogus" }
{ Id = "FSharp.Core"
  Version = "5.0.1"
  Description =
               "FSharp.Core redistributables from F# Tools version 5.0.0 For F# 5.0.  Contains code from the F# Software Foundation."
  Summary = ""
  IsLatestVersion = true
  Authors = "Microsoft"
  Owners = "Microsoft"
  Tags = "Visual,F#,Compiler,FSharp,functional,programming"
  ProjectUrl = "https://github.com/dotnet/fsharp"
  LicenseUrl = "https://www.nuget.org/packages/FSharp.Core/5.0.1/license"
  Title = "FSharp.Core" }

For the second package, it returned FSharp.Core instead of Pulumi.FSharp.Kubernetes since query service matches on package title, this is the underlying GET request that was used to request the package information:
https://azuresearch-usnc.nuget.org/query?q=title:Pulumi.FSharp.Kubernetes

The current FAKE release uses V1 of NuGet Feed which is deprecated and is no longer supported. The current WIP release has fixed this and uses the V3 NuGet Feed which will return results as shown above. So your problem should be fixed by the next release.

Thanks

[UnoSD]

I tried again today and I had the same result, I will try to do it from a Windows machine and maybe a fresh checkout of the repository.

Full context for reference: https://github.com/UnoSD/Pulumi.FSharp.Extensions/blob/f43ad73fa849e7c5585da5d0f756274e6dfc409e/build.fsx#L209

Thank you

EDIT:

Same issue checking out the repository from scratch on a Arch Linux box

OK, same problem on a Windows VM, so it must be something in my repository

[matthid]

Note that fake writes a lockfile (https://github.com/UnoSD/Pulumi.FSharp.Extensions/blob/f43ad73fa849e7c5585da5d0f756274e6dfc409e/build.fsx.lock) to provide a stable build. So you need to delete it to force fake to update to latest packages to get the bugfix.

[UnoSD]

@matthid same issue on a fresh clone of the repo, but I also just tried to remove the build.fsx.lock as suggested and I get the error anyway

[yazeedobaid]

@UnoSD In your build.fsx file you have pinned FSharp.Core to 4.7.0 which forces Paket to install FAKE modules with version 5.19.1 Which is an old version of FAKE. Could you please try to pin FSharp.Core to 4.7.2 and try again?

[UnoSD]

thank you for the help @yazeedobaid

that is an interesting development to the story, pinning to 4.7.2 got rid of the error, but now the get package function returns the wrong package.

getFullName provider |>
Trace.traceImportantfn "%A"

getFullName provider |> 
NuGet.getLatestPackage (NuGet.getRepoUrl()) |>
Trace.traceImportantfn "%A"

returns:

"Pulumi.FSharp.AzureAD"
{ Id = "Azure.Storage.Blobs"
  Version = "12.8.1"
  Description =
               "This client library enables working with the Microsoft Azure Storage Blob service for storing binary and text data.
      For this release see notes - https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/README.md and https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/CHANGELOG.md
      in addition to the breaking changes https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/BreakingChanges.txt
      Microsoft Azure Storage quickstarts and tutorials - https://docs.microsoft.com/en-us/azure/storage/
      Microsoft Azure Storage REST API Reference - https://docs.microsoft.com/en-us/rest/api/storageservices/
      REST API Reference for Blob Service - https://docs.microsoft.com/en-us/rest/api/storageservices/blob-service-rest-api"
  Summary = ""
  IsLatestVersion = true
  Authors = "Microsoft"
  Owners = "Microsoft"
  Tags =
        "Microsoft,Azure,Storage,Blobs,Blob,StorageScalable,windowsazureofficial,azureofficial"
  ProjectUrl =
              "https://github.com/Azure/azure-sdk-for-net/blob/Azure.Storage.Blobs_12.8.1/sdk/storage/Azure.Storage.Blobs/README.md"
  LicenseUrl =
              "https://www.nuget.org/packages/Azure.Storage.Blobs/12.8.1/license"
  Title = "Azure.Storage.Blobs" }

The package search string is therefore "Pulumi.FSharp.AzureAD", but the package returned is "Azure.Storage.Blobs"

Looking at your previous message, it should match the package title, is there any way to match on ID instead? The package does not have a title specified in the NuSpec.

Thank you

EDIT:

I've updated the packages to include the "title" in the NuSpec, but I wonder if this change is going to break other people's code, too. Given that it was working before, I presume it was searching on ID instead in the previous implementation.

[yazeedobaid]

It seems that the packageid is the query string that is the one returning exact results. I tried the id property but it is the same as the title.

For example, searching for Microsoft.AspNetCore.Authentication.AzureAD.UI returns results for Microsoft.AspNetCore.Authentication.Abstractions:
https://azuresearch-usnc.nuget.org/query?q=id:%22Microsoft.AspNetCore.Authentication.AzureAD.UI%22

But if you replace the id with packageid it will match exact and returns information for Microsoft.AspNetCore.Authentication.AzureAD.UI

The old API was broken since NuGet has deprecated V1 of NuGet feed APIs, I can work on a PR to give more control on the API form to use for NuGet. I'm thinking of the following options:

// search by title by default, or should it change to search for packageid by default?
"FAKE" |> NuGet.getLatestPackage (NuGet.getRepoUrl())

// search by packageid, id, owner, see this link for all available options
"FAKE" |> NuGet.getLatestPackageBy "packageid" (NuGet.getRepoUrl())

Do u have any other options or thoughts on it?

[UnoSD]

Personally, I would be OK with the search by package title or ID at this stage (as I have updated the title in my package, too).

As for what would be right, I presume it should be retro-compatible with the previous behaviour not to break other people's scripts, I was able to find the package by the package ID and the nuspec did not have a title before, so I presume PID would be the most non-disruptive option for search by default; then it would be nice to have also search by different properties, but I don't know if the previous implementation also searched into those.

Please do let me know if I can help.

[UnoSD]

thank you for the help