Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dotnet: use unpacked packages in store #327651

Merged
merged 6 commits into from
Jul 31, 2024

Conversation

corngood
Copy link
Contributor

@corngood corngood commented Jul 16, 2024

Description of changes

This is still a bit of a WIP, although it does pass nixpkgs-review on linux and darwin, and fetch-deps works on everything in nixpkgs. I mostly just want to do some more cleanup passes and pull out some incidental changes into separate commits.

The idea here is that we keep packages in $out/share/nuget/packages/[name]/[version], unpacked, in a structure that's compatible with what you'd have in ~/.nuget/packages. Then in a hook, we point $NUGET_FALLBACK_PACKAGES at all the dependencies. We also create a source, which contains 'stub' .nupkg files, containing only the .nuspec, and add it to the root nuget.config.

There are still some rough edges. For example, dotnet tool restore doesn't really play nice with $NUGET_PACKAGES, and we still need to treat the package directory as a 'source' for projects that don't have locked dependencies. In order to deal with this, we pass installable = true to mkNugetDeps, which tells it to create full .nupkg files instead of stubs.

Other highlights:

  • we patch (e.g. interpreter, rpaths, shebangs) all packages in nix-store, rather than after restore
  • fetch-deps is now more closely based on the package derivation
  • we strip existing nuget.config files (except in fetch-deps), and create a new one in the build root
  • output packages are always unpacked (even if a .nupkg also exists), which stops runtime dependencies from being obfuscated
  • projectReferences is now obsolete, and packages can be used from any build input

FYI @NixOS/dotnet @Smaug123 (aside: would you like to be added to the dotnet team?)

Questions:

  • should we leave .nupkg generation for (dependent) build time, and keep them out of the store altogether?

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 16, 2024
@corngood corngood force-pushed the dotnet-unpacked-packages branch 3 times, most recently from c9733ce to 14186bc Compare July 16, 2024 14:50
@corngood
Copy link
Contributor Author

Result of nixpkgs-review pr 327651 run on x86_64-linux 1

4 packages failed to build:
  • python312Packages.pyannote-audio
  • python312Packages.pyannote-audio.dist
  • whisper-ctranslate2
  • whisper-ctranslate2.dist
141 packages built:
  • ArchiSteamFarm
  • alttpr-opentracker
  • am2rlauncher
  • avalonia-ilspy
  • azure-functions-core-tools
  • beatsabermodmanager
  • bicep
  • bililiverecorder
  • blendfarm
  • boogie
  • btcpayserver
  • btcpayserver-altcoins
  • cavalier
  • celeste64
  • certdump
  • csharp-ls
  • csharpier
  • csharprepl
  • cyclonedx-cli
  • dafny
  • denaro
  • depotdownloader
  • discordchatexporter-cli
  • dotnet-aspnetcore
  • dotnet-aspnetcore_7
  • dotnet-aspnetcore_8
  • dotnet-outdated
  • dotnet-runtime
  • dotnet-runtime_7
  • dotnet-runtime_8
  • dotnet-sdk
  • dotnet-sdk_7
  • dotnet-sdk_8
  • dotnetCorePackages.aspnetcore_9_0
  • dotnetCorePackages.dotnet_8.aspnetcore
  • dotnetCorePackages.dotnet_8.runtime
  • dotnetCorePackages.dotnet_8.sdk
  • dotnetCorePackages.dotnet_8.sdk.artifacts
  • dotnetCorePackages.dotnet_8.sdk.packages
  • dotnetCorePackages.dotnet_8.vmr
  • dotnetCorePackages.dotnet_9.aspnetcore
  • dotnetCorePackages.dotnet_9.runtime
  • dotnetCorePackages.dotnet_9.sdk
  • dotnetCorePackages.dotnet_9.sdk.artifacts
  • dotnetCorePackages.dotnet_9.sdk.packages
  • dotnetCorePackages.dotnet_9.vmr
  • dotnetCorePackages.patch-nupkgs
  • dotnetCorePackages.runtime_9_0
  • dotnetCorePackages.sdk_6_0_1xx
  • dotnetCorePackages.sdk_7_0_1xx
  • dotnetCorePackages.sdk_7_0_3xx
  • dotnetCorePackages.sdk_8_0_1xx
  • dotnetCorePackages.sdk_9_0
  • eventstore
  • fable
  • famistudio
  • fantomas
  • formula
  • fsautocomplete
  • galaxy-buds-client
  • garnet
  • gh-gei
  • git-credential-manager
  • github-runner
  • gitversion
  • godot3-mono
  • godot3-mono-debug-server
  • godot3-mono-export-templates
  • godot3-mono-headless
  • godot3-mono-headless.dev
  • godot3-mono-headless.man
  • godot3-mono-server
  • godot3-mono.dev
  • godot3-mono.man
  • ilspycmd
  • inklecate
  • jackett
  • jellyfin
  • jetbrains.clion
  • jetbrains.rider
  • kavita
  • knossosnet
  • kryptor
  • libation
  • lidarr
  • lubelogger
  • lumafly
  • marksman
  • mqttmultimeter
  • msbuild
  • naps2
  • nbxplorer
  • netcoredbg
  • networkminer
  • nexusmods-app
  • nexusmods-app-unfree
  • nuget-to-nix
  • omnisharp-roslyn
  • openra
  • opentabletdriver
  • openutau
  • osu-lazer
  • pablodraw
  • parabolic
  • pbm
  • pinta
  • pre-commit
  • pre-commit.dist
  • prowlarr
  • ps3-disc-dumper
  • pupdate
  • python311Packages.asteroid-filterbanks
  • python311Packages.asteroid-filterbanks.dist
  • python311Packages.clr-loader
  • python311Packages.clr-loader.dist
  • python311Packages.pyannote-audio
  • python311Packages.pyannote-audio.dist
  • python311Packages.pythonnet
  • python311Packages.pythonnet.dist
  • python312Packages.asteroid-filterbanks
  • python312Packages.asteroid-filterbanks.dist
  • python312Packages.clr-loader
  • python312Packages.clr-loader.dist
  • python312Packages.pythonnet
  • python312Packages.pythonnet.dist
  • radarr
  • readarr
  • retrospy
  • roslyn
  • roslyn-ls
  • ryujinx
  • scarab
  • slskd
  • sonarr
  • space-station-14-launcher
  • technitium-dns-server
  • tone
  • torrentstream
  • vintagestory
  • wasabibackend
  • xivlauncher

@ofborg ofborg bot added 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 101-500 10.rebuild-linux: 101-500 and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Jul 16, 2024
@GGG-KILLER
Copy link
Contributor

GGG-KILLER commented Jul 17, 2024

Looks good to me so far.

we currently output .nupkg files next to the unpacked packages (as in ~/.nuget/packages). could this be avoided?

Maybe if in fetch-deps we generated a nuget lockfile we could avoid this, but other than that I don't see much of an alternative.

The issue is that nuget lockfiles don't support multi-RID restores, as it only works for the current RID, and if you put in a dependency for a different RID in the lockfile it'll complain that it doesn't need it.
Having one lockfile per RID also wasn't possible last time I tried.

maybe the .nupkgs can be empty and still function as a source for the purpose of version detection?

Maybe if we keep only the nuspec file in there we could somehow hack the restore algo into supporting it, but I don't know how I feel about it, feels a bit flaky.

maybe source and packages should be separate outputs?

I like this idea, I think it'd make things less confusing as one is supposed to be the target directory and the other a source.

@corngood
Copy link
Contributor Author

Maybe if we keep only the nuspec file in there we could somehow hack the restore algo into supporting it, but I don't know how I feel about it, feels a bit flaky.

I'll do some more testing around this. What's in here right now might not be required for all packages. We could possibly allow multiple options (best first):

  1. add all dependency package dirs to fallbackPackageFolders in nuget.config
  2. create NUGET_FALLBACK_PACKAGES with symlinks to dependencies
  3. create NUGET_PACKAGES with symlinks to dependencies
  4. do (3) and also add a package source with all the dependencies .nupkgs (pretty much what this PR currently does)

maybe source and packages should be separate outputs?

One problem with this is that it obfuscates runtime deps, unless we make the .nupkgs uncompressed, or extract the dependencies.

@GGG-KILLER
Copy link
Contributor

GGG-KILLER commented Jul 17, 2024

I'll do some more testing around this. What's in here right now might not be required for all packages. We could possibly allow multiple options (best first):

  1. add all dependency package dirs to fallbackPackageFolders in nuget.config
  2. create NUGET_FALLBACK_PACKAGES with symlinks to dependencies
  3. create NUGET_PACKAGES with symlinks to dependencies
  4. do (3) and also add a package source with all the dependencies .nupkgs (pretty much what this PR currently does)

I might be mistaken, but 1 and 2 seem to be the same in practice? They do seem like the best option considering that it's immutable and dotnet restore won't try to modify them, but considering we don't want any networking at all in dotnet restore and already have all the dependencies listed and downloaded, I don't see much of a difference between 1, 2 and 3 as in theory none of them would modify the source dir.

Given that most projects don't have lockfiles at all, I don't think we'll be able to get rid of overriding the source with a directory full of nupkgs as it'll try to use online sources instead if we don't do it.

For now I think the most realistic path would be to proceed with 4 as it works on projects without lockfiles and maybe look into requiring lockfiles for projects or something similar so that we can get rid of the directory full of .nupkgs entirely.

@corngood
Copy link
Contributor Author

I might be mistaken, but 1 and 2 seem to be the same in practice?

Yeah, more or less. It's just that NUGET_FALLBACK_PACKAGES only supports a single dir.

They do seem like the best option considering that it's immutable and dotnet restore won't try to modify them,

Currently nuget-to-nix has to search through one package dir with a mix of built and downloaded packages, so in theory it would simplify that.

Given that most projects don't have lockfiles at all, I don't think we'll be able to get rid of overriding the source with a directory full of nupkgs as it'll try to use online sources instead if we don't do it.

One thing I'm not sure of, and was going to test, is if we can do something like put empty files in place of the .nupkg files. If it's only using the source to determine which versions are available, maybe that'll work?

Another crazier option might be to make a fake http source using a daemon?

@GGG-KILLER
Copy link
Contributor

GGG-KILLER commented Jul 17, 2024

Yeah, more or less. It's just that NUGET_FALLBACK_PACKAGES only supports a single dir.

It actually supports multiple, but they need to be separated by semicolons:

The value must contain the full path or paths to the fallback package folders. Multiple paths may be included by setting the value to a ; delimited list.

Currently nuget-to-nix has to search through one package dir with a mix of built and downloaded packages, so in theory it would simplify that.

How so? I thought it only searched through stuff that dotnet restore pulled, so if there's any dependencies on csprojs or nix-built stuff then there wouldn't be any .nupkgs there for it to find.

One thing I'm not sure of, and was going to test, is if we can do something like put empty files in place of the .nupkg files. If it's only using the source to determine which versions are available, maybe that'll work?

Another crazier option might be to make a fake http source using a daemon?

I've asked a question in the .NET Discord to the nuget folks about it, will report back if I get anything, but I think we could get away with putting in a .nupkg with just the .nuspec inside. It needs testing though.

The fake HTTP daemon wouldn't work because it'd still try to download .nupkgs from the daemon so it'd be the same as using a directory as a source.

@corngood
Copy link
Contributor Author

corngood commented Jul 17, 2024

It actually supports multiple, but they need to be separated by semicolons:

Ah yeah, I don't know why I thought that. I swear I looked it up in the code before, but it's definitely splitting on ;.

How so? I thought it only searched through stuff that dotnet restore pulled, so if there's any dependencies on csprojs or nix-built stuff then there wouldn't be any .nupkgs there for it to find.

It searches $NUGET_PACKAGES, which includes all the things that have been symlinked from dependencies (sdk packages, etc). It looks at .nupkg.metadata, which is empty in packages from the nix store. Interestingly, (unlike dotnet restore) dotnet tool restore doesn't seem to fill in .nupkg.metadata with the source it used to download.

The fake HTTP daemon wouldn't work because it'd still try to download .nupkgs from the daemon so it'd be the same as using a directory as a source.

I wasn't sure if it would actually download them or just enumerate the available versions.

I think we could get away with putting in a .nupkg with just the .nuspec inside. It needs testing though.

This is a good idea to try.

@corngood
Copy link
Contributor Author

I think we could get away with putting in a .nupkg with just the .nuspec inside. It needs testing though.

It does seem to work with an uncompressed zip containing only the nuspec. Something like that would be safe to put in outputs.

@corngood corngood force-pushed the dotnet-unpacked-packages branch from 14186bc to cc31676 Compare July 24, 2024 01:53
@corngood
Copy link
Contributor Author

Result of nixpkgs-review pr 327651 run on x86_64-linux 1

12 packages failed to build:
  • dotnetCorePackages.dotnet_9.aspnetcore
  • dotnetCorePackages.dotnet_9.runtime
  • dotnetCorePackages.dotnet_9.sdk
  • dotnetCorePackages.dotnet_9.sdk.artifacts
  • dotnetCorePackages.dotnet_9.sdk.packages
  • dotnetCorePackages.dotnet_9.vmr
  • python311Packages.pyannote-audio
  • python311Packages.pyannote-audio.dist
  • python312Packages.pyannote-audio
  • python312Packages.pyannote-audio.dist
  • whisper-ctranslate2
  • whisper-ctranslate2.dist
135 packages built:
  • ArchiSteamFarm
  • alttpr-opentracker
  • am2rlauncher
  • avalonia-ilspy
  • azure-functions-core-tools
  • beatsabermodmanager
  • bicep
  • bililiverecorder
  • blendfarm
  • boogie
  • btcpayserver
  • btcpayserver-altcoins
  • cavalier
  • celeste64
  • certdump
  • csharp-ls
  • csharpier
  • csharprepl
  • cyclonedx-cli
  • dafny
  • denaro
  • depotdownloader
  • discordchatexporter-cli
  • dotnet-aspnetcore
  • dotnet-aspnetcore_7
  • dotnet-aspnetcore_8
  • dotnet-outdated
  • dotnet-runtime
  • dotnet-runtime_7
  • dotnet-runtime_8
  • dotnet-sdk
  • dotnet-sdk_7
  • dotnet-sdk_8
  • dotnetCorePackages.aspnetcore_9_0
  • dotnetCorePackages.dotnet_8.aspnetcore
  • dotnetCorePackages.dotnet_8.runtime
  • dotnetCorePackages.dotnet_8.sdk
  • dotnetCorePackages.dotnet_8.sdk.artifacts
  • dotnetCorePackages.dotnet_8.sdk.packages
  • dotnetCorePackages.dotnet_8.vmr
  • dotnetCorePackages.nugetPackageHook
  • dotnetCorePackages.patchNupkgs
  • dotnetCorePackages.runtime_9_0
  • dotnetCorePackages.sdk_6_0_1xx
  • dotnetCorePackages.sdk_7_0_1xx
  • dotnetCorePackages.sdk_7_0_3xx
  • dotnetCorePackages.sdk_8_0_1xx
  • dotnetCorePackages.sdk_9_0
  • eventstore
  • fable
  • famistudio
  • fantomas
  • formula
  • fsautocomplete
  • galaxy-buds-client
  • garnet
  • gh-gei
  • git-credential-manager
  • github-runner
  • gitversion
  • godot3-mono
  • godot3-mono-debug-server
  • godot3-mono-export-templates
  • godot3-mono-headless
  • godot3-mono-headless.dev
  • godot3-mono-headless.man
  • godot3-mono-server
  • godot3-mono.dev
  • godot3-mono.man
  • ilspycmd
  • inklecate
  • jackett
  • jellyfin
  • jetbrains.clion
  • jetbrains.rider
  • kavita
  • knossosnet
  • kryptor
  • libation
  • lidarr
  • lubelogger
  • lumafly
  • marksman
  • mqttmultimeter
  • msbuild
  • naps2
  • nbxplorer
  • netcoredbg
  • networkminer
  • nexusmods-app
  • nexusmods-app-unfree
  • nuget-to-nix
  • omnisharp-roslyn
  • openra
  • opentabletdriver
  • openutau
  • osu-lazer
  • pablodraw
  • parabolic
  • pbm
  • pinta
  • pre-commit
  • pre-commit.dist
  • prowlarr
  • ps3-disc-dumper
  • pupdate
  • python311Packages.asteroid-filterbanks
  • python311Packages.asteroid-filterbanks.dist
  • python311Packages.clr-loader
  • python311Packages.clr-loader.dist
  • python311Packages.pythonnet
  • python311Packages.pythonnet.dist
  • python312Packages.asteroid-filterbanks
  • python312Packages.asteroid-filterbanks.dist
  • python312Packages.clr-loader
  • python312Packages.clr-loader.dist
  • python312Packages.pythonnet
  • python312Packages.pythonnet.dist
  • radarr
  • readarr
  • retrospy
  • roslyn
  • roslyn-ls
  • ryujinx
  • scarab
  • slskd
  • sonarr
  • space-station-14-launcher
  • tagger
  • technitium-dns-server
  • tone
  • torrentstream
  • vintagestory
  • wasabibackend
  • xivlauncher

@corngood corngood force-pushed the dotnet-unpacked-packages branch from cc31676 to 02099a5 Compare July 24, 2024 13:26
@corngood
Copy link
Contributor Author

corngood commented Jul 24, 2024

I've updated the description to match the latest version. Now we:

  • point $NUGET_FALLBACK_PACKAGES at the dependencies

We also unconditionally:

  • create a 'stub' source and add it to nuget.config

In theory this would be optional if all dependencies were locked, but that's rare enough that I'm not sure it makes sense for it to be configurable yet.

If linkNugetPackages is set, we:

  • link all $NUGET_FALLBACK_PACKAGES into $NUGET_PACKAGES

This is required to work around upstream bugs like tunnelvisionlabs/ReferenceAssemblyAnnotator#94

If installable is passed to mkNugetDeps, we:

  • make full .nupkg files in the source instead of stubs.

This is required for dotnet tool install, because it can only install from the .nupkg, and not from an installed package. It's also required for some VMR dependencies.

@corngood corngood force-pushed the dotnet-unpacked-packages branch from 02099a5 to 53decad Compare July 24, 2024 16:28
@corngood
Copy link
Contributor Author

Result of nixpkgs-review pr 327651 run on x86_64-linux 1

6 packages failed to build:
  • python311Packages.pyannote-audio
  • python311Packages.pyannote-audio.dist
  • python312Packages.pyannote-audio
  • python312Packages.pyannote-audio.dist
  • whisper-ctranslate2
  • whisper-ctranslate2.dist
141 packages built:
  • ArchiSteamFarm
  • alttpr-opentracker
  • am2rlauncher
  • avalonia-ilspy
  • azure-functions-core-tools
  • beatsabermodmanager
  • bicep
  • bililiverecorder
  • blendfarm
  • boogie
  • btcpayserver
  • btcpayserver-altcoins
  • cavalier
  • celeste64
  • certdump
  • csharp-ls
  • csharpier
  • csharprepl
  • cyclonedx-cli
  • dafny
  • denaro
  • depotdownloader
  • discordchatexporter-cli
  • dotnet-aspnetcore
  • dotnet-aspnetcore_7
  • dotnet-aspnetcore_8
  • dotnet-outdated
  • dotnet-runtime
  • dotnet-runtime_7
  • dotnet-runtime_8
  • dotnet-sdk
  • dotnet-sdk_7
  • dotnet-sdk_8
  • dotnetCorePackages.aspnetcore_9_0
  • dotnetCorePackages.dotnet_8.aspnetcore
  • dotnetCorePackages.dotnet_8.runtime
  • dotnetCorePackages.dotnet_8.sdk
  • dotnetCorePackages.dotnet_8.sdk.artifacts
  • dotnetCorePackages.dotnet_8.sdk.packages
  • dotnetCorePackages.dotnet_8.vmr
  • dotnetCorePackages.dotnet_9.aspnetcore
  • dotnetCorePackages.dotnet_9.runtime
  • dotnetCorePackages.dotnet_9.sdk
  • dotnetCorePackages.dotnet_9.sdk.artifacts
  • dotnetCorePackages.dotnet_9.sdk.packages
  • dotnetCorePackages.dotnet_9.vmr
  • dotnetCorePackages.nugetPackageHook
  • dotnetCorePackages.patchNupkgs
  • dotnetCorePackages.runtime_9_0
  • dotnetCorePackages.sdk_6_0_1xx
  • dotnetCorePackages.sdk_7_0_1xx
  • dotnetCorePackages.sdk_7_0_3xx
  • dotnetCorePackages.sdk_8_0_1xx
  • dotnetCorePackages.sdk_9_0
  • eventstore
  • fable
  • famistudio
  • fantomas
  • formula
  • fsautocomplete
  • galaxy-buds-client
  • garnet
  • gh-gei
  • git-credential-manager
  • github-runner
  • gitversion
  • godot3-mono
  • godot3-mono-debug-server
  • godot3-mono-export-templates
  • godot3-mono-headless
  • godot3-mono-headless.dev
  • godot3-mono-headless.man
  • godot3-mono-server
  • godot3-mono.dev
  • godot3-mono.man
  • ilspycmd
  • inklecate
  • jackett
  • jellyfin
  • jetbrains.clion
  • jetbrains.rider
  • kavita
  • knossosnet
  • kryptor
  • libation
  • lidarr
  • lubelogger
  • lumafly
  • marksman
  • mqttmultimeter
  • msbuild
  • naps2
  • nbxplorer
  • netcoredbg
  • networkminer
  • nexusmods-app
  • nexusmods-app-unfree
  • nuget-to-nix
  • omnisharp-roslyn
  • openra
  • opentabletdriver
  • openutau
  • osu-lazer
  • pablodraw
  • parabolic
  • pbm
  • pinta
  • pre-commit
  • pre-commit.dist
  • prowlarr
  • ps3-disc-dumper
  • pupdate
  • python311Packages.asteroid-filterbanks
  • python311Packages.asteroid-filterbanks.dist
  • python311Packages.clr-loader
  • python311Packages.clr-loader.dist
  • python311Packages.pythonnet
  • python311Packages.pythonnet.dist
  • python312Packages.asteroid-filterbanks
  • python312Packages.asteroid-filterbanks.dist
  • python312Packages.clr-loader
  • python312Packages.clr-loader.dist
  • python312Packages.pythonnet
  • python312Packages.pythonnet.dist
  • radarr
  • readarr
  • retrospy
  • roslyn
  • roslyn-ls
  • ryujinx
  • scarab
  • slskd
  • sonarr
  • space-station-14-launcher
  • tagger
  • technitium-dns-server
  • tone
  • torrentstream
  • vintagestory
  • wasabibackend
  • xivlauncher

@Smaug123
Copy link
Contributor

(I'm currently on holiday and diligently not doing anything that looks like work, but would in principle be interested in 1. this PR and 2. being on the dotnet team!)

@corngood
Copy link
Contributor Author

@Smaug123 have a good holiday and feel free to make a PR to add yourself here:

@Smaug123
Copy link
Contributor

By the way, don't take my "I'm interested" as a desire to block this PR on my account if it's ready to go in before I get back!

@corngood corngood marked this pull request as ready for review July 28, 2024 15:06
Copy link

@nixf-tidy-review nixf-tidy-review left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Hi! nixf-tidy code linter found some issues in your touched files. If you believe I'm reporting false positives, feel free to ping @inclyc.

I'm a review bot that based on git-diff thus there might be comments to previous code. Our nixpkgs CI will skip them, but you can also fix them if you like!

@corngood corngood force-pushed the dotnet-unpacked-packages branch from e3b0fd5 to f32ec36 Compare July 29, 2024 17:00
@corngood
Copy link
Contributor Author

Result of nixpkgs-review run on x86_64-linux 1

147 packages built:
  • ArchiSteamFarm
  • alttpr-opentracker
  • am2rlauncher
  • avalonia-ilspy
  • azure-functions-core-tools
  • beatsabermodmanager
  • bicep
  • bililiverecorder
  • blendfarm
  • boogie
  • btcpayserver
  • btcpayserver-altcoins
  • cavalier
  • celeste64
  • certdump
  • csharp-ls
  • csharpier
  • csharprepl
  • cyclonedx-cli
  • dafny
  • denaro
  • depotdownloader
  • discordchatexporter-cli
  • dotnet-aspnetcore
  • dotnet-aspnetcore_7
  • dotnet-aspnetcore_8
  • dotnet-outdated
  • dotnet-runtime
  • dotnet-runtime_7
  • dotnet-runtime_8
  • dotnet-sdk
  • dotnet-sdk_7
  • dotnet-sdk_8
  • dotnetCorePackages.aspnetcore_9_0
  • dotnetCorePackages.dotnet_8.aspnetcore
  • dotnetCorePackages.dotnet_8.runtime
  • dotnetCorePackages.dotnet_8.sdk
  • dotnetCorePackages.dotnet_8.sdk.artifacts
  • dotnetCorePackages.dotnet_8.sdk.packages
  • dotnetCorePackages.dotnet_8.vmr
  • dotnetCorePackages.dotnet_9.aspnetcore
  • dotnetCorePackages.dotnet_9.runtime
  • dotnetCorePackages.dotnet_9.sdk
  • dotnetCorePackages.dotnet_9.sdk.artifacts
  • dotnetCorePackages.dotnet_9.sdk.packages
  • dotnetCorePackages.dotnet_9.vmr
  • dotnetCorePackages.nugetPackageHook
  • dotnetCorePackages.patchNupkgs
  • dotnetCorePackages.runtime_9_0
  • dotnetCorePackages.sdk_6_0_1xx
  • dotnetCorePackages.sdk_7_0_1xx
  • dotnetCorePackages.sdk_7_0_3xx
  • dotnetCorePackages.sdk_8_0_1xx
  • dotnetCorePackages.sdk_9_0
  • eventstore
  • fable
  • famistudio
  • fantomas
  • formula
  • fsautocomplete
  • galaxy-buds-client
  • garnet
  • gh-gei
  • git-credential-manager
  • github-runner
  • gitversion
  • godot3-mono
  • godot3-mono-debug-server
  • godot3-mono-export-templates
  • godot3-mono-headless
  • godot3-mono-headless.dev
  • godot3-mono-headless.man
  • godot3-mono-server
  • godot3-mono.dev
  • godot3-mono.man
  • ilspycmd
  • inklecate
  • jackett
  • jellyfin
  • jetbrains.clion
  • jetbrains.rider
  • kavita
  • knossosnet
  • kryptor
  • libation
  • lidarr
  • lubelogger
  • lumafly
  • marksman
  • mqttmultimeter
  • msbuild
  • naps2
  • nbxplorer
  • netcoredbg
  • networkminer
  • nexusmods-app
  • nexusmods-app-unfree
  • nuget-to-nix
  • omnisharp-roslyn
  • openra
  • opentabletdriver
  • openutau
  • osu-lazer
  • pablodraw
  • parabolic
  • pbm
  • pinta
  • pre-commit
  • pre-commit.dist
  • prowlarr
  • ps3-disc-dumper
  • pupdate
  • python311Packages.asteroid-filterbanks
  • python311Packages.asteroid-filterbanks.dist
  • python311Packages.clr-loader
  • python311Packages.clr-loader.dist
  • python311Packages.pyannote-audio
  • python311Packages.pyannote-audio.dist
  • python311Packages.pythonnet
  • python311Packages.pythonnet.dist
  • python312Packages.asteroid-filterbanks
  • python312Packages.asteroid-filterbanks.dist
  • python312Packages.clr-loader
  • python312Packages.clr-loader.dist
  • python312Packages.pyannote-audio
  • python312Packages.pyannote-audio.dist
  • python312Packages.pythonnet
  • python312Packages.pythonnet.dist
  • radarr
  • readarr
  • retrospy
  • roslyn
  • roslyn-ls
  • ryujinx
  • scarab
  • slskd
  • sonarr
  • space-station-14-launcher
  • tagger
  • technitium-dns-server
  • tone
  • torrentstream
  • vintagestory
  • wasabibackend
  • whisper-ctranslate2
  • whisper-ctranslate2.dist
  • xivlauncher

@corngood corngood marked this pull request as draft July 30, 2024 13:02
@corngood
Copy link
Contributor Author

This is actually breaking some tests, like nexusmods-app.tests.help:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Extensions.Hosting.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.

File name: 'Microsoft.Extensions.Hosting.Abstractions, Version=8.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
/build/.attr-0l2nkwhif96f51f4amnlf414lhl4rv9vh8iffyp431v6s28gsr90: line 1:     5 Aborted                 (core dumped) /nix/store/riv0cmlbvkfydvglbgcc5iip6fdhjnm7-nexusmods-app-0.4.1/bin/NexusMods.App --help

Working on that now.

@corngood corngood force-pushed the dotnet-unpacked-packages branch from f32ec36 to 2ae0a24 Compare July 30, 2024 15:21
@corngood corngood force-pushed the dotnet-unpacked-packages branch from 2ae0a24 to f01dc2c Compare July 31, 2024 01:04
@corngood corngood marked this pull request as ready for review July 31, 2024 01:05
@github-actions github-actions bot added the 8.has: documentation This PR adds or changes documentation label Jul 31, 2024
@corngood corngood mentioned this pull request Jul 31, 2024
13 tasks
@corngood
Copy link
Contributor Author

I fixed another problem with nexusmods-app, and now all tests in the packages built by nixpkgs-review pass on linux. I also updated the docs to reflect the projectReferences change.

@GGG-KILLER GGG-KILLER added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Jul 31, 2024
@corngood
Copy link
Contributor Author

I'm planning on merging this soon. I just wanted to check out some closure sizes. Some things will be larger uncompressed, but should be similar compressed.

I also checked a few packages to confirm that build times are improved, and it seems likely to save in the neighborhood of 30s from restore in a package with a reasonable number of dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: dotnet Language: .NET 6.topic: python 8.has: documentation This PR adds or changes documentation 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 101-500 10.rebuild-linux: 101-500 12.approvals: 1 This PR was reviewed and approved by one reputable person
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants