There isn't a clear canonical way to refer to a specific package version.

[thomastjeffery]

Context

The overwhelming majority of packages in the Nix ecosystem did not come from the Nix ecosystem. As such, the projects these packages are made to provide tend to use incompatible organization philosophies like version numbering.

Even so, users expect Nix to provide multiple instances of packages that are delineated by version number.

Nix Design

In Nix, a package is described in two places:

1. The derivation: Somewhere in nixpkgs/pkgs is a default.nix that describes where to get the source, how to build it, and what files to keep.

2. The global attribute path: The package name is defined in pkgs/top-level/all-packages.nix. The name evaluates a nix expression using the derivation defined in nixpkgs/pkgs/somewhere/package-name/default.nix.

It is assumed that a derivation will be built from the latest source available. To update a package, the derivation is edited to build from a newer source version, tested, and the changes to the derivation are added to a git PR to be merged with nixpkgs/master.

It isn't uncommon for a derivation to depend on a different version of a package than what is provided by its default.nix. For example, many C/C++ programs do not build with the latest GCC.

When different versions of the same project are provided, they each need a unique derivation file and attribute path name.

It is recommended in the manual that extra package versions are created as separate derivations alongside the default derivation, and the version name is appended to the package name:

If there is only one version of a package, its Nix expression should be named e2fsprogs/default.nix. If there are multiple versions, this should be reflected in the filename, e.g. e2fsprogs/1.41.8.nix and e2fsprogs/1.41.9.nix. The version in the filename should leave out unnecessary detail.

All versions of a package must be included in all-packages.nix to make sure that they evaluate correctly.

For example, there currently exist the attribute paths gcc6, gcc7, gcc8, gcc9, and gcc10 alongside gcc, which provides gcc 9.2.0.

Nix Usage

When installing a package, a user will generally refer to its attribute path name, as defined in all-packages.nix.

When a user updates their system with nixos-rebuild --upgrade [switch|boot] or nix-env -u [packagename|*], nix gets whatever derivations are defined in the latest nixpkgs commit.

NOTE: This is made a little more complicated by channels. Just remember that channels refer to git branch names.

Problem

It is fairly common for a user to want to use a different package version than the default one provided in their nix channel. In order to use it, the user must find it.

There are three ways this can be accomplished:

1. If there is already a derivation provided by the user's channel, the user need only find its attribute path name. This has gotten easier with nix search, but isn't always reliable. Usually, packages follow the naming convention of appending the version to the package name, but this is not always the case.

2. If there is not a derivation provided in the user's channel, the user must find a hash that points to the desired derivation version, and use pinning to refer to the package by hash.
   The hash can be found using this third-party tool, and used as described here.

3. If there is not a derivation that provides the desired package version, the user much create their own. Suddenly, a user who just wanted to use a simple and elegant package manager needs to learn how to write a nix derivation, or at least how (and where) to modify an existing one and use that.
   This is usually done by checking out nixpkgs/master, and either altering the existing derivation (or a copy) to one that builds the desired package version.
   This method is described in the manual here.

None of these options is obvious, clear, canonical, and easy to use.

Discussion has been happening on this topic for about five years (off and on) here: #9682

[vcunat]

Perhaps it's just me, but I'm not sure where you aim at. I think the intended canonical way will be Flakes, but it was linked in the other thread already, so perhaps I'm missing something you want? That is, apart from more standardization of attribute naming, for the relatively uncommon cases where we do keep multiple versions in nixpkgs.

---

Generally, I believe that in practice the main work is in maintenance of those versions – ensuring that the combinations you'd like keep working together (up to exponential number), producing binaries for those combinations (machine time & space), patching important (security) bugs, etc. I'd rather try pushing the community towards using relatively few combinations of versions and configurations, as it just appears to make life easier (through sharing that work, basically).

For instance, what do you do after you find your desired version in some older nixpkgs commit? I believe typically you don't really want to just use the older nixpkgs version. If that was OK, why would any distro bother with stable releases? (And I omit the complication of having more of such dependencies.) I expect you typically want something like forward-porting that version to current nixpkgs stable branch (or master) – which might just work trivially or it might require further changes.

[markuskowa]

Is this maybe what you are looking for: https://matthewbauer.us/blog/all-the-versions.html

[spinus]

It is assumed that a derivation will be built from the latest source available.

where did you get this assumption from? is it your assumption? I didn't see this assumption anywhere else.

I think nix ecosystem would be solving different problems than ones you express and desire. Let's look at nix website:
Reproducible
Declarative
Reliable
Those are problems nix is trying to solve, not to deliver as many packages in all of different versions. It's giving you a tool to do it in a stable manner, but that's not the goal. How much work would be to support so much software.

[jeff-hykin]

@vcunat

I'm not sure where you aim at

Trivial Example

I want to install Lua 5.0.x globally (assume I don't care about ANYTHING else)

1. How can I list every package/channel/derivation that contains Lua 5.0.x
2. What exact command do I run to install that version globally

(Now repeat the process for Python 3.6.x and Ruby 2.5.x)

Problem

If your answer is "well there's no straightforward way of doing that" then that is what this issue is about: we want a straightforward way of doing it because it is a frequent basic need

If there is a straightforward way, my understanding is that will be the end of this issue

Discussion

I believe @thomastjeffery's post is long because he was diving into the details of how this process could potentially be made simple and the current state of why it is difficult

[thomastjeffery]

As far as I am concerned, until a solution does not need to be searched for by users, this is an open issue.

There needs to be a straightforward way for users to install different versions of packages.

There needs to be a straightforward way to create derivations that depend on specific versions of packages.

Right now, there may be ways of doing those things, but those methods are not obvious or straightforward.

As I stated in the original issue, I do not have enough expertise to lead this discussion. I have done my best to outline the problem, and will be happy to answer any questions about it, and try to understand proposed solutions.

[vcunat]

My point of view is that if there still are good use cases for Lua 5.0, Python 3.6, etc. we should have them as attributes in the current nixpkgs repo. That way they will be comfortable to use, security problems will be easier to fix on those packages, etc. (Say, we probably won't build all pythonPackages with 3.6 on Hydra, but that's a tiny detail.) Those versions were typically removed because no known person used them.

[8573]

I want to install Lua 5.0.x globally (assume I don't care about ANYTHING else)

I'm afraid I don't remember the prior art on this from when I used other distros: How is this done with apt-get and other common tools for installing programs globally?

[jeff-hykin]

My point of view is that if there still are good use cases for Lua 5.0, Python 3.6, etc. we should have them as attributes in the current nixpkgs repo. That way they will be comfortable to use, security problems will be easier to fix on those packages, etc. (Say, we probably won't build all pythonPackages with 3.6 on Hydra, but that's a tiny detail.) Those versions were typically removed because no known person used them.

That's good in theory, but just because someone/some group doesn't think it's a good use case doesn't somehow change that I need a particular version. If I'm a security researcher studying vulnerabilities, or a someone working in a government environment with versions that take decades to be approved, or any number of other cases: my "good use case" and your "good use case" are going to be different.

If nix can't meet that need, then I'm just going to use something like brew, apt, or pacman that has a larger package base, and then I'll use luavm, rbenv, and asdf to manage versions on an individual basis like before.

Maybe I'm missing something but what's even the point of having a functional packaging system when you can't reproduce the inputs? There's no use in guaranting an output if you can't reproduce the inputs, which is the same as any other package manager. They all supply "good enough" packages.

[jeff-hykin]

I want to install Lua 5.0.x globally (assume I don't care about ANYTHING else)

I'm afraid I don't remember the prior art on this from when I used other distros: How is this done with apt-get and other common tools for installing programs globally?

In the current state of things without nix, you have to get a version manager specific to that binary: rbenv or rvm for Ruby, nvm for node, etc. Then use those tools to install multiple versions of ruby/node/etc.

Why not just do that?

1. It seems like that would be against the nix philosophy to install a version manager within nix (aka dynamic version mutation instead of functional versioning), since nix itself is supposed to manage versions functionally.
2. In that case, if I was going to do that, I just wouldn't use nix at all and I'd use a normal package manager. But even then its far from ideal having 10 different version managers, each of which are managing versions of language-specific-package-managers (e.g. pip) which manage versioning of packages.

[vcunat]

Maybe I'm missing something but what's even the point of having a functional packaging system when you can't reproduce the inputs? [...]

You can. Just check out the historic nixpkgs commit you want and you will reproduce the build, including all the old dependencies, etc. I've done repo-wide git bisect over long history periods, multiple times, it's really nice. (Most tarballs should get mirrored on cache.nixos.org IIRC and old binary builds are there as well.) Note that typically those packages will depend on vulnerable OpenSSL, for example. If that's what you want... I expect you mainly need a tool that will find you a suitable nixpkgs revision (some of the linked ones perhaps).

[jeff-hykin]

Just check out the historic nixpkgs commit you want

How can I do that when I can't find the nixpkgs commit that has the version I need??? (literally the title of this issue)

[vcunat]

As I wrote you'd need some tool for that, e.g. this one (which I only know thanks to this GitHub issue).

[jeff-hykin]

I expect you mainly need a tool

I'm sorry we are still failing to communicate this: we expect nix to be that tool / incorporate that functionality. Only nix can create/enforce an official versioning standard, and until that happens no tool will have a canonical way search versions.

some tool for that, e.g. this one

The tool is a work-around and exists because a community member had the need so frequently, and the existing method fell so short, that they took the time to build and host an entire website as a make-shift solution which: doesn't search all packages, breaks on non-normal version methods, and breaks when versions are stored as derivations. Lazamar isn't the only one who took the time to create their own workaround, Matthew Bauer wrote a command line one thats more convenient, but it misses more versions than lazamar's tool. Having these tools as the pseudo-official solution really undermines the reliable development cycle I believe nix is trying to create.

[jeff-hykin]

I just don't understand how nix has the best command line installer I've ever seen, a one-liner complete with colors, transparent sudo usage, and explicit mention of all changes to my system. But then immediately falls on its face when I try to complete step 1: install my first package with a version that matches the version needed for my pre-existing project. I shouldn't need to use a community-member-made self-hosted website just to complete step 1 of using a package manager to install a very popular version of a package.

Its like getting a car, but not the keys to the car, and then having to search the internet for where to find the keys, and then having someone refer you to their friend that knows where to get lots (but not all) keys, and that friend can probably get a key that's good-enough if you jiggle it to get it to fit in the ignition. You could search through every possible key in the world yourself, or you could just decide to take the car to the shop and pay to have them change the ignition, either way is less than ideal.

[jeff-hykin]

Is this maybe what you are looking for: https://matthewbauer.us/blog/all-the-versions.html

Thanks a lot for this @markuskowa, that is much closer to what I've been looking for. Sadly it lists less versions than lazamar's but it's much easier to use. You might want to add the example to your comment so people don't need to read most/all of the blog post to get the answer. Maybe something like:

To get versions for emacs run

nix eval "(builtins.attrNames (import (builtins.fetchurl https://matthewbauer.us/generate-versions.nix) {}).emacs)"

(it will take awhile)

To get versions for firefox run

nix eval "(builtins.attrNames (import (builtins.fetchurl https://matthewbauer.us/generate-versions.nix) {}).firefox)"

To install version 25.0.1 of firefox run

LC_ALL=C nix run "(import (builtins.fetchurl https://matthewbauer.us/generate-versions.nix) {}).firefox.\"25.0.1\"" -c firefox

[vcunat]

Having these tools as the pseudo-official solution really undermines the reliable development cycle I believe nix is trying to create.

I believe that stuff discussed in this thread so far has never been meant among the main applications of NixPkgs, but I don't see that as a problem. Don't get me wrong, I'm (personally) not at all opposed to having some "more official" way of finding the versions you need.

But the process starts the same – people from the community implement stuff and then it gets merged, documented, etc. For big changes it may be advisable to go through an RFC beforehand (if you want to avoid working on something that gets refused in the end).

[jeff-hykin]

But the process starts the same – people from the community implement stuff

Fair enough, thats a really good point 👍

it may be advisable to go through an RFC

Thanks for the advice, I didn't know of nix's RFC system. I think this issue is hoping to create a community discussion to develop an actionable of a plan of how to best integrate versions with nix. Basically: listen to people's use-cases, hear from users who understand nix's feature set and goals, talk to design a system that is ideal both to use and to implement, then, if it requires major changes to nix, go the route of an RFC and develop a pull request.

[jeff-hykin]

so far has never been meant among the main applications of NixPkgs

I think I am genuinely misunderstanding some aspect about nix. Maybe this thread isn't the place for this question: but what are the other main applications? I've read the Nix Package Manager Guide, but everything in there about reproducibility seems to point to installing (and therefore finding) specific versions. I mean the first title in chapter 1 is multiple versions

Any other multiple-simultaneous-version managers: asdf, kiex, gvm, rvm, rbenv, pyenv, jenv, nvm, Renv, luavm, all have a list-available-versions as either their 1st or top-5 most-used commands. But nix has 5.3K stars, so there must be something I'm not understanding if nix is missing something so vital

[vcunat]

OK, I'll try to keep explaining 🤔

Command that lists versions... has always been there:

$ nix-env -qa llvm | cat
llvm-10.0.0
llvm-5.0.2
llvm-6.0.1
llvm-7.1.0
llvm-7.1.0
llvm-8.0.1
llvm-9.0.1

but in NixPkgs we choose to restrict the number of maintained versions (which has been discussed at length in these threads), so there are few packages that have multiple versions and having this many is very rare. That approach is take by most distros, because QA is best that way. I understand that this combination can seem confusing.

But even that way the multi-versioning gets utilized a lot, e.g. for rollbacks. Perhaps it's worth noting that with Nix the "installed versions" bring all transitive dependencies down to libc and they can contain configuration (or whole OS), contrary to typical multi-version management.

So instead we focused this discussion on installing versions from other/"historic" NixPkgs commits, and there's no built-in tool to search in there... well I tried to explain I personally see limited use for that. I think usually people choose the NixPkgs version they want (say, latest release-20.03) and not some particular package version (apart from cases like LLVM) – or alternatively they already had some package(s) and they keep them "unupdated" while replacing others with different versions. Also Flakes are now being introduced as another way of "managing versions".

[jeff-hykin]

Okay, that was helpful and better explains #9682 (comment)

In that case though, why would Nix opt to concurrently maintain multiple versions when historic versions can be depended upon? In other words, why do this (my current understanding of the packages):

current
    llvm-10.0.0
    llvm-5.0.2
    llvm-6.0.1
    llvm-7.1.0
    llvm-7.1.0
    llvm-8.0.1
    llvm-9.0.1

historic commit#a35lk6djfi3
    llvm-5.0.2
    llvm-6.0.1
    llvm-7.1.0
    llvm-7.1.0
    llvm-8.0.1
    llvm-9.0.1

historic commit#fjo350395
    llvm-5.0.2
    llvm-6.0.1
    llvm-7.1.0
    llvm-7.1.0
    llvm-8.0.0

Instead of doing this:

current
    llvm-10.0.0

historic commit#a35lk6djfi3
    llvm-9.0.1

historic commit#a35lk6djfi3
    llvm-9.0.0

historic commit#G98olk6djf
    llvm-8.0.1

historic commit#C35lk6djfi3
    llvm-8.0.0

historic commit#B252335lk6
    llvm-7.1.0

historic commit#Hfh593fjo35
    llvm-7.0.0

If the answer has to do with this:

bring all transitive dependencies down to libc

Why can't current-packages depend on packages in historic commits?

[vcunat]

They could. Well, it's not as simple due to many "packages" being involved in the dependency graph, but some variants of this are not difficult. If there's a good use case for some old versions, we just bring them to current NixPkgs, because it's easier to manage. EDIT: well, Flakes will be a comfortable way, too, IIRC.

[jeff-hykin]

because it's easier to manage

is there a resource I could read to explain what makes it easier? Closest thing I have found is a video clip here

Without detailed knowledge, it seems to me that it would be better to create a consistent flat system (that depends on history) rather than some kind of mixed system with an arbitrary "good-enough" use case as a decision point. Why have a decision point instead of creating tooling that would make it easy to use any historic version, regardless of who thinks a version is good enough or not.

[vcunat]

That way it's easier e.g. to fix serious (security) problems in the packages, including all their dependencies. Also, packages don't live in isolation – one NixPkgs commit is a set of packages that should work together well (for example, the build farm runs VM tests on full systems). A different combination (of historic versions) might work well or it might not, especially for packages that are more tightly coupled to each other.

[thomastjeffery]

Don't forget that there are two timelines happening here:

1. The timeline for versions of a package.
2. The timeline for the derivation implementing a specific package version.

Sometimes, you want to change something in a derivation that implements an older package version. For example, the source for llvm-8.0.1 might be moved tomorrow, which would break any new builds of your llvm-8.0.1 derivation.

[jeff-hykin]

I believe I have a relatively simple back-end solution; a database with mappings for every package e.g. (package version) => (commit hash). Every commit to master would trigger an update to the database using a CI hook.

This would also optimize nix-env -qA since it's currently significantly slower than doing a curl call to lazamars site. It should also partly address your recently mentioned concerns @thomastjeffery

I believe currently the channel(s) are being scanned for versions of a package, which is pretty inefficient. Lazamar's tool is doing 5 week snapshots which is missing some versions.

The "database" need not be an actual database, although it certainly could be. In theory it could be a single JSON file although that would be impractical. To get around having one giant file, there could be a json file for each package, with versions as the keys and commit hashes as values
E.g.

nameOfPackage.json
(package version) => (commit hash)

The data structure could also be made more complex to show which versions of the package were available on a stable channel.

Note the commit hash would not be the first (aka oldest) commit with the version but instead the chronologically most-recent commit with the version. This is intentional to address the issue @thomastjeffery mentioned where the source of the version changed, even though the version did not.

I've built similar versions of all the components needed to implement this, the only thing slowing me down is learning nix, which I have sunk a considerable amount of time into between reading the manual and trying to understand nix --help.

Examples for the following would be really helpful for making progress on this

1. A command for searching for versions of a package within a specific commit
2. [completed] A command for listing all package names from specific commit
3. [completed/discovered] A command for installing a package derivation from a pinned commit
4. [completed/discovered] The code for a shell.nix that would install a specific deviation from a pinned package

If I can get those working then there will be a clear path towards an RFC

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/re-using-nixpkgs-derivations-for-different-package-versions/9236/2

[vivook]

I'm glad I found this thread and the one it came from. I just installed nix and have just uninstalled it. There are 2 reasons:

1 - not possible to search for which package contains a binary (e.g. where is sed?)
2 - not possible to pin to arbitrary versions without jumping through the hoops described here

It's a shame. I thought this could be a good solution to building reproducible developer environments like the web site says, but I guess not. It's beyond me why @jeff-hykin had to write so many posts to explain the simple concept of point 2 above.

Farewell nix 👎

[rsynnest]

1 - not possible to search for which package contains a binary (e.g. where is sed?)

@vivook this is totally doable! You can install nix-index which provides the nix-locate command which does exactly what you need. I discovered this via the very handy Nix to Debian phrasebook

curl -L https://nixos.org/nix/install | sh	# reinstall nix :)
nix-env -iA nixpkgs.nix-index			# install nix-index
nix-index					# build the index, takes a while (~5 minutes)
nix-locate --top-level --minimal --at-root --whole-name "/bin/sed"
# RESULTS:
# toybox.out
# gnused_422.out
# gnused.out
# busybox.out

Another cool approach is comma, which transparently runs a nixpkgs binary without actually installing anything to the system. Great for trying stuff out. So in your case you would install comma then run , sed --help

You can also search packages the traditional way with nix search. You may have to get creative with such a short common name like "sed", but remember nix search also searches package descriptions. nix search " sed" (note the leading space) works great, returns 5 packages with the 2nd one being sed. Most things are much easier to find.

[rsynnest]

Back to the topic of discussion, to me, this version history feature seems like a worthwhile addition. Communities like Ruby, Python, etc. provide dozens of versions going back decades, despite the fact they are no longer maintained, so why not Nix? The popularity of tools like nvm, pyenv, etc shows there is a clear user demand, and I think Nix is uniquely capable of offering a clean solution to this common user problem.

@jeff-hykin 's proposal of a database tracking versions seems like a good approach to an up-to-date metadata service like this. This seems like it would be a good fit for integrating into NixOS Search.

Guix has some similar tooling setup for browsing past versions of packages and even comparing derivations across versions. Perhaps this upcoming RFC could borrow from there or reach out for tips? Judging by their README they are doing something very similar to what @jeff-hykin is proposing.

[yuuyins]

it's pretty obvious to me nix should have a simpler way for users to install a version not in their nixpkgs handling all the needed logic in a transparent way, i.e. i just say pname-version, did not find it in active nixpkgs/input? automagically try finding older definitions someway and do the logic someway... found derivation definition and not in the cache? nix says "we don't support pname-version you're on your own" and builds it. and for those who want more control to manually keep using overlays/overrides, different input revisions for nixpkgs, etc.

another thing is that nix just collides binaries when there are different versions with same binary name. i think for that nix should instead automatically append version to the binary name (better yet, keep the collided binary but symlink the binary to pname-version) so that it will be available to PATH. well appending version only would not solve the issue, we would need the hash as well pname-version-hash, or part of that hash. unison-lang.org might shred some light into this issue as well.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixpkgss-current-development-workflow-is-not-sustainable/18741/62

[jeff-hykin]

The root reason for me is; the maintainers seem indifferent to disscussions around fixing foundational problems that totally undermine reliability/stability/purity.

The issue with a particular package ? => just create an issue on the package manager (nixpkg) repo
and want to publish a tiny update to your package? => you need a PR against nixpkg directly

were just nails in the coffin below the flashing "this is un-maintenable/un-scaleable" sign 😕

I'm excited to announce these^ statements may no longer be true as of that conversation in the comment above
Nixpkgs current development workflow is not sustainable.

Update for this Issue

Version search is a small piece of a larger issue: general package search-ability. And I'm hopeful the discussion/proposal here will get some traction since it paves the way for solving this problem and the maintainability problem.

I should mention that my old statement:

How can I possibly care about the new flakes system when there isn't even a canonical name/version for packages?

has aged quite poorly. Now that I have much more experience and understand flakes, I can say the Nix overhaul I was talking about is (or can be) flakes. And core contributors have been working on that since before this issue was opened. Flakes are a standardization of package format, and this proposal is trying get searchable fields added to the format.

[nxg]

This is something of a +1, but expressed as a (re?)statement of a use-case, which I haven't seen, in black and white, in this issue or its progenitor (but which I may very well have missed in the epic flow). This is related to @jeff-hykin 's point in a comment above.

Say I have a collection of nix-installed software that works for my project, now, and I want to be able to reproduce that in future or tell someone else how to. How do I do that? That ‘tell someone else’ could be me writing in a publication ‘download this published data from <URL>, then (install NixPkgs and) install tools A, B and C using nix-shell my-magic-software-set.nix, and you will see what I'm seeing and can reproduce my research’. I want those instructions to be good for at least ten years. How do I assemble the contents of my-magic-software-set.nix?

If I have a working suite of software versions just now (but I know the previous release had a bug and I'm worried the next one will have a regression), I can look at the $PATH to work out the hash of the derivation and then I can write that down somewhere, and later, if I'm understanding Lazamar's site correctly, use that to retrieve that particular version, and all of the dependencies of that specific version. I might end up rebuilding the world, but that's OK. That's all a bit DIY, though – I'd hope that NixPkgs would make this a lot easier, and that it was documented somewhere other than a github issue.

This process is what I personally thought was NixPkgs USP, and the fact that I couldn't find anything about it in the manual was puzzling me greatly until I tried searching the manual with google, and stumbled across the progenitor issue here. To be clear: I long believed that the above scenario was the point of NixPkgs.

Note: despite the issue title, this isn't a question about versions, as such. One may agree with commenters above, that ‘versions don't really mean anything’, or with the less extreme position that ‘at least they're useful as searchable metadata’. I don't necessarily care what versions are in my magic-software-set, I just have to freeze them, for the sake of my own sanity and that of my article's readers.

The above assumes I get a hash by carefully storing it when I've got a working version (that's what notes/research-logs/record-keeping is for). It would be nice to be able to search (including searching versions-as-metadata) for what was the current versions when ‘damn – I'm sure this was working six months ago – what version was Nix-current then?’ Lazamar's database can help here, but I find myself here, trying to work out how to unearth a package which isn't one that they cover. And as others have said, one is surprised to have to rely on a third-party site for this.

Despite the length of this issue (and indeed of this comment), I'm still convinced that I'm missing something obvious, and that someone is about to say that this particular use-case is already trivially satisfied by X, Y and Z (where none of X, Y or Z are ‘clone the entire git repo’). I'm sure that TFM is trying to tell me this, but it assumes I am more comfortable with the Nix language than I am (it's clearly a great language, and I'm sure it'll be fun to learn at some point, but that might be a project for another day).

[8573]

Say I have a collection of nix-installed software that works for my project, now, and I want to be able to reproduce that in future or tell someone else how to. How do I do that? That ‘tell someone else’ could be me writing in a publication ‘download this published data from <URL>, then (install NixPkgs and) install tools A, B and C using nix-shell my-magic-software-set.nix, and you will see what I'm seeing and can reproduce my research’. I want those instructions to be good for at least ten years.

What about statically compiling the program and storing the executable? I imagine that could be cheaper in both effort and storage space than directly solving this ticket's posited problem and needing to keep all the (transitive!) dependencies around somewhere (at least all the source code, which counts as dependencies in Nix). I think (or am I wrong?) that currently the NixOS Foundation is bearing all the costs of storing built nixpkgs packages indefinitely, but I don't suppose that can continue for ever.

[nxg]

What about statically compiling the program and storing the executable?

Static compiles might indeed be a possibility in some cases, though in the case of macOS, which relies heavily on dynamic libraries, I suspect building a ‘static’ executable could be a project in itself (though this isn't something I've investigated in depth).

The problem is that that wouldn't deal with the case of publishing a recipe (and that's literally publishing, in a journal article): ‘do X and you will be able to reproduce these results yourself.’

Re storage: I wouldn't expect to be able to download packages of historical derivations, merely the link to the location of the source (third-party location), the patches and build recipe (in NixPkgs), and the transitive links to the dependencies (also NixPkgs). My understanding is that those patches and build recipe are what are hosted by NixPkgs, not the source. (But I'm commenting in some ignorance here, since I'm aware that Nix would be a lovely rabbit-hole to explore, so I'm being cautious about how far I stray behind the front desk.)

Obviously, this depends on a third-party source location not changing, and that can't be guaranteed. There's a different research-reproducibility rabbit-hole we could run down here, but if a build broke because the source had disappeared, then boo-hoo, but it's not NixPkgs fault.

[jeff-hykin]

I'm still convinced that I'm missing something obvious, and that someone is about to say that this particular use-case is already trivially satisfied by

@nxg

You're not missing anything if you're aware of what's below.

You can use nix-pinning to get old versions (all results on lazamars site show the code for pinning). But, there is a risk. If package1 is from nixpkg commit1 and package2 is from nixpkgs commit2, there is no guarantee they will work together. For standalone binarys that's fine, but for packages that interact (ex: package1=python3, package2=numpy) it can result is terrible dynamic/shared library issues. A lot of that isn't nixpkgs fault, it's just the bad modern system design we have, like the recent breakage that glibc caused. Additionally there is no guarantee that a package will behave the same on a different system (linux/mac).

If all the packages for a project are from the same nixpkg commit, then it is very realistic to freeze all of them. If an example would help, let me know and I'll provide the code. Create a shell.nix file, use the mkShell tool within the nix language, load a pinned nixpkgs (as a variable), then list all the packages in build inputs. Once that file is made, it's as simple as running nix shell --pure in that directory for anyone to reproduce the project environment exactly (security flaws included).

The downsides are:

• system dependencies must be added manually, there is no equivalent to apt-get install
• package dependencies (pip install, npm install, gem install, go get, etc) not only can't be automated, but are actually much more difficult than system dependencies. Because node and an npm package interact, custom nix code is needed. Understanding nix lang becomes much more of a requirement, and even then it can be quite difficult to get things working (overlays and overrides).
• must use lazamars tools for finding older versions, and risk compatibility issues especially with package dependencies
• tools like git/npm still read from your $HOME and can cause "well it works on my machine" behaviors
• trying to add two binaries with the same name (like two versions of git) does not have an elegant solution, nixpkgs just goes with the last one mentioned in the shell.nix
• anything that needs open ssl certificates can have problems
• the default shell is bash and there's no way to change it

[jeff-hykin]

Some tools like niv have tried to alleviate some downsides. I haven't found any particularly comprehensive, understable or easy to use. I finished my own framework last year to reduce a lot of these problems, but I want to simplify the API before doing a public release.

[nxg]

Many thanks, @jeff-hykin – what I was missing was ‘nix-pinning’. Googling for that, I found ‘Towards reproducibility: pinning Nixpkgs’, which I think I possibly should have been aware of already (is that list of guides new?).

With that, and with a bit of trial-and-error at https://status.nixos.org, I was able to find a channel which contained the specific tool version I needed. That means that I also now understand what Lazamar's tool is doing (each Nix ‘channel’ is a git branch, and the tool identifies the first commit on each channel/branch which has the searched-for tool version – yes?); I was clearly confused about that before.

I completely agree with you on the list of challenges to actual reproducible software. Pinning lets NixPkgs provide step one of a process, and I don't think NixPkgs need aim further up the ladder than that. Even thinking of wrestling with Python package versioning within NixPkgs seems overambitious to me (venvs are everyone's friend!).

As is often the case after one has been cordially pointed to the right page of the FM, I feel a little foolish at not having known the answer already. I don't remember seeing the current contents of learn before, with the list of guides prominently higher than the discussion of Nix pills: if that's always been there, then now I do feel a bit dim; if that's newish, then excellent work whoever reworked the page.

Final reflections:

• This does match the use-case I described, and is just about as easy as it should be, once one knows that ‘pinning’ is the right term.
• I now see that this present issue is about something different: ‘I'm fine with the channel I'm on, but I want a different version of specific package X’. The above discussion makes more sense now than it did. It's always a little hard to work out, post-enlightenment, why one was confused, but in case it's useful...
• The notion of ‘channel’ is not, to me, as clear and obvious as the author of the relevant section of the manual thinks it is. That text describes a channel as ‘a place that contains a set of Nix expressions and a manifest’ – that's a definition in terms of the implementing technology. A definition that's more hand-waving, but possibly more accessible to the naive (eg, me) would be something like ‘a channel is a set of package versions which are expected to work together; the most prominent example of a channel is the set of packages in a “release” such as nixpkgs-21.09-darwin’. Realising that would I believe have given me more traction with the NixPkgs docs at an earlier stage.

This issue is probably as much a documentation issue as a functionality one. The challenge of writing documentation is always the challenge of trying to remember not understanding something. I hope that recording here the nature of my ignorance, while that precious ignorance is fast-fading away, will help advance the issue a little bit.

[yuuyins]

This issue is probably as much a documentation issue as a functionality one. The challenge of writing documentation is always the challenge of trying to remember not understanding something. I hope that recording here the nature of my ignorance, while that precious ignorance is fast-fading away, will help advance the issue a little bit.

@nxg https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html#flake-inputs

but it is missing using in outputs, like using multiple nixpkgs through specialArgs

[SuperSandro2000]

There are three ways this can be accomplished:

1. If there is already a derivation provided by the user's channel, the user need only find its attribute path name. This has gotten easier with nix search, but isn't always reliable. Usually, packages follow the naming convention of appending the version to the package name, but this is not always the case.

This is essentially fixed by search.nixos.org, nix search and a search through the source code in worst case.

2. If there is not a derivation provided in the user's channel, the user must find a hash that points to the desired derivation version, and use pinning to refer to the package by hash.

This would result in mixing nixpkgs versions which generally works but can lead to major problems due to impureness in some parts of nixos like the graphics driver or every program that uses LD_LIBRARY_PATH, so it generally should be avoided whenever possible.

If nix can't meet that need, then I'm just going to use something like brew, apt, or pacman that has a larger package base

That is wrong, depending on how you count packages, nixpkgs is biggest package repository and closely behind is the AUR.

and then I'll use luavm, rbenv, and asdf to manage versions on an individual basis like before.

Those tools where created because all the classical package manager also did not support every version of an ecosystem. For nix there are already some community projects which try to do this for some ecosystems and that is place where I see those projects. Having them outside of nixpkgs allows them to rapidly make changes, point there focus on the specific ecosystem or even use features which are not suitable for nixpkgs like IFD. Also fully automated imports of an entire ecosystem like pypi are possible which are not suited for nixpkgs.

---

I read some comments in this thread and the only actionable things except the general well known documentation problem that I identified would require creating a RFC because they changes they require are beyond big.

[jeff-hykin]

After 3 years a lot has changed, but I've never stopped thinking about (or stopped running into) this problem. Literally as I'm writing this I'm waiting on a nix-shell building python 3.7 using a nixpkgs commit from 2018. The LD_LIBRARY_PATH issue you mention @SuperSandro2000 is very present as well.

If nix can't meet that need, then I'm just going to use something like brew, apt, or pacman that has a larger package base

That is wrong, depending on how you count packages, nixpkgs is biggest package repository and closely behind is the AUR.

I didn't mean this as an attack but rather a statement of practicality; even right now I'm having to use apt-get and pyenv while I'm unable to get python3.7 working in nix-shell. Doesn't mean I'm not trying to get it working in nix, it just means I can't rely on it. "Bigger" is just the number of times I can brew install/apt-get/pacman something, but can't nix-install the same thing (like the arduino-ide on M1 or the moveit simulator for ROS). Maybe including pacman in the list was unfair as I don't use it all that often.

the only actionable things

I do want to focus this issue on the actionable items instead of complaints.

would require creating a RFC because they changes they require are beyond big

I agree, there is no quick fix, even just making a full roadmap at this moment is impossible because there is so much to do.
That said, actions can (and are) being taken, so here's what I've noticed and what I have been exploring.

Action Item 1: Solve the LD_LIBRARY_PATH (permanently)

I have spent a lot of time learning about dynamic linking, glib, ldd, patchelf etc specifically for this problem. I still have a lot left to learn, like getting caught up with how nix-ld works, and if it can be used for this. Based on what I do understand, it seems very much possible that the whole LD_LIBRARY_PATH problem is solvable by constantly updating the RPATH, rather than having lib files that break whenever LD_LIBRARY_PATH is specified. "Possible' doesn't mean without major complications. Instructions that would normally say "add X to your LD_LIBRARY_PATH" will need to be converted into running a patching tool that changes the RPATH for things like .venv files, and if patching existing nixpkgs files, a tool will need to create new derivations with the modified RPATHS. Again, I still don't see anything that makes it impossible and I've been exploring it with some hacked together tools since I often have to mix and match nix commit versions to get packages I need.

I think mixing versions of nixpkgs is necessary to meet all the practical situations of real-life projects, and this is a step towards making it feasible for users who know nothing about dynamic libraries.

If you think this is impossible (literally impossible not just impractical) I'd be happy to hear about it @SuperSandro2000

Action Item 2: Solve "a search through the source code in worst case"

This is essentially fixed

It is not fixed.

Great progress has been made, and I'll discuss actions in a moment, but it is necessary to make it clear searching (not even searching for a version) is still a major problem for nixpkgs. This one I've spent the most time on, and have (what I think is) a good grasp on.

Action Item 2.1: Binary Name search

The smaller aspect of binary name search is one of the problems I believe has been effectively solved. For example, back in 2020 it was extremely difficult for me to find a package for ls because there was no package named ls. At the time I ended up installing lscolors as a makeshift solution because thats all I could find. Now, using search.nixos.org, searching for ls returns coreutils-full as the first result which is what I would've wanted in the first place. A fantastic improvement.

Problem 2.2: Nested Attributes

Now, lets search for another common package AVFoundation for MacOS. It is needed quite frequently for building and does exist on nixpkgs with an exact attribute-name match.

As a user I would probably conclude Nixpkgs just doesn't have it, but lets try searching for it using Github anyways.

None of the first several results give hints as to what to put for nix-env -iA. I'll leave it as an exercise for the reader to figure out the attribute path.

Mechanical, electrical, and civil engineers use apt-get all the time without ever knowing what "grep" is or how to use it, much less vim/git. Yet a common package on nix requires someone to understand advanced file grepping, cloning repos, and/or inner knowledge of how nix structures itself to come up with more clever searches that narrow down the results. I will consider this an issue worth solving as long as this remains the case.

Action Item 2.3: Uniformity

Part of the problem with matching nested attributes (causing the problem above) is the lack of uniformity across packages. There is a recent RFC for flattening Nixpkgs into a uniform structure, that is a major step in the right direction for both maintainability and making it easier to search. That said, it doesn't address nested packages yet.

Right now nixpkgs is still spagetti, uniform packages helps a lot but there's no clearly communicated difference between a package used by nobody and a package that is essential just for stdenv to exist, or a folder with a default.nix that isn't even a package. More confusingly are the packages that are unnecessarily both core and non-core because they're run multiple times using fixpoint recusion. Two months ago I created nixpkgs-skeleton to try and slice through this confusion to help myself and others understand what parts of nixpkgs were "core". Its a far cry from cleaning everything up but its a start.

Action Item 2.4: Indexing & Exploration

To create a searchable interface, there needs to be some way of exploring nixpkgs, and there's at least 2.

1. Through nix evaluation
2. Through static anaylsis

Action Item 2.4.1.1: Static Analysis: Tree Sitter

Last month I finally made a static anaylsis breakthrough getting the nix-tree-sitter running (written by someone else) and using it to create a nix-bundler system (simlar to how javascript bundlers work). This is the basis for extracting information about nested package attributes, but has some common static analysis problems I'll mention in 2.4.1.3.

Action Item 2.4.1.2: Static Analysis: RNix

One week after that I found out about the rnix-parser which is a more comprehensive parser written in rust. Its what is being used for automatic re-writes for making the nixpkgs structure flat.

Action Item 2.4.1.3: Static Analysis: Problems

There's at least two problems that hurt static analysis significantly.

1. The with keyword (syntactic sugar) ruins static analysis. Its impossible to know the source of a variable inside mutliple with statements, and even inside only one with statement it is problematic. I would like to eventually propose deprecating with, similar to how Javascript deprecated their own with keyword for similar reasons. Actually it looks like someone has beat me to it.
2. Highly dynamic fields, such as packages that accept their version number as an argument, and then interpolate it into a string are difficult to analyze.

Action Item 2.4.2: Evaluation as Analysis

The try-to-evaluate functionality of nix is somewhat limited and difficult to use. For example, back when I first started using nix-env queries to build a search index, there were certain packages on certain nixpkgs commits that fail on darwin, causing the entire query to fail, meaning no data about any packages after that are returned. Instead there would need to be 80,000 individual queries just to make sure that failures of one package didn't make it impossible to get information about other packages.

More recently I've attempted to write an expression for recursively exploring packages, to drill down into those attributes. However there is a similar issue, packages fail to evaluate, some failures cannot be caught, so I had make a fork and edit out all the cases that would throw uncatchable errors. Having a mode for exploring nixpkgs that allows for everything to be caught would be quite helpful.

Another aspect is that because of the lazy evaluation try-eval often needs to be combined with deepSeq which often doesn't work for getting around the problem. As I learned recently, there is a makeshift work around specifically for derivations recurseForDerivations which I'll be checking out next time I'm working on this task.

Evaluation is more comprehensive since it accounts for dynamic attribtues, but this can also cause problems when evaulating packages for non-supported systems. For example, trying to get the version number of a cuda package on a non-cuda machine.

I worked a bit on a propsal for flakes to have a meta or static tag that was evaluatable on every machine. But later settled that the flake.lock format would be better suited for storing indexable data.

Action Item 3: Version Publications

Those things above are mostly just working on package discovery, even without versions, but for versions, compatibility checks etc, final action is working on a system of simulaniously distributed and centralized packages based of this. Where packages can be published and added to a registry independent of whether or not they are integrated into nixpkgs. This will require a server with authentication listening for publish commands, as well as its own search system. I can't say I'm close, but I have been writing code for this for a couple years now.

In terms of actionable steps, getting flakes nailed down will be extremely helpful for this case. Both with the flake-lock being great equivlent to something similar to a package.json, and with the inputs more easily allowing pulling in code from sources other than nixpkgs.

TLDR

There aren't many messages, exciting demos/changes, but a lot of actions have been done and are worth doing towards solving this issue. I might complain a lot, but its because I want nixpkgs to be all it can be instead of what it is.

[SuperSandro2000]

Action Item 2.1: Binary Name search

The smaller aspect of binary name search is one of the problems I believe has been effectively solved. For example, back in 2020 it was extremely difficult for me to find a package for ls because there was no package named ls. At the time I ended up installing lscolors as a makeshift solution because thats all I could find. Now, using search.nixos.org, searching for ls returns coreutils-full as the first result which is what I would've wanted in the first place. A fantastic improvement.

That's probably pretty easy to fix when we have replaced command-not-found with nikx-locate, someone would just need to write some converter to elastic and expose i on the website.

Now, lets search for another common package AVFoundation for MacOS. It is needed quite frequently for building and does exist on nixpkgs with an exact attribute-name match.

There is probably a recurseInto somewhere missing or we are not evaluating those parts on linux, so those things are missed. Not fully sure.

---

Don't really have something to say on the other things right now.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/best-practice-for-pinning-version-of-individual-packages/6194/7

[dudicoco]

devbox solves this problem by letting you specify package versions within a json file, it's a great nix wrapper for mere mortals.

[jeff-hykin]

So I think there's still a problem worth solving here, but it's not really a problem with devbox as much as it is that this issue thread is an upstream issue for devbox.

I've been working on/using a devbox-like tool ("virkshop" the virtual workshop). I want to provide similar features like versions but both myself and devbox can't AFAIK because:

1. It's not possible to programatically find/list all available versions (please correct me if devbox has invented their own static-analysis or dynamic-analysis tool that solves this; that kind of solution would close this whole issue thread)
2. Even when you find a package, pinning the nixpkgs version (which devbox does let you do) will cause stuff like the LD_LIBRARY_PATH problem that SuperSandro2000 brought up a few comments ago.

All that said; I would still highly recommend trying out devbox in the meantime! Once this issue is fixed devbox should simply work better than ever before. I didn't know about it till it was mentioned here (thank you @dudicoco), and I'm pretty excited to try it out and see if it can fully replace the tool I've been working on.

[dudicoco]

So there's still a problem, but it's not really with devbox moreso this issue thread is an upstream issue for devbox.

I've been working on/using a devbox-like tool ("virkshop" the virtual workshop). I want to provide similar features like versions but both myself and devbox can't because:

1. It's not possible to programatically find/list all available versions (please correct me if devbox has invented their own static-analysis or dynamic-analysis tool that solves this; that kind of solution would close this whole issue thread)
2. Even when you find a package, pinning the nixpkgs version (which devbox does let you do) will cause stuff like the LD_LIBRARY_PATH problem that SuperSandro2000 brought up a few comments ago.

All that said; I would still highly recommend trying out devbox in the meantime! Once this issue is fixed devbox should simply work better than ever before. I didn't know about it till it was mentioned here (thank you @dudicoco), and I'm pretty excited to try it out and see if it can fully replace the tool I've been working on.

@jeff-hykin devbox does support searching for available package versions within its CLI: https://www.jetpack.io/devbox/docs/cli_reference/devbox_search/

[jeff-hykin]

Answer Update: How to find and install a specific version

Note: Devbox is not designed for this, but it is the most exhaustive solution at the moment. Devbox is a framework, it's not really designed to work "with nix", it moreso uses nix under the hood. This tutorial is only how to use it to find/install nix packages versions.

# install devbox
curl -fsSL https://get.jetpack.io/devbox | bash       # can also use nix: nix-env -iA nixpkgs.devbox

# exhaustive search
devbox search nodejs --show-all

# nodejs  (20.3.1, 20.2.0, 20.0.0, 19.9.0, 19.8.1, 19.8.0, 19.7.0, 19.6.1, 19.6.0, 19.5.0, 19.4.0, 19.2.0, 19.1.0, 19.0.1, 19.0.0, 18.16.1, 18.16.0, 18.15.0, 18.14.2, 18.14.1, 18.14.0, 18.13.0, 18.12.1, 18.12.0, 18.10.0, 18.9.1, 18.9.0, 18.8.0, 18.7.0, 18.6.0, 18.5.0, 18.4.0, 18.3.0, 18.2.0, 18.1.0, 18.0.0, 17.9.0, 17.8.0, 17.7.2, 17.5.0, 17.4.0, 17.3.1, 17.3.0, 17.2.0, 17.1.0, 17.0.1, 16.20.1, 16.20.0, 16.19.1, 16.19.0, 16.18.1, 16.18.0, 16.17.1, 16.17.0, 16.16.0, 16.15.1, 16.15.0, 16.14.2, 16.14.0, 16.13.2, 16.13.1, 16.13.0, 16.12.0, 16.10.0, 16.9.1, 16.9.0, 16.8.0, 16.7.0, 16.6.2, 16.6.1, 16.6.0, 16.5.0, 16.4.2, 16.4.1, 16.4.0, 16.3.0, 16.2.0, 16.1.0, 16.0.0, 15.14.0, 15.13.0, 15.12.0, 15.11.0, 15.10.0, 15.9.0, 15.8.0, 15.7.0, 15.6.0, 15.5.1, 15.5.0, 15.4.0, 15.3.0, 15.2.1, 15.2.0, 15.1.0, 15.0.1, 14.21.3, 14.21.2, 14.21.1, 14.21.0, 14.20.1, 14.20.0, 14.19.3, 14.19.2, 14.19.1, 14.19.0, 14.18.3, 14.18.2, 14.18.1, 14.18.0, 14.17.6, 14.17.5, 14.17.4, 14.17.3, 14.17.2, 14.17.1, 14.17.0, 14.16.1, 14.16.0, 14.15.5, 14.15.4, 14.15.3, 14.15.1, 14.15.0, 14.14.0, 14.9.0, 14.8.0, 14.7.0, 14.6.0, 14.5.0, 14.4.0, 13.14.0, 12.22.12, 12.22.11, 12.22.10, 12.22.9, 12.22.8, 12.22.7, 12.22.6, 12.22.5, 12.22.4, 12.22.3, 12.22.2, 12.22.1, 12.22.0, 12.21.0, 12.20.2, 12.20.1, 12.20.0, 12.19.1, 12.19.0, 12.18.4, 12.18.3, 12.18.2, 12.18.1, 12.18.0, 10.24.1, 10.24.0, 10.23.3, 10.23.2, 10.23.1)
# nodejs-slim  (20.3.1, 20.2.0, 20.0.0, 19.9.0, 19.8.1, 19.8.0, 19.7.0, 19.6.1, 19.6.0, 19.5.0, 19.4.0, 19.2.0, 19.1.0, 19.0.1, 19.0.0, 18.16.1, 18.16.0, 18.15.0, 18.14.2, 18.14.1, 18.14.0, 18.13.0, 18.12.1, 18.12.0, 18.10.0, 18.9.1, 18.9.0, 18.8.0, 18.7.0, 18.6.0, 18.5.0, 18.4.0, 18.3.0, 18.2.0, 18.1.0, 18.0.0, 17.9.0, 17.8.0, 17.7.2, 17.5.0, 17.4.0, 17.3.1, 17.3.0, 17.2.0, 17.1.0, 17.0.1, 16.20.1, 16.20.0, 16.19.1, 16.19.0, 16.18.1, 16.18.0, 16.17.1, 16.17.0, 16.16.0, 16.15.1, 16.15.0, 16.14.2, 16.14.0, 16.13.2, 16.13.1, 16.13.0, 16.12.0, 16.10.0, 16.9.1, 16.9.0, 16.8.0, 16.7.0, 16.6.2, 16.6.1, 16.6.0, 16.5.0, 16.4.2, 16.4.1, 16.4.0, 16.3.0, 16.2.0, 16.1.0, 16.0.0, 15.14.0, 15.13.0, 15.12.0, 15.11.0, 15.10.0, 15.9.0, 15.8.0, 15.7.0, 15.6.0, 15.5.1, 15.5.0, 15.4.0, 15.3.0, 15.2.1, 15.2.0, 15.1.0, 15.0.1, 14.21.3, 14.21.2, 14.21.1, 14.21.0, 14.20.1, 14.20.0, 14.19.3, 14.19.2, 14.19.1, 14.19.0, 14.18.3, 14.18.2, 14.18.1, 14.18.0, 14.17.6, 14.17.5, 14.17.4, 14.17.3, 14.17.2, 14.17.1, 14.17.0, 14.16.1, 14.16.0, 14.15.5, 14.15.4, 14.15.3, 14.15.1, 14.15.0, 14.14.0, 14.9.0, 14.8.0, 14.7.0, 14.6.0, 14.5.0, 14.4.0, 13.14.0, 12.22.12, 12.22.11, 12.22.10, 12.22.9, 12.22.8, 12.22.7, 12.22.6, 12.22.5, 12.22.4, 12.22.3, 12.22.2, 12.22.1, 12.22.0, 12.21.0, 12.20.2, 12.20.1, 12.20.0, 12.19.1, 12.19.0, 12.18.4, 12.18.3, 12.18.2, 12.18.1, 12.18.0, 10.24.1, 10.24.0, 10.23.3, 10.23.2, 10.23.1)
# nodejs_latest  (20.0.0, 19.9.0, 19.8.1, 19.8.0, 19.7.0, 19.6.1, 19.6.0, 19.5.0, 19.4.0, 19.2.0, 19.1.0, 19.0.1, 19.0.0, 18.10.0, 18.9.1, 18.9.0, 18.8.0, 18.7.0, 18.6.0, 18.5.0, 18.4.0, 18.3.0, 18.2.0, 18.1.0, 18.0.0, 16.14.2, 16.14.0, 16.13.2, 16.13.1, 16.13.0, 16.12.0, 16.10.0, 16.9.1, 16.9.0, 16.8.0, 16.7.0, 16.6.2, 16.6.1, 16.6.0, 16.5.0, 16.4.2, 16.4.1, 16.4.0, 16.3.0, 16.2.0, 16.1.0, 16.0.0, 15.14.0, 15.13.0, 15.12.0, 15.11.0, 15.10.0, 15.9.0, 15.8.0, 15.7.0, 15.6.0, 15.5.1, 15.5.0, 15.4.0, 15.3.0, 15.2.1, 15.2.0, 15.1.0, 15.0.1, 14.9.0, 14.8.0, 14.7.0, 14.6.0, 14.5.0, 14.4.0)
# nodejs-slim_latest  (20.0.0, 19.9.0, 19.8.1, 19.8.0, 19.7.0, 19.6.1, 19.6.0, 19.5.0, 19.4.0, 19.2.0, 19.1.0, 19.0.1, 19.0.0, 18.10.0, 18.9.1, 18.9.0, 18.8.0, 18.7.0, 18.6.0, 18.5.0, 18.4.0, 18.3.0, 18.2.0, 18.1.0, 18.0.0, 16.14.2, 16.14.0, 16.13.2, 16.13.1, 16.13.0, 16.12.0, 16.10.0, 16.9.1, 16.9.0, 16.8.0, 16.7.0, 16.6.2, 16.6.1, 16.6.0, 16.5.0, 16.4.2, 16.4.1, 16.4.0, 16.3.0, 16.2.0, 16.1.0, 16.0.0, 15.14.0, 15.13.0, 15.12.0, 15.11.0, 15.10.0, 15.9.0, 15.8.0, 15.7.0, 15.6.0, 15.5.1, 15.5.0, 15.4.0, 15.3.0, 15.2.1, 15.2.0, 15.1.0, 15.0.1, 14.9.0, 14.8.0, 14.7.0, 14.6.0, 14.5.0, 14.4.0)
# nodejs-16_x-openssl_1_1  (16.19.0, 16.18.1, 16.18.0, 16.17.1, 16.17.0)

# for your whole system
devbox global add nodejs@19.9.0
devbox global install

# for your specific project/folder
devbox init
devbox add nodejs@19.9.0
devbox shell # mimicing nix-shell

To get the nixpkg hash of the package (so it can be installed without devbox, or used in a shell.nix) you'll have to open up your devbox.lock

• If global installing, check cat $XDG_DATA_HOME/devbox/global/default/devbox.lock and/or cat ~/.local/share/devbox/global/default/devbox.lock
• if local then it will just be in the local ./devbox.lock

Caveats / Notes:

1. "Search api lacks x86-darwin info" is in a comment and I imagine there are many other such caveats because of this issue (packages don't have a consistent way of listing their versions within nixpkgs)
2. Says "experimental" on the Devbox site (at least at the moment, July 24th 2023)
3. The LD_LIBRARY_PATH problem mentioned a few comments above is still going to be present

Thanks again for the update @dudicoco. I'll have to spend more time reading their source code to figure out how they're doing it (it is still pretty tough for me to imagine they perfectly/ideally solved version search). But this is definitely the best most-comprehensive version search for nixpkgs I've seen, so they've got to be doing something right!

[jeff-hykin]

Update: The Version-Search Results are now Updated Daily! (In sync with nixpkgs unstable)

From the guys at devbox: https://www.nixhub.io/ (Which is also powers their CLI search in the comment above). As of today; as soon as a version is available in nixpkgs unstable; within 24 hours it'll show up on those search tools.

Note: There are still search caveats, like Mac M1, but this is still a huge amount of progress on this issue and all the credit goes to the devbox team. They also have a great article here talking about how they canoncialize names and versions since Nixpkgs still doesn't try to.

I met with 3 devbox engineers last month and we've been working together to 1. spearhead getting all the missing versions cataloged and 2. get all the code to be open source. Some nice incremental achievement is finished a script that can fully exhaustively brute force search all attributes (which AFAIK not even hydra or nix-env is capable of). The script/approach is too slow for daily updates, but it will help us figure out why nix-env misses packages. If we can figure that out and fix it, then we will have an entirely clear path for indexing every version that's ever existed on nixpkgs in a efficient way, even super old nicpkgs. From there it'll be a matter of building the final bits of infrastructure, open sourcing it, and getting it to be part of the normal nix-system/nix-search.

Update (Nov 2023): The devbox guys still haven't open sourced the indexing tool which is sad, but I haven't pinged them about it.

LD_PATH is still a problem, but honestly that problem belongs in a different issue.

[SuperSandro2000]

We now all know about devbox but could we not turn this thread into an ad for it? That would be nice.

[jeff-hykin]

@SuperSandro2000 I agree and I see why it comes off that way.

There were no complaints when I posted a tutorial for Lazamar's website, probably because their website is designed to work with nix, and its only job is version search. (Clearly Lazamar had no ulterior motive)

I do not have a single project that uses devbox; I have never indented to advertise anything other than a how to find/install specific versions of nix packages. The devbox tutorial had a lot of detail because (unfortunately) Devbox is a bulky framework, and devbox's version search isn't designed to work with nix.
Getting a nixpkgs commit hash out of Devbox is an extreme pain compared to Lazamar's site. It took me quite a while to find how to get the nixpkgs hash, so I wanted to make a tutorial for the people who cared about that.

I agree that^ difference is worth clarifying so I added a disclaimer at the top, and tried to make the nix-without-devbox usage more prominent. Let me know if you think it can be better @SuperSandro2000

I'll also do another answer summary to show all the options.

[jeff-hykin]

TLDR; How to find and install a specific version

1. Search for a version here https://history.nix-packages.com/ (the successor to Lazamar's website; RikudoeSage's site!) and start at Step 1 of this tutorial
2. If the version isn't on there, then try finding it using devbox tutorial here.

[RikudouSage]

I'd just add that the instructions are there for each version if you click into the version's detail.

[bryango]

I think I have found a "native" solution to the problem of installing old versions (at least for my own use case). In summary, to install an old package with nix,

• locate a cached old version of the package on hydra: https://hydra.nixos.org
• install the cached closure via nix profile install
  or builtins.fetchClosure

A detailed write-up is here: https://github.com/bryango/cheznix/blob/master/doc/cached-versions.md