Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: new meta attr for source code repository #293838

Open
3 tasks
lolbinarycat opened this issue Mar 6, 2024 · 73 comments · Fixed by #294347 · May be fixed by #330756
Open
3 tasks

Feature request: new meta attr for source code repository #293838

lolbinarycat opened this issue Mar 6, 2024 · 73 comments · Fixed by #294347 · May be fixed by #330756

Comments

@lolbinarycat
Copy link
Contributor

Problem

most packages use meta.homepage to link to the source code repository, but for projects with an actual homepage (such as luajit), finding the source code often requires clicking around various links to try to find the official repo.

for packages that use fetchFromGitHub or similar, you can usually piece the url back together with some effort, but for projects that use a source tarball (again, like luajit), you're out of luck.

Proposal

a meta.repository field that points to an http-browsable source tree.

for packages without a separate homepage, you could just set the meta.repository field instead.

Checklist


Add a 👍 reaction to issues you find important.

@SuperSandro2000
Copy link
Member

There is meta.downloadPage which is documented https://github.com/NixOS/nixpkgs/blob/5bfab70cdf63ad75f1d7d0facc59fb9f49668811/doc/stdenv/meta.chapter.md#downloadpage-var-meta-downloadpage

@lolbinarycat
Copy link
Contributor Author

a download page and browsable source tree are entirely different things. a package that has a dedicated homepage is also very likely to have a dedicated page for downloading binaries.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 8, 2024

most packages use meta.homepage to link to the source code repository, but for projects with an actual homepage (such as luajit), finding the source code often requires clicking around various links to try to find the official repo.

So, what?
This is a decision from the upstream maintainers to not provide an easy link to their CVS repos.
Ask them for an easier-to-find link.

Indeed there are many times we find the opposite: the software has only the repository as its homepage.
E.g. I keep bqn-mode in github and never set a dedicated homepage for it.

@lolbinarycat
Copy link
Contributor Author

This is a decision from the upstream maintainers to not provide an easy link to their CVS repos.
Ask them for an easier-to-find link.

same logic can be applied to say that meta.downloadPage and meta.changelog shouldn't exist.

project homepages are usually designed for users, not developers, in my experience they rarely put the git repo front and center.

Indeed there are many times we find the opposite: the software has only the repository as its homepage.
E.g. I keep bqn-mode in github and never set a dedicated homepage for it.

if you mean packages that don't have a dedicated homepage, and use the git forge instead: yes, i am well aware of that. i mention it in the original post.

if you mean packages that have a dedicated homepage, and use this git forge despite this: as far as i can tell this is incorrect metadata. the homepage should point to the homepage, doing otherwise is just confusing.

@AndersonTorres
Copy link
Member

same logic can be applied to say that meta.downloadPage and meta.changelog shouldn't exist.

meta.downloadPage is useful when the download page is wildly different from the homepage.
That's useful for automated scripts.

meta.changelog has useful info for package maintainers.

A pointer for an eye-candy webpage showing the files unpacked in the CVS? This is just eyecandy.

And the CVS itself is useful for those who want to contribute or hack with the project themselves.
For a package maintainer, this is just downloadPage, albeit it is not an adequate name.

project homepages are usually designed for users, not developers

Yep.
Convince them we programmers are more important than those mere users...

@Artturin
Copy link
Member

meta.repositories used to exist but was removed in 33cce15

@Aleksanaa
Copy link
Member

I've mentioned the same problem here: https://discourse.nixos.org/t/problems-regarding-meta-homepage-and-link-to-repository/39821. So thanks for raising an issue.

@Aleksanaa
Copy link
Member

Aleksanaa commented Mar 15, 2024

meta.downloadPage is useful when the download page is wildly different from the homepage. That's useful for automated scripts.

The real download link is always obvious: it's in the src field. This excludes a small number of packages with a download link hidden behind the paywall. But in this case, downloadPage is still useless for automated scripts or package maintainers. In fact, not a large proportion of software packages really use downloadPage.

We often do this instead:

nix-repl> :l <nixpkgs>

nix-repl> pkgs.mailspring.src.url 
"https://github.com/Foundry376/Mailspring/releases/download/1.13.3/mailspring-1.13.3-amd64.deb"

nix-repl> pkgs.go-musicfox.src.url 
"https://github.com/go-musicfox/go-musicfox/archive/v4.3.1.tar.gz"

nix-repl> pkgs.transmission.src.url
"https://github.com/transmission/transmission.git"

I believe the automated scripts do the same, because in this way they even don't have to parse the Nix code themselves. For package maintainers, they can read the code, and figuring out how to bump source versions shouldn't be a problem.

A pointer for an eye-candy webpage showing the files unpacked in the CVS? This is just eyecandy.

And the CVS itself is useful for those who want to contribute or hack with the project themselves.
For a package maintainer, this is just downloadPage, albeit it is not an adequate name.

In fact, link to a repository is often much more useful than the downloadPage. We can roughly observe whether this project is in a good development state (I will not define this term here, it is related to the user's personal needs). On third-party code hosting platforms, we can often find the issue area in a fixed location (rather than a random place), so that we can have a general understanding of the quality of the program, submit issues, or find workarounds to known problems.

This also applies to package maintainers (including the maintainer who created this package and some random people who wants to bump the package), since some dependencies are often hidden in the cmake or meson build files, some files may need to be installed manually, and some code needs to be patched to adapt to NixOS. However, unlike the downloadPage, sometimes we are not using the src related to the repository (such as mirror, upstream deb package, or simply download link not including the repo)

meta.changelog has useful info for package maintainers.

For maintainers, code modifications between version tags are far more important than the changelog, because they really directly reflect changes in dependencies and other metadata.

Convince them we programmers are more important than those mere users...

Users and developers are often not separate in the Linux community, especially in a distribution that requires writing code in an unfamiliar language to configure the system. It makes no sense to discuss the needs of the two in isolation.

Indeed there are many times we find the opposite: the software has only the repository as its homepage.

Yes, this begs another question: What exactly is a homepage? In my opinion, a repository is a repository, while a special introduction page is a homepage. Although I did see many repositories with very detailed READMEs, and they may indeed be used as homepages, the repository itself still brings too much distraction. I would prefer that in tools like search.nixos.org, when users click on a link named "homepage" or "repository", they have clear expectations of where they will be directed. This is also a UX issue, but there may be my personal preference, and other people may have different views.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 15, 2024

The real download link is always obvious: it's in the src field.

This is a bit more complicated. The src field does not necessarily tells us what is the newest release.
There are some wackos that use the same link and didn't even provide a versioned tarball.

The best example: RIES!

http://www.mrob.com/pub/ries/src/ries.c.txt

I believe the automated scripts do the same, because in this way they even don't have to parse the Nix code themselves.

Many updater scripts are generated by the derivation itself. It's not the shell script that parses Nix, it's Nix code that generates the script.

On third-party code hosting platforms, we can often find the issue area in a fixed location (rather than a random place), so that we can have a general understanding of the quality of the program, submit issues, or find workarounds to known problems.

Again, eye-candy for people interested in hacking/contributing to the code.

meta.changelog has useful info for package maintainers.

For maintainers, code modifications between version tags are far more important than the changelog, because they really directly reflect changes in dependencies and other metadata.

Are you seriously saying that a whole diff -Naur between two releases of Linux kernel, with nothing else, is way more important than the changelog the kernel developers wrote?

Worse, are you suggesting programmers (including but not limited to Linux kernel programmers) are so unorganized that the changelogs they wrote have zero relevance or negative reliability when compared to a machine-generated diff -Naur?

Users and developers are often not separate in the Linux community

  1. Often not is not the same as never.
  2. Usually programmers are smarter than users, and smart enough to figure themselves by being very keen to look for some keywords such as “star”, “contribute”, “submit an issue”, or “development”, or find the GitHub icon in the sidebar.
    1. This is especially true about the users of a distribution that requires writing code in an unfamiliar language to configure the system (Bash?).
  3. It does not change the argument: the upstream webmasters should provide an easier-to-find link for their VCSes to begin with, not the package managers.

@Aleksanaa
Copy link
Member

Aleksanaa commented Mar 15, 2024

On third-party code hosting platforms, we can often find the issue area in a fixed location (rather than a random place), so that we can have a general understanding of the quality of the program, submit issues, or find workarounds to known problems.

Again, eye-candy for people interested in hacking/contributing to the code.

I don't see what I'm saying having anything to do with hacking/contributing to the code; on the contrary, it's what you would call "normal users" doing on a daily basis.

For maintainers, code modifications between version tags are far more important than the changelog, because they really directly reflect changes in dependencies and other metadata.

Are you seriously saying that a whole diff -Naur between two releases of Linux kernel, with nothing else, is way more important than the changelog the kernel developers wrote?

No; for some software, the author may have a very well-written changelog, including changes in the building method; but for other projects, the diff of the build script and the addition and deletion of some special data files have provided sufficient information for package maintainers. Changes to specific functional code often do not result in package changes such as adding or removing dependencies or modifying the build process, unless a specific error is thrown in the build process.

Worse, are you suggesting programmers (including but not limited to Linux kernel programmers) are so unorganized that the changelogs they wrote have zero relevance or negative reliability when compared to a machine-generated diff -Naur?

To some extent, yes. I've seen nearly all developers who write in great detail about functional changes for their users, but there aren't that many developers who inform the build process or dependency changes for the package maintainers.

Users and developers are often not separate in the Linux community

  1. Often not is not the same as never.

I'm not trying to please everyone. There are certainly an amount of users who will not benefit from it. But this doesn't render it useless.

  1. Usually programmers are smarter than users, and smart enough to figure themselves by being very keen to look for some keywords such as “star”, “contribute”, “submit an issue”, or “development”, or find the GitHub icon in the sidebar.

But that doesn't change the unpleasant process. If we can save them time and energy, then why not?

  1. It does not change the argument: the upstream webmasters should provide an easier-to-find link for their VCSes to begin with, not the package managers.

Yes, logically they should. But what actually happens? We don't always create tools to solve a modeled problem, we also need to solve these problems in reality, especially if we happen to have the ability to do so (I mean, as shown in the PR above, some common fetchers already contain this information).

@AndersonTorres
Copy link
Member

Let me be the devil's advocate here.

What is your goal here, by providing meta.repository? It is just an eye-candy view of the VCS? Or is the repository itself?

Will you prefer a webpage ready to be opened in a web browser, like this GitWeb interface for GCC or this mirror of GNU Emacs?

Or the URL for the repo ready to be consumed by Git, like this from GCC - git://gcc.gnu.org/git/gcc.git - or this from Emacs - git.savannah.gnu.org/git/emacs.git?

meta.repository only makes sense if it can be used as input for a git command (or the corresponding for other VCSes).

@lolbinarycat
Copy link
Contributor Author

@Artturin thanks, i did not know that. i would hope that automatically setting the field from src metadata would make it more useful. additionally, having a single value should make it easier for sites like search.nixos.org to use and present it.

@lolbinarycat
Copy link
Contributor Author

A pointer for an eye-candy webpage showing the files unpacked in the CVS? This is just eyecandy.

  1. issue trackers exist outside of the source tree. they are not viewable via a git clone, but nearly every git forge links them from the repo page.
  2. cloning the entirety of a large project when all you want is a quick glance at a single file is a waste of time, bandwidth, and disk space.

Convince them we programmers are more important than those mere users...

making drastic changes to the culture of every open source project is a lot harder than adding a few lines of code.

@lolbinarycat
Copy link
Contributor Author

Yes, this begs another question: What exactly is a homepage? In my opinion, a repository is a repository, while a special introduction page is a homepage. Although I did see many repositories with very detailed READMEs, and they may indeed be used as homepages, the repository itself still brings too much distraction. I would prefer that in tools like search.nixos.org, when users click on a link named "homepage" or "repository", they have clear expectations of where they will be directed. This is also a UX issue, but there may be my personal preference, and other people may have different views.

i already brought up a possible solution to this in issue i submitted to nixos search: if meta.homepage and meta.repository are set to the same value, only show the repository link.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 19, 2024

A pointer for an eye-candy webpage showing the files unpacked in the CVS? This is just eyecandy.

  1. issue trackers exist outside of the source tree. they are not viewable via a git clone, but nearly every git forge links them from the repo page.

Therefore you do not want the source code repository, you want a very specific eyecandy. You are naming it in a misleading way - and arguing for it in a misleading way too.

The "issue tracker" of Linux kernel is a freaking mailing list. Indeed many projects still use them - hell, SourceHut was created a decade ago and they use mailing lists!

Is a mailing list an acceptable value for the meta.eyecandySite you call "repository"? (Don't worry, many of them have HTML-rendered, HTTP-reachable backups...)

  1. cloning the entirety of a large project when all you want is a quick glance at a single file is a waste of time, bandwidth, and disk space.

fetchgit does this and no one complained.

Also, git allows capturing a single file since at least 1.8.

making drastic changes to the culture of every open source project is a lot harder than adding a few lines of code.

Like making all of them using GitHub exclusively?

@lolbinarycat
Copy link
Contributor Author

Therefore you do not want the source code repository, you want a very specific eyecandy. You are naming it in a misleading way - and arguing for it in a misleading way too.

issue trackers exist outside of the source tree, therefore i don't want the source code repository?
i know that's being overly literal, but i don't know what you're actually trying to say.

the issue tracker being available via the repository webpage is not a an official feature, it is simply a nice situational benefit for users.

The "issue tracker" of Linux kernel is a freaking mailing list. Indeed many projects still use them - hell, SourceHut was created a decade ago and they use mailing lists!

ok, so the repository page of those projects simply won't link to the issue tracker. i don't think that's a huge deal.

Is a mailing list an acceptable value for the meta.eyecandySite you call "repository"? (Don't worry, many of them have HTML-rendered, HTTP-reachable backups...)

no, only an http-browsable source tree is an acceptable value, as the documentation states

fetchgit does this and no one complained.

  1. fetchgit puts its files in a garbage-collected, deduplicated, cached file store.
  2. people do complain about nix build speeds being slow. it's the main downside of nix.

Also, git allows capturing a single file since at least 1.8.

additional mental load of:

  1. remembering/looking up the command to do that
  2. reconstructing the git url from the fetcher, or entering a nix repl to evaluate src.gitRepoUrl
  3. deciding which files to download
  4. deciding where to put those files
  5. remembering to remove those files when you're done

as opposed to just typing !nixpkgs PACKAGE_NAME and clicking a few links.

Like making all of them using GitHub exclusively?

what?

this proposal works equally well with sites like gitea, gitlab, and SourceHut. it can even work with custom solutions like git.kernel.org.

95% of software projects have some form of http-browsable source tree, and those that don't can simply not set the field and be no worse off than if the field didn't exist.

@rhendric
Copy link
Member

project homepages are usually designed for users, not developers, in my experience they rarely put the git repo front and center.

Then maybe instead of ‘http-browsable source tree’ being the defining characteristic of this new field, we could say that it's for homepages targeting contributors, and meta.homepage is for homepages targeting users? Then there's less confusion over when to use this (for humans who want to interact with contributor-centric resources for the package) versus when to use src (for automated processes to get the current source) or src.gitRepoUrl (for getting an entire Git repository) or meta.downloadPage (for manually downloading the current source or binaries).

@lolbinarycat
Copy link
Contributor Author

Then maybe instead of ‘http-browsable source tree’ being the defining characteristic of this new field, we could say that it's for homepages targeting contributors, and meta.homepage is for homepages targeting users?

personally i think "http-browsable source tree" is much more descriptive than "homepage targeting contributors", as the latter requires making a subjective judgement on what the intended target audience of the page is (and most pages usually target both)

maybe "a webpage where the package's source code can be viewed" would be easier to understand.

@rhendric
Copy link
Member

(and most pages usually target both)

I don't know, if your archetypal example is get-project.com for meta.homepage and github.com/project-team/project for meta.repository, I think it's pretty clear that one page is user-facing and one page is contributor-facing.

And if a project only exposes a Bugzilla instance but not an HTTP-browsable VCS, I'd still consider that to be a contributor-facing home.

@lolbinarycat
Copy link
Contributor Author

I don't know, if your archetypal example is get-project.com for meta.homepage and github.com/project-team/project for meta.repository, I think it's pretty clear that one page is user-facing and one page is contributor-facing.

And if a project only exposes a Bugzilla instance but not an HTTP-browsable VCS, I'd still consider that to be a contributor-facing home

i think in that case (if it was frequent enough), we would want a separate issueTracker or bugReport field, instead of overloading the meaning of meta.repository.

my goal with this is to make metadata easier to understand, as currently homepage is somewhat overloaded, and often has an unexpected value (eg. the homepage for gforth is set to its git repo, instead of gforth.org)

when you click the url labeled "homepage", you should go to the project's homepage, and when you click the url labeled "repository" you should go to the project's repository.

@rhendric
Copy link
Member

Then I guess I don't know why you want this field either, given that src.gitRepoUrl will take you to the project's repository.

@lolbinarycat
Copy link
Contributor Author

src.gitRepoUrl is specific to git, undocumented, only set by certain fetchers, and not designed to be overridden by package maintainers.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 20, 2024

Therefore you do not want the source code repository, you want a very specific eyecandy. You are naming it in a misleading way - and arguing for it in a misleading way too.

issue trackers exist outside of the source tree, therefore i don't want the source code repository? i know that's being overly literal, but i don't know what you're actually trying to say.

Your intention is to allow only eye-candy forges or forge-like sites, and call them a generic and misleading name "repository"

ok, so the repository page of those projects simply won't link to the issue tracker. i don't think that's a huge deal.

Repositories that are not meta.repositories.

I have not seen such an ironic stance since the C keyword const not meaning constant.

no, only an http-browsable source tree is an acceptable value, as the documentation states

Why such a discrimination (against ugly non-eyecanded repos), I ask again?

1. fetchgit puts its files in a garbage-collected, deduplicated, cached file store.

Was this an argument for or against using fetchgit?

2. people do complain about nix build speeds being slow.  it's the main downside of nix.

In the worst of cases, you can grab the raw git command.

additional mental load of:

1. remembering/looking up the command to do that

2. reconstructing the git url from the fetcher, or entering a nix repl to evaluate `src.gitRepoUrl`

3. deciding which files to download

4. deciding where to put those files

5. remembering to remove those files when you're done

There is a thing called shell script. It can automate a tuckload of boring and forgettable tasks.

Like making all of them using GitHub exclusively?

what?

this proposal works equally well with sites like gitea, gitlab, and SourceHut. it can even work with custom solutions like git.kernel.org.

You are not targetting general-purpose repositories, but a very specific style of site.
In other words, you are targetting forges.

95% of software projects have some form of http-browsable source tree, and those that don't can simply not set the field and be no worse off than if the field didn't exist.

  1. Then your arguments about being hard to find those source trees are weaker than mine about multi-valued backup mirrors.
  2. Arguments that rely on percentages and minorities are very funny sometimes.

and when you click the url labeled "repository" you should go to the project's repository.

"But only if its project's repository is an HTTP-browsable eyecandied site" - FTFY.

@lolbinarycat
Copy link
Contributor Author

Your intention is to allow only eye-candy forges or forge-like sites, and call them a generic and misleading name "repository"

don't tell me what my intention is.

git.kernel.org would certainly qualify for this value, and that doesn't have any of the features you would associate with a typical forge

You are not targetting general-purpose repositories, but a very specific style of site.
In other words, you are targetting forges.

once again: stop telling me what i am trying to do.

"But only if its project's repository is an HTTP-browsable eyecandied site" - FTFY.

i don't care about whether a webpage looks good or not. you simply decided that i did and keep asserting that i do every two sentences.

it's 2024, and the world runs on http. i'm not really happy about it, but i'm not going to decrease the utility of my metadata fields out of spite.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 20, 2024

Your intention is to allow only eye-candy forges or forge-like sites, and call them a generic and misleading name "repository"

don't tell me what my intention is.

I don't need - you did it already:

issue trackers exist outside of the source tree. they are not viewable via a git clone, but nearly every git forge links them from the repo page.

Further, the whole "search engine runs on HTTP, then it can't include non-HTTP links" argument plus the examples about multiple purposes of forges meta.repositories...

git.kernel.org would certainly qualify for this value, and that doesn't have any of the features you would associate with a typical forge

Then nothing hinders a Gemini link.

once again: stop telling me what i am trying to do.

Are you targeting general-purpose repositories?

i don't care about whether a webpage looks good or not. you simply decided that i did and keep asserting that i do every two sentences.

Then a Gemini link is acceptable, correct?

it's 2024, and the world runs on http. i'm not really happy about it, but i'm not going to decrease the utility of my metadata fields out of spite.

Allowing Gemini or IPFS links decrease the utility of your metadata fields? How?

@ghost
Copy link

ghost commented Apr 4, 2024

  `repository` would be the first to pull directly from a complex attribute not already part of `meta`, which is a threshold worth pausing at, because of precisely the sort of problem we've just witnessed.

@rhendric what is the problem we just witnessed? we made a mistake fixed it and have a better implementation because of it that is immune to the previous failure. requiring a perfect first implementation is not a good faith argument.

@lolbinarycat
Copy link
Contributor Author

More like no longer needs to be tested via CI, because for most packages there'll be nothing to test.

most packages. the entire point of testing is to catch edge cases.

As soon as ‘everyone’ is more than one nice-to-have link in the search app, I'll entertain this objection.

well guess what, i have immediate plans for other stuff i would like to do with it so... yeah it's more that nixos-search

As already noted, addressing eval errors outside of Nix is easier than inside, and consumers can always filter the packages down to a set they're interested in.

as already noted, doing so results in massive performance penalties.

You're proposing changing what src attributes are required to evaluate under what conditions just so that you can put logic intended to populate a search result field in Nixpkgs instead of in the search app—this is clearly a case of the tail wagging the dog.

this is simply misrepresenting my intent. i already said there are other usecases. i would really appreciate if people would stop telling me what i want.

@rhendric
Copy link
Member

rhendric commented Apr 4, 2024

@a-n-n-a-l-e-e, I'm not requiring a perfect first implementation.

The issue is that the scope increased from ‘add a field to meta’ to ‘add a field to meta and also create a new guarantee for src to evaluate successfully in more contexts’, and that second bit is harder and more error-prone. If we agree on the value of undertaking both parts, then making a mistake in the first attempt of the second part isn't a big deal—fix it and move on, as you say. But the mistake is evidence that the second part is harder, and that even if it looks like we get it right now, problems may arise again in the future (and that comment in the PR is evidence that we're expecting those problems—and that resolution mechanism (find the comment based on stack trace breadcrumbs) is not great!). And given that the second part is harder, I assert that it isn't worth it to attempt and we should take the scope of the work back to just the first part.

@lolbinarycat

most packages. the entire point of testing is to catch edge cases.

But you don't have to test what you don't implement! I'm really confused by our disagreement here. You want to change src and then test that the change is good; I want to refrain from changing src and then there's nothing to do or test.

well guess what, i have immediate plans for other stuff i would like to do with it so... yeah it's more that nixos-search

How about discussing those plans then? That might convince me that implementing the feature the way you want is worth doing even if it involves changing the guarantees around src.

as already noted, doing so results in massive performance penalties.

And that's much less of a big deal if it doesn't happen in the heart of Nixpkgs for everyone's evaluation.

this is simply misrepresenting my intent. i already said there are other usecases. i would really appreciate if people would stop telling me what i want.

Sorry you're frustrated, but I'm going off of what you've said in this thread so far, and I think the above comment is the first time you're mentioning any other use cases in a non-hypothetical way.

@lolbinarycat
Copy link
Contributor Author

But you don't have to test what you don't implement! I'm really confused by our disagreement here. You want to change src and then test that the change is good; I want to refrain from changing src and then there's nothing to do or test.

i thought you wanted to not implement it in nixpkgs.

guess what, if you move the code somewhere else, it still exists!

@lolbinarycat
Copy link
Contributor Author

github sucks on mobile

@rhendric
Copy link
Member

rhendric commented Apr 4, 2024

i thought you wanted to not implement it in nixpkgs.

guess what, if you move the code somewhere else, it still exists!

This isn't the gotcha that you think it is, so the attempt at a dunk isn't exactly going to be convincing.

What you would be testing if this logic were in Nixpkgs is not the correctness of the logic itself. The logic is pretty dang simple—use meta.repository if it exists, and use src.meta.repository if it doesn't, and adjust for lists appropriately. That's a function that could be unit tested very easily, but it's not what we're discussing, I think. It seems to me that when you say you want the correctness tested in Nixpkgs, what you want to ensure is that every derivation in Nixpkgs has a src that can be accessed in this way, across all platforms and under any other weird conditions. That's hard to test and hard to guarantee going forward.

I'm proposing not making that change to the expectations around src. Implement the trivial function elsewhere. If you plan on using it in multiple places, stick it in a library. Maybe it even deserves to live in lib. So yes, that code still exists! And you can test it if you want! What won't be tested, because it won't change, is that src can be accessed in all the contexts that you might want meta. If consumers try to access src in those contexts, they can fail and recover from the failure in whatever way is appropriate for them, because that's not Nixpkgs's concern.

@AndersonTorres
Copy link
Member

AndersonTorres commented Apr 4, 2024

Did you born yesterday?

@AndersonTorres, this is rude. OP may be misguided but that isn't license to take this tone.

Not my intention, just an expression of surprise.

For someone throwing arguments like "a lot of this was made before" without backing them up, it is hilarious that the most obvious example is now requiring evidence!

@AndersonTorres
Copy link
Member

AndersonTorres commented Apr 4, 2024

requiring a perfect first implementation is not a good faith argument

the first one broke - twice, if you can't remember, Ghostie...

@lolbinarycat
Copy link
Contributor Author

That's hard to test

it's really not, it's basically just nesting two for-each loops.

in my experience it isn't difficult to make src eval either.

hard to guarantee going forward.

  1. nixpkgs doesn't guarantee stability outside of stable releases
  2. i've only ever seen one pattern that breaks this expectation: indexing src based on the value of the current system

@rhendric
Copy link
Member

rhendric commented Apr 4, 2024

That's hard to test

it's really not, it's basically just nesting two for-each loops.

That'll test that it works in the specific configuration that it's tested in. But a given src could depend on any configuration value, not just system, and you aren't going to test every possible permutation in CI.

hard to guarantee going forward.

  1. nixpkgs doesn't guarantee stability outside of stable releases
  2. i've only ever seen one pattern that breaks this expectation: indexing src based on the value of the current system

Yeah, so this really comes down to a risk appetite thing. If you're a move-fast-break-things person, you might see the risk as worth it. You're looking at the things you see and saying they don't look so bad. I'm looking at the spaces where there can be things I don't see and saying why risk it? I don't think there have been enough eyeballs on this for me to feel comfortable with work that has a good chance of slipping time bombs into Hydra, which take precious time and effort to resolve. Particularly when consumers who want this logic can shoulder the burden of evaluating it themselves. Why not start there and propose moving things into Nixpkgs after you've proved the value of the work?

@ghost
Copy link

ghost commented Apr 4, 2024

@a-n-n-a-l-e-e, I'm not requiring a perfect first implementation.

The issue is that the scope increased from ‘add a field to meta’ to ‘add a field to meta and also create a new guarantee for src to evaluate successfully in more contexts’, and that second bit is harder and more error-prone.

so we are not trying to guarantee that src evaluates correctly. the new implementation doesn't require src to evaluate meta attributes.

@lolbinarycat
Copy link
Contributor Author

That'll test that it works in the specific configuration that it's tested in. But a given src could depend on any configuration value, not just system, and you aren't going to test every possible permutation in CI.

theoretically, but i don't think they're likely to in practice.

anyways, by that argument, some combination of config options could cause nixpkgs as a whole to stop evaluating, since we're not testing every permutation.

@rhendric
Copy link
Member

rhendric commented Apr 4, 2024

@a-n-n-a-l-e-e

so we are not trying to guarantee that src evaluates correctly. the new implementation doesn't require src to evaluate meta attributes.

The new implementation being #300313? Which contains this code?

      # NOTE: this will fail if src fails to eval, in that case either set meta.repository manually to prevent this default from being evaluated, or just make sure src doesn't fail to eval.
      repository =
        if attrs ? src.meta.homepage
        then attrs.src.meta.homepage
        else if attrs ? srcs && isList attrs.srcs
        then unlist (map (src: src.meta.homepage) (filter (src: src ? meta.homepage) attrs.srcs))
        else [];

That's the code that doesn't require src? The code that fails ‘if src fails to eval’? Can you clarify?

@lolbinarycat

anyways, by that argument, some combination of config options could cause nixpkgs as a whole to stop evaluating, since we're not testing every permutation.

Yes, some combination of config options could do that. I don't know what point you're trying to make with that statement.

@ghost
Copy link

ghost commented Apr 4, 2024

@a-n-n-a-l-e-e

so we are not trying to guarantee that src evaluates correctly. the new implementation doesn't require src to evaluate meta attributes.

The new implementation being #300313? Which contains this code?

      # NOTE: this will fail if src fails to eval, in that case either set meta.repository manually to prevent this default from being evaluated, or just make sure src doesn't fail to eval.
      repository =
        if attrs ? src.meta.homepage
        then attrs.src.meta.homepage
        else if attrs ? srcs && isList attrs.srcs
        then unlist (map (src: src.meta.homepage) (filter (src: src ? meta.homepage) attrs.srcs))
        else [];

That's the code that doesn't require src? The code that fails ‘if src fails to eval’? Can you clarify?

right. src not evaluating will not prevent access to other fields in meta, which was the issue before. trying to evaluate repository will still be problematic but i'm not sure it's worth adding additional checks to repository to prevent errors related to src. it's possible, checking that sourceProvence isSource, but perhaps not necessary. my point is that adding the repository field will not break current usage.

@AndersonTorres
Copy link
Member

AndersonTorres commented Apr 4, 2024

it's possible, checking that sourceProvence isSource

this is another assumption that can be broken easily.

  • sourceProvenance == fromSource does not imply it "evaluates" in all platforms. What about assembly?

  • sourceProvenance == binaryNativeCode does not imply it does not "evaluate" in all platforms. What about object files for Windows that are used in forensics and pentesting?

    Indeed we can argue font glyphs are exactly like that.

(currently I am more inclined to an "outside nix" approach (scripting, manual fixup, treewide, small steps), in case someone asks me.)

@lolbinarycat
Copy link
Contributor Author

What about assembly?

unless you are using import-from-derivation (which is banned in nixpkgs), assembly will never be interpreted at eval time, only at build time.

Indeed we can argue font glyphs are exactly like that.

font glyphs are binary data, not binary code. it could be argued that truetype fonts contain vm bytecode, but they certainly do not contain native code.

@ghost
Copy link

ghost commented Apr 5, 2024

so i wanted to see how many aborts are generated from trying pkg.meta.repository over all packages with the hydra systems x64/aarch64 linux & darwin. so hacked the script below which runs the check meta eval. Running the script i found 19 files that needed to be modified to throw. (probably better way to do this but here is a hacked 1st attempt)

script.sh
#!/usr/bin/env bash
# https://github.com/NixOS/ofborg?tab=readme-ov-file#running-meta-checks-locally
curl -OL https://raw.githubusercontent.com/NixOS/ofborg/released/ofborg/src/outpaths.nix
GC_INITIAL_HEAP_SIZE=4g nix-env -f ./outpaths.nix -qaP --no-name --out-path --arg checkMeta true > out-paths

# transform out-paths to:
# pkgs: with pkgs; [
#   pkg-name.meta.repository or null
#   ...
# ]
# need to remove packages with names that include '+' and .if. and pkgs-lib-tests
cat out-paths  | \
    gawk '{gsub("\\.[^.]*$", ".meta.repository or null", $1); print $1}' | \
    sort -u | \
    sed -e '/haskellPackages.if/d' \
        -e '/pkgs-lib-tests/d' \
        -e '/+/d' | \
    awk 'BEGIN{ print "pkgs: with pkgs; ["}
         { print $0 }
         END{ print "]"}' > out-paths.nix

for sys in x86_64-darwin x86_64-linux aarch64-darwin aarch64-linux; do
    nix eval --impure --expr 'with import ./. { system = '"\"$sys\""'; }; map (t: builtins.tryEval t) ((import ./out-paths.nix) pkgs)'
done
nixpkgs.diff add repository to meta and add throw to 19 files
diff --git a/pkgs/applications/audio/renoise/default.nix b/pkgs/applications/audio/renoise/default.nix
index d3462ecc6ad5..a56d43876fb0 100644
--- a/pkgs/applications/audio/renoise/default.nix
+++ b/pkgs/applications/audio/renoise/default.nix
@@ -39,7 +39,7 @@ in stdenv.mkDerivation rec {
     releasePath
   else
     let
-      platform = platforms.${stdenv.system};
+      platform = platforms.${stdenv.system} or (throw "unsupported system ${stdenv.hostPlatform.system}");
       urlVersion = lib.replaceStrings [ "." ] [ "_" ] version;
     in fetchurl {
       url =
diff --git a/pkgs/applications/gis/udig/default.nix b/pkgs/applications/gis/udig/default.nix
index 8510918d4cab..264445336cd3 100644
--- a/pkgs/applications/gis/udig/default.nix
+++ b/pkgs/applications/gis/udig/default.nix
@@ -13,7 +13,7 @@ let
       hash = "sha256-Ihk3InHB3/tEYRqH2ozhokz2GN8Gfig5DJkO/8P1LJs=";
     };
   };
-  src = srcs.${stdenv.hostPlatform.system};
+  src = srcs.${stdenv.hostPlatform.system} or (throw "unsupported system ${stdenv.hostPlatform.system}");
 
   meta = with lib; {
     description = "User-friendly Desktop Internet GIS";
diff --git a/pkgs/applications/graphics/sane/backends/brscan4/default.nix b/pkgs/applications/graphics/sane/backends/brscan4/default.nix
index ec8efcc27c2a..c15b3033a265 100644
--- a/pkgs/applications/graphics/sane/backends/brscan4/default.nix
+++ b/pkgs/applications/graphics/sane/backends/brscan4/default.nix
@@ -21,7 +21,7 @@ stdenv.mkDerivation rec {
       url = "https://download.brother.com/welcome/dlf006645/${pname}-${version}.amd64.deb";
       sha256 = "sha256-Gpr5456MCNpyam3g2qPo7S3aEZFMaUGR8bu7YmRY8xk=";
     };
-  }."${stdenv.hostPlatform.system}";
+  }."${stdenv.hostPlatform.system}" or (throw "unsupported system ${stdenv.hostPlatform.system}");
 
   unpackPhase = ''
     ar x $src
diff --git a/pkgs/applications/misc/1password-gui/default.nix b/pkgs/applications/misc/1password-gui/default.nix
index 32825d3ba523..5a7e4914f8cd 100644
--- a/pkgs/applications/misc/1password-gui/default.nix
+++ b/pkgs/applications/misc/1password-gui/default.nix
@@ -51,7 +51,7 @@ let
   };
 
   src = fetchurl {
-    inherit (sources.${channel}.${stdenv.system}) url hash;
+    inherit (sources.${channel}.${stdenv.system} or (throw "unsupported system ${stdenv.hostPlatform.system}")) url hash;
   };
 
   meta = with lib; {
diff --git a/pkgs/applications/networking/cluster/hadoop/default.nix b/pkgs/applications/networking/cluster/hadoop/default.nix
index d5bae9ad885b..5f44e39d008c 100644
--- a/pkgs/applications/networking/cluster/hadoop/default.nix
+++ b/pkgs/applications/networking/cluster/hadoop/default.nix
@@ -35,7 +35,7 @@ let
       src = fetchurl {
         url = "mirror://apache/hadoop/common/hadoop-${finalAttrs.version}/hadoop-${finalAttrs.version}"
               + optionalString stdenv.isAarch64 "-aarch64" + ".tar.gz";
-        inherit (platformAttrs.${stdenv.system}) hash;
+        inherit (platformAttrs.${stdenv.system} or (throw "Unsupported system: ${stdenv.system}")) hash;
       };
       doCheck = true;
 
diff --git a/pkgs/applications/networking/instant-messengers/discord/default.nix b/pkgs/applications/networking/instant-messengers/discord/default.nix
index 40e9c77e57a7..423f6cb2f1db 100644
--- a/pkgs/applications/networking/instant-messengers/discord/default.nix
+++ b/pkgs/applications/networking/instant-messengers/discord/default.nix
@@ -52,7 +52,7 @@ let
     };
     aarch64-darwin = x86_64-darwin;
   };
-  src = srcs.${stdenv.hostPlatform.system}.${branch};
+  src = srcs.${stdenv.hostPlatform.system}.${branch} or (throw "${stdenv.hostPlatform.system} not supported on ${branch}");
 
   meta = with lib; {
     description = "All-in-one cross-platform voice and text chat for gamers";
diff --git a/pkgs/applications/science/electronics/picoscope/default.nix b/pkgs/applications/science/electronics/picoscope/default.nix
index c7242117b68c..3aa5cb22aa52 100644
--- a/pkgs/applications/science/electronics/picoscope/default.nix
+++ b/pkgs/applications/science/electronics/picoscope/default.nix
@@ -49,7 +49,7 @@ let
   # If we don't have a platform available, put a dummy version here, so at
   # least evaluation succeeds.
   sources =
-    (lib.importJSON ./sources.json).${stdenv.system} or { picoscope.version = "unknown"; };
+    (lib.importJSON ./sources.json).${stdenv.system} or (throw "unsupported system ${stdenv.system}");
 
   scopePkg = name:
     { url, version, sha256 }:
diff --git a/pkgs/applications/version-management/radicle-upstream/default.nix b/pkgs/applications/version-management/radicle-upstream/default.nix
index 97fec6257dd1..69c960ed78e8 100644
--- a/pkgs/applications/version-management/radicle-upstream/default.nix
+++ b/pkgs/applications/version-management/radicle-upstream/default.nix
@@ -15,7 +15,7 @@ let
       sha256 = "sha256-EuWGbn6qggi8/9Rci8iaXfuVKE+QXb1BHEYDvotR/q4=";
     };
   };
-  src = srcs.${stdenv.hostPlatform.system};
+  src = srcs.${stdenv.hostPlatform.system} or (throw "unsupported system ${stdenv.hostPlatform.system}");
 
   contents = appimageTools.extract { inherit name src; };
 
diff --git a/pkgs/applications/virtualization/firecracker/default.nix b/pkgs/applications/virtualization/firecracker/default.nix
index d9bab2169623..2c3ec6c9a75b 100644
--- a/pkgs/applications/virtualization/firecracker/default.nix
+++ b/pkgs/applications/virtualization/firecracker/default.nix
@@ -13,7 +13,7 @@ let
 
   dlbin = sha256: fetchurl {
     url = "${baseurl}/v${version}/firecracker-v${version}-${suffix}.tgz";
-    sha256 = sha256."${stdenv.hostPlatform.system}";
+    sha256 = sha256."${stdenv.hostPlatform.system}"or (throw "unsupported system ${stdenv.hostPlatform.system}");
   };
 
 in
diff --git a/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix b/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix
index 40681faae8ae..d03419e96570 100644
--- a/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix
+++ b/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix
@@ -80,7 +80,7 @@ let result = stdenv.mkDerivation rec {
     in requireFile {
       name = "jdk-${productVersion}u${patchVersion}-${platformName}.tar.gz";
       url = "http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html";
-      sha256 = sha256.${stdenv.hostPlatform.system};
+      sha256 = sha256.${stdenv.hostPlatform.system} or (throw "unsupported system ${stdenv.hostPlatform.system}");
     };
 
   nativeBuildInputs = [ file makeWrapper ]
diff --git a/pkgs/development/compilers/temurin-bin/jdk-darwin-base.nix b/pkgs/development/compilers/temurin-bin/jdk-darwin-base.nix
index 14871813273d..8f6831a7907c 100644
--- a/pkgs/development/compilers/temurin-bin/jdk-darwin-base.nix
+++ b/pkgs/development/compilers/temurin-bin/jdk-darwin-base.nix
@@ -24,7 +24,7 @@ let
       sourcePerArch.${cpuName}.version or (throw "unsupported CPU ${cpuName}");
 
     src = fetchurl {
-      inherit (sourcePerArch.${cpuName}) url sha256;
+      inherit (sourcePerArch.${cpuName} or (throw "unsupported system ${stdenv.hostPlatform.system}")) url sha256;
     };
 
     # See: https://github.com/NixOS/patchelf/issues/10
diff --git a/pkgs/development/libraries/libcef/default.nix b/pkgs/development/libraries/libcef/default.nix
index d6d098110af8..ce83c3c22cd4 100644
--- a/pkgs/development/libraries/libcef/default.nix
+++ b/pkgs/development/libraries/libcef/default.nix
@@ -73,7 +73,7 @@ let
   platforms."aarch64-linux".sha256 = "16sbfk599h96wcsmpbxlwsvq0n1pssmm8dpwmjsqfrn1464dvs68";
   platforms."x86_64-linux".sha256 = "1wa4nv28saz96kar9svdarfz6c4rnbcqz0rqxzl9zclnhfzhqdiw";
 
-  platformInfo = builtins.getAttr stdenv.hostPlatform.system platforms;
+  platformInfo = platforms.${stdenv.hostPlatform.system} or (throw "unsupported system ${stdenv.hostPlatform.system}");
 in
 stdenv.mkDerivation rec {
   pname = "cef-binary";
diff --git a/pkgs/development/tools/azure-static-sites-client/default.nix b/pkgs/development/tools/azure-static-sites-client/default.nix
index 08e56de7335b..452976b7282a 100644
--- a/pkgs/development/tools/azure-static-sites-client/default.nix
+++ b/pkgs/development/tools/azure-static-sites-client/default.nix
@@ -21,7 +21,7 @@ let
   };
   sources = {
     "x86_64-linux" = fetchBinary "linux-x64";
-    "x86_64-darwin" = fetchBinary "macOS";
+    "x86_64-darwin" = fetchBinary "osx-x64";
   };
 in
 stdenv.mkDerivation {
diff --git a/pkgs/servers/meteor/default.nix b/pkgs/servers/meteor/default.nix
index 0ce2189568cb..7830d309b082 100644
--- a/pkgs/servers/meteor/default.nix
+++ b/pkgs/servers/meteor/default.nix
@@ -20,7 +20,7 @@ in
 stdenv.mkDerivation {
   inherit version;
   pname = "meteor";
-  src = srcs.${system};
+  src = srcs.${system} or (throw "unsupported system ${system}");
 
   #dontStrip = true;
 
diff --git a/pkgs/stdenv/generic/check-meta.nix b/pkgs/stdenv/generic/check-meta.nix
index 1cd1ae6dd72e..f1f486b115a8 100644
--- a/pkgs/stdenv/generic/check-meta.nix
+++ b/pkgs/stdenv/generic/check-meta.nix
@@ -10,7 +10,9 @@ let
     concatMapStrings
     concatMapStringsSep
     concatStrings
+    filter
     findFirst
+    head
     isDerivation
     length
     concatMap
@@ -39,6 +41,11 @@ let
     toPretty
   ;
 
+  unlist = list:
+  if length list == 1
+  then head list
+  else list;
+
   # If we're in hydra, we can dispense with the more verbose error
   # messages and make problems easier to spot.
   inHydra = config.inHydra or false;
@@ -303,6 +310,10 @@ let
       (listOf str)
       str
     ];
+    repository = union [
+      (listOf str)
+      str
+    ];
     downloadPage = str;
     changelog = union [
       (listOf str)
@@ -452,8 +463,14 @@ let
     let
       outputs = attrs.outputs or [ "out" ];
       hasOutput = out: builtins.elem out outputs;
-    in
-    {
+    in {
+      repository =
+        if attrs ? src.meta.homepage
+        then attrs.src.meta.homepage
+        else if attrs ? srcs && isList attrs.srcs
+        then unlist (map (src: src.meta.homepage) (filter (src: src ? meta.homepage) attrs.srcs))
+        else [];
+
       # `name` derivation attribute includes cross-compilation cruft,
       # is under assert, and is sanitized.
       # Let's have a clean always accessible version here.
diff --git a/pkgs/tools/archivers/rar/default.nix b/pkgs/tools/archivers/rar/default.nix
index 3799afd736ef..fabcfde887e0 100644
--- a/pkgs/tools/archivers/rar/default.nix
+++ b/pkgs/tools/archivers/rar/default.nix
@@ -37,7 +37,7 @@ stdenv.mkDerivation {
   pname = "rar";
   inherit version;
 
-  src = fetchurl (srcs.${stdenv.hostPlatform.system});
+  src = fetchurl (srcs.${stdenv.hostPlatform.system} or (throw "unsupported system ${stdenv.hostPlatform.system}"));
 
   dontBuild = true;
 
diff --git a/pkgs/tools/misc/geekbench/5.nix b/pkgs/tools/misc/geekbench/5.nix
index 7784b3632a73..49efe2ea0bf9 100644
--- a/pkgs/tools/misc/geekbench/5.nix
+++ b/pkgs/tools/misc/geekbench/5.nix
@@ -26,7 +26,7 @@ stdenv.mkDerivation {
   inherit version;
   pname = "geekbench";
 
-  src = fetchurl (sources.${stdenv.system});
+  src = fetchurl (sources.${stdenv.system} or (throw "unsupported system ${stdenv.hostPlatform.system}"));
 
   dontConfigure = true;
   dontBuild = true;
diff --git a/pkgs/tools/misc/geekbench/6.nix b/pkgs/tools/misc/geekbench/6.nix
index 4ac5f1d0e49a..373d93dc193c 100644
--- a/pkgs/tools/misc/geekbench/6.nix
+++ b/pkgs/tools/misc/geekbench/6.nix
@@ -27,7 +27,7 @@ stdenv.mkDerivation {
   inherit version;
   pname = "geekbench";
 
-  src = fetchurl (sources.${stdenv.system});
+  src = fetchurl (sources.${stdenv.system} or (throw "unsupported system ${stdenv.hostPlatform.system}"));
 
   dontConfigure = true;
   dontBuild = true;
diff --git a/pkgs/tools/misc/hakuneko/default.nix b/pkgs/tools/misc/hakuneko/default.nix
index 0ad8c5907a70..0359da603ccb 100644
--- a/pkgs/tools/misc/hakuneko/default.nix
+++ b/pkgs/tools/misc/hakuneko/default.nix
@@ -36,7 +36,7 @@ stdenv.mkDerivation rec {
       url = "https://github.com/manga-download/hakuneko/releases/download/v${version}/hakuneko-desktop_${version}_linux_i386.deb";
       sha256 = "32017d26bafffaaf0a83dd6954d3926557014af4022a972371169c56c0e3d98b";
     };
-  }."${stdenv.hostPlatform.system}";
+  }."${stdenv.hostPlatform.system}" or (throw "unsupported system ${stdenv.hostPlatform.system}");
 
   dontBuild = true;
   dontConfigure = true;
diff --git a/pkgs/tools/security/sonar-scanner-cli/default.nix b/pkgs/tools/security/sonar-scanner-cli/default.nix
index 553019299ba7..b97a2bc3a105 100644
--- a/pkgs/tools/security/sonar-scanner-cli/default.nix
+++ b/pkgs/tools/security/sonar-scanner-cli/default.nix
@@ -19,7 +19,7 @@ in stdenv.mkDerivation rec {
   inherit version;
   pname = "sonar-scanner-cli";
 
-  src = fetchurl sonarScannerArchPackage.${stdenv.hostPlatform.system};
+  src = fetchurl sonarScannerArchPackage.${stdenv.hostPlatform.system} or (throw "unsupported system ${stdenv.hostPlatform.system}");
 
   nativeBuildInputs = [ unzip ];
 

[edit] opened PRs to change aborts to throw (and fix a typo)

@AndersonTorres
Copy link
Member

AndersonTorres commented Apr 5, 2024

What about assembly?

unless you are using import-from-derivation (which is banned in nixpkgs), assembly will never be interpreted at eval time, only at build time.

Irrelevant. There is nothing hindering someone to write code like "throw when host is not x86" when facing a fromSource.

font glyphs are binary data, not binary code.

Data-code distinction lose most of its explanatory power since before the first worm, passing through undergrounds like Forth, Lisp and Phrack, and nowadays with the dreadful xzdoor.

Further it misses the point I tried to convey: a non-fromSource can evaluate despite not being fromSource.

Nonetheless my catastrophic example does not affect the argument: neither fromSource implies src will eval, nor non-fromSource implies src will break.

@lolbinarycat
Copy link
Contributor Author

There is nothing hindering someone to write code like "throw when host is not x86"

a throw error would be fine, actually. it would be caught by a tryEval in release-attrpaths-superset

calling abort would not be fine, and would probably cause CI to fail.

the only exceptions i know are calling abort within src on an unsupported platform, or in a transient derivation which is not exposed to hydra.

so... don't do that?

@rhendric
Copy link
Member

rhendric commented Apr 6, 2024

right. src not evaluating will not prevent access to other fields in meta, which was the issue before. trying to evaluate repository will still be problematic but i'm not sure it's worth adding additional checks to repository to prevent errors related to src. it's possible, checking that sourceProvence isSource, but perhaps not necessary. my point is that adding the repository field will not break current usage.

You don't know that. Current usage can include toJSONing the meta attributes of every package in the list. This currently works—I just confirmed it—and would break with your proposed change.

I'm fine with #300286 but I think any PR that makes meta depend on src, even in the way you're currently endorsing, should be reviewed by more of the community before it is merged.

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/when-should-meta-attributes-be-computed/42833/1

@lolbinarycat
Copy link
Contributor Author

Current usage can include toJSONing the meta attributes of every package in the list

if that is a usage we want to support then it should have a CI test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment