Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testers.nixosTest: Deprecate #293891

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

testers.nixosTest: Deprecate #293891

wants to merge 5 commits into from

Conversation

roberth
Copy link
Member

@roberth roberth commented Mar 6, 2024 •
edited
Loading

Description of changes

It's time to migrate.
runNixOSTest is a better alternative, that supports modules properly, as well as running on macOS.
At this point, the existence of nixosTest leads to uncertainty, and its users are missing out on the new features.

🚀

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 8.has: documentation This PR adds or changes documentation label Mar 6, 2024
@roberth roberth mentioned this pull request Mar 6, 2024
Merged
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixostest-vs-runtest/26419/6

@AndersonTorres
Copy link
Member

Why is this not working?

@roberth
Copy link
Member Author

roberth commented Mar 7, 2024

Why is this not working?

nixosTest is not completely broken, but has issues, as described.

Maybe I don't understand the question, but I think I've explained that in the diff, and I've added extra context in the PR description.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 7, 2024 •
edited
Loading

I was talking about the ofborg red status.
Apologies for not being clear.

DanielSidhion
DanielSidhion approved these changes Mar 7, 2024
View reviewed changes
Copy link
Member

@DanielSidhion DanielSidhion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on the docs changes. Idk what's the reasoning behind the ofborg red status though.

@delroth delroth added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Mar 7, 2024
infinisil
infinisil reviewed Mar 7, 2024
View reviewed changes
pkgs/build-support/testers/default.nix Outdated
@@ -118,6 +118,7 @@
* make sense for this evaluation of Nixpkgs.
*/
nixosTesting =
lib.warn "testers.nixosTest is deprecated. Use testers.runNixOSTest instead."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ofborg failure could be fixed by moving this warning into the function body under test:. Non ideal that this is necessary though..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind, looks like that doesn't work either..

Thinking about this again, I believe we need to not expose the attribute at all if config.allowAliases = false (which then also prevents uses in Nixpkgs).

@delroth delroth removed the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Mar 10, 2024
@ShamrockLee ShamrockLee mentioned this pull request Mar 10, 2024
Merged
13 tasks
roberth and others added 3 commits April 22, 2024 21:38
@roberth
testers.nixosTest: Deprecate
43bb838
@roberth @DanielSidhion
doc/testers: Edit deprecation warning for nixosTest
4e69ce4 
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
@roberth
testers.nixosTest: Move warning and add link
4e54377 
This way ofborg doesn't see it and fail.
Unfortunately, it will cause more warnings than it otherwise
would, if multiple tests are evaluated in one go.
@roberth roberth force-pushed the deprecate-nixosTest branch 2 times, most recently from c1dad81 to 73cdcdd Compare April 22, 2024 23:53
@roberth roberth force-pushed the deprecate-nixosTest branch from 73cdcdd to c44c917 Compare April 23, 2024 11:01
@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: testing Tooling for automated testing of packages and modules labels Apr 23, 2024
@ofborg ofborg bot added 8.has: clean-up 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Apr 23, 2024
@roberth roberth requested a review from infinisil April 23, 2024 12:36
@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 4, 2024
@thiagokokada thiagokokada mentioned this pull request Jul 26, 2024
Merged
yfyf added a commit to yfyf/playos that referenced this pull request Sep 20, 2024
@yfyf
Switch to testers.runNixOSTest
bb68b4b 
`nixosTest` is to-be-depercated, see here:
        NixOS/nixpkgs#293891
yfyf added a commit to yfyf/playos that referenced this pull request Sep 24, 2024
@yfyf
Switch to testers.runNixOSTest
5dda177 
`nixosTest` is to-be-depercated, see here:
        NixOS/nixpkgs#293891
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DanielSidhion DanielSidhion DanielSidhion approved these changes

@infinisil infinisil Awaiting requested review from infinisil

Assignees
No one assigned
Labels
2.status: merge conflict This PR has merge conflicts with the target branch 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: testing Tooling for automated testing of packages and modules 8.has: clean-up 8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@roberth @nixos-discourse @AndersonTorres @DanielSidhion @infinisil @delroth @wegank