runNixOSTest fails on Darwin ( tests.testers.runNixOSTest-example fails to build)

[ShamrockLee]

Describe the bug

#282401 should have made NixOS Tests available on Darwin. This should include package tests produced via tester runNixOSTest, but the OfBorg build result shows it isn't. An example is tests.testers.runNixOSTest-example

error: a 'x86_64-linux' with features {} is required to build '/nix/store/rsjmz848mv48kil2s92vnzy4n7q53rcz-builder.pl.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}

See the OfBorg test result in #292759.

Steps To Reproduce

Steps to reproduce the behavior on MacOS:

1. Set up Nix on MacOS
2. git clone https://github.com/NixOS/nixpkgs.git
3. cd nixpkgs
4. Checkout commit
5. Build tests.testers.runNixOSTest-example
6. See build failure

Steps to reproduce the behavior on GitHub:

1. Go to tests.references: use runNixOSTest instead of nixosTest #292759 or any PR targeting the Nixpkgs master branch.
2. Comment @ofborg build tests.testers.runNixOSTest-example

Expected behavior

Package tests written by testers.runNixOSTest, including tests.testers.runNixOSTest-example, should run on MacOS. Related OfBorg build should be green.

Screenshots

Additional context

OfBorg build log links:

• x86_64-darwin
  https://logs.ofborg.org/?key=nixos/nixpkgs.292759&attempt_id=932a1ae7-c6fa-4f38-bd08-6f26b23ed12a

Notify maintainers

@Gabriella439

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

---

Add a 👍 reaction to issues you find important.

[Gabriella439]

The macOS support for NixOS tests only adds support for running them but not building them. You still need access to a Linux builder to build those tests.

[ShamrockLee]

Thanks for the quick response!

The macOS support for NixOS tests only adds support for running them but not building them. You still need access to a Linux builder to build those tests.

What's the difficulty to build NixOS tests on macOS? Isn't it just a QEMU virtual machine?

[Gabriella439]

It is possible to build Linux products on NixOS using something like darwin.linux-builder but we don't yet have a way to do that in the course of a Nix build. See:

https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder

[lilyinstarlight]

So I'm thinking about adding a config.disableDarwinNixOSTests to nixpkgs to make

nixpkgs/nixos/lib/testing/nodes.nix

Lines 19 to 34 in 795c30f

	guestSystem =
	if hostPlatform.isLinux
	then hostPlatform.system
	else
	let
	hostToGuest = {
	"x86_64-darwin" = "x86_64-linux";
	"aarch64-darwin" = "aarch64-linux";
	};
	supportedHosts = lib.concatStringsSep ", " (lib.attrNames hostToGuest);
	message =
	"NixOS Test: don't know which VM guest system to pair with VM host system: ${hostPlatform.system}. Perhaps you intended to run the tests on a Linux host, or one of the following systems that may run NixOS tests: ${supportedHosts}";
	in
	hostToGuest.${hostPlatform.system} or (throw message);

throw for any attempt to build NixOS tests on Darwin, to give CI an escape hatch to return the old behavior

Does anyone have opinions on this?

I'll probably have a PR prepared today or tomorrow

[ShamrockLee]

The macOS support for NixOS tests only adds support for running them but not building them. You still need acces to a Linux builder to build those tests.

Does that mean, those derivations for testing firs got built on the host if not on the cache, and then run in the VM?

[doronbehar]

So I'm thinking about adding a config.disableDarwinNixOSTests to nixpkgs to make

Any progress on that @lilyinstarlight ?