singularity-tools: refactor and add runImageInLinuxVM; apptainer.passthru.tests.exec-image-in-linux-vm: init

[ShamrockLee]

Description of changes

• Refactor singularity-tools:
  • Replace mkLayer with pkgs.writeClosure. This prevents unnecessary copy of the closures just to get the reference list. (done in trivial-builders: replace writeReferencesToFile with writeClosure #178717)
  • Drop storeDir from the override interface.
  • Use directory name .singularity.d in sandbox for both apptainer and singularity
  • Split the buildImage into multiple reusable build helpers.
    • Add passthru.buildscriptPackage to each of the image derivation, which can be executed to build the image outside of Nix Store. This saves the space and CPU time needed to spawn a VM, allocate a virtual disk file, and copy the result image out from the store.
    • Make each image build helper run preImageBuild and postImageBuild hooks to provide a uniform overriding interface.
    • Split the privileged, VM-spawning build process into buildImageInLinuxVM. Add argument supportImageRuning to opt in image running support during build.
• Add runImageInLinuxVM which wraps around buildImageInLinuxVM.
• Add apptainer.passthru.tests.exec-image-in-linux-vm to automate the binary functionality tests. If applied, we can test the image building and running functionality of Apptainer simply by running the Nix CI, and version bumping PRs made by r-ryantm can be merged directly when all the tests is passed.
  It's currently impossible for singularity (Sylabs SingularityCE) to run images in Nix sandbox, since it (still) requires a loop device to mount the image, which is not available in the Nix sandbox.

Depends on #178717.

Initial effort for #224636.

Cc: @jbedo @SomeoneSerge @GaetanLepage

 _______________
< Hello, world! >
 ---------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.11 Release Notes (or backporting 23.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[ShamrockLee]

Result of nixpkgs-review run on x86_64-linux 1

[ShamrockLee]

@ofborg build apptainer.passthru.tests singularity.passthru.tests