New API Porting Guide

[Pacman99]

Hey everyone! We just did a lot of refactoring in the core branch and if you are updating to the latest changes I would recommend you read this guide on how to update.

This is a pretty quick explanation, to get more detail take a look at the doc folder which carefully goes over all the new API arguments - I also linked relevant doc sections to the headers.

The main things to notice are the changes to extern, suites and overrides.

Suites

Because of how important suites are to the configuration we decided it would be good to just define them in the flake.nix itself. You will notice a suites argument under the nixos section. You should copy all the content from your suites/default.nix to that argument.

Extern

The cool part about extern is that you can see everything a devos system depends on in one file. We believe that this information should actually be in the flake.nix too. So you can now just use different API arguments to pass external module and overlays.

Overlays

You can just put overlays in the overlays argument for the nixos channel. Where necessary, all external overlays will be automatically filtered out by inspecting your inputs.

Modules

There is a dedicated externalModules argument for this purpose.

specialArgs

We currently no longer allow passing specialArgs to host - its possible but quite difficult. The reasoning behind this was that you should add modules in the flake.nix directly instead of passing them to hosts and adding them. You can add modules to different hosts by adding lines to the hosts argument.

Overrides

Module overrides are no longer directly demonstrated by default, because of how finicky they are. It is possible by making use of the disabledModules key and the latestModulesPath special argument.

Package overrides, on the other hand, can be done in the overlays/overrides.nix file. There are plenty of examples there.

---

Generally as your merging, make sure to drop extern, overrides, and suites and move the code from there either to your flake.nix or the relevant files.
If you would like to split those things up into separate files, you can always just import files into the api arguments. Bonus: to de-clutter, you can leave out any import statements, where you would otherwise put them. Most items are piped through a maybeImport.

If you have any issues or comments about the API please comment here or make a dedicated issue. We would love to hear feedback!

[benneti]

Hey when pulling in the latest core into my config I get:

• error: gnome3.corePackages is removed since 2019-08-25: please use services.gnome3.core-shell.enable.
• I have desktopManager.gnome3.enable = true;, but rg corePackages gives me no results.

As the only change from my last working config was trying to switch to the new layout (twice the first try I did not see there was this guide) I guess there might be a name clash or something similar from this.
(it could be that this is due to another commit, too as my last merge was three weeks ago 82b73cf .)

EDIT: also happens for the direnv where I think no gnome modules and or packages should be used. Additionally show-trace only traces it back to the /nix/store/2y8xq739cy0klq7mh92fxziahda70ryd-source/pkgs/desktops/gnome/default.nix:332:3 in nixpkgs

[Pacman99]

error: gnome3.corePackages is removed since 2019-08-25: please use services.gnome3.core-shell.enable.

This is very interesting. At first look I would think it was because of using an option and then updating nixpkgs. But considering you never set corePackages and the option is from over a year ago, that doesn't really make sense.

Can you post the full stack trace output of the configuration building, that should help with debugging. Also if possible, seeing your repository's code could be helpful - if its private then just the stack trace should be okay.

[benneti]

thank you for taking the time, here is the link to the broken config https://cloud.tissot.de/gitea/benneti/nixos/src/branch/broken and here is the traceback:

error: gnome.corePackages is removed since 2019-08-25: please use `services.gnome.core-shell.enable`

       … while evaluating the attribute 'corePackages'

       at /nix/store/2y8xq739cy0klq7mh92fxziahda70ryd-source/pkgs/desktops/gnome/default.nix:332:3:

          331|   # added 2019-08-25
          332|   corePackages = throw "gnome.corePackages is removed since 2019-08-25: please use `services.gnome.core-shell.enable`";
             |   ^
          333|   optionalPackages = throw "gnome.optionalPackages is removed since 2019-08-25: please use `services.gnome.core-utilities.enable`";

       … while evaluating 'op'

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:3:19:

            2| let
            3|   op = sum: path: val:
             |                   ^
            4|     let

       … from call site

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:30:18:

           29|     builtins.foldl'
           30|       (sum: key: op sum (path ++ [ key ]) val.${key})
             |                  ^
           31|       sum

       … while evaluating anonymous lambda

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:30:13:

           29|     builtins.foldl'
           30|       (sum: key: op sum (path ++ [ key ]) val.${key})
             |             ^
           31|       sum

       … from call site

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:29:5:

           28|   recurse = sum: path: val:
           29|     builtins.foldl'
             |     ^
           30|       (sum: key: op sum (path ++ [ key ]) val.${key})

       … while evaluating 'recurse'

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:28:24:

           27|
           28|   recurse = sum: path: val:
             |                        ^
           29|     builtins.foldl'

       … from call site

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:21:8:

           20|     # recurse into that attribute set
           21|       (recurse sum path val)
             |        ^
           22|     else

       … while evaluating 'op'

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:3:19:

            2| let
            3|   op = sum: path: val:
             |                   ^
            4|     let

       … from call site

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:30:18:

           29|     builtins.foldl'
           30|       (sum: key: op sum (path ++ [ key ]) val.${key})
             |                  ^
           31|       sum

       … while evaluating anonymous lambda

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:30:13:

           29|     builtins.foldl'
           30|       (sum: key: op sum (path ++ [ key ]) val.${key})
             |             ^
           31|       sum

       … from call site

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:29:5:

           28|   recurse = sum: path: val:
           29|     builtins.foldl'
             |     ^
           30|       (sum: key: op sum (path ++ [ key ]) val.${key})

       … while evaluating 'recurse'

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:28:24:

           27|
           28|   recurse = sum: path: val:
             |                        ^
           29|     builtins.foldl'

       … from call site

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:35:1:

           34| in
           35| recurse { } [ ] tree
             | ^

       … while evaluating anonymous lambda

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/flattenTree.nix:1:1:

            1| tree:
             | ^
            2| let

       … from call site

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/default.nix:101:23:

          100|   #   }
          101|   flattenTree = tree: import ./flattenTree.nix tree;
             |                       ^
          102|

       … while evaluating 'flattenTree'

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/default.nix:101:17:

          100|   #   }
          101|   flattenTree = tree: import ./flattenTree.nix tree;
             |                 ^
          102|

       … from call site

       at /nix/store/a5kj99c62gd9ybk0rx61vj0gk227i9l0-source/packagesFromOverlaysBuilderConstructor.nix:74:32:

           73|         # flatten nested sets with "/" delimiter then drop disallowed packages
           74|         filterPackages system (flattenTree exportPackages);
             |                                ^
           75|

       … while evaluating 'filterAttrs'

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/filterPackages.nix:7:23:

            6|   nameValuePair = name: value: { inherit name value; };
            7|   filterAttrs = pred: set:
             |                       ^
            8|     listToAttrs (

       … from call site

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/filterPackages.nix:30:1:

           29| in
           30| filterAttrs sieve packages
             | ^

       … while evaluating anonymous lambda

       at /nix/store/2d4rhqdx6jcm8i747aar73rr80dqg9xi-source/filterPackages.nix:2:9:

            1| { allSystems }:
            2| system: packages:
             |         ^
            3| let

       … from call site

       at /nix/store/a5kj99c62gd9ybk0rx61vj0gk227i9l0-source/packagesFromOverlaysBuilderConstructor.nix:74:9:

           73|         # flatten nested sets with "/" delimiter then drop disallowed packages
           74|         filterPackages system (flattenTree exportPackages);
             |         ^
           75|

       … while evaluating 'packagesFromOverlaysBuilder'

       at /nix/store/a5kj99c62gd9ybk0rx61vj0gk227i9l0-source/packagesFromOverlaysBuilderConstructor.nix:8:37:

            7|
            8|       packagesFromOverlaysBuilder = channels:
             |                                     ^
            9|         /**

       … from call site

       at /nix/store/a5kj99c62gd9ybk0rx61vj0gk227i9l0-source/systemFlake.nix:195:29:

          194|             (optionalAttrs (args ? ${builder})
          195|               { ${output} = args.${builder} pkgs; });
             |                             ^
          196|       in

       … while evaluating anonymous lambda

       at /nix/store/a5kj99c62gd9ybk0rx61vj0gk227i9l0-source/flake.nix:38:20:

           37|           lhs // mapAttrs
           38|             (name: value:
             |                    ^
           39|               if isAttrs value then lhs.${name} or { } // value

       … from call site

       … while evaluating the attribute 'packages'
sudo: /tmp/nixos-rebuild.n1fALU/result/bin/switch-to-configuration: command not found
warning: error(s) occurred while switching to the new configuration
/etc/nixos

[Pacman99]

Thank you for the detailed report! I think I know exactly what is going on now. I opened an issue to explain the problem in digga: #13.

Although this might actually a problem that is being triggered by a function in flake-utils-plus. So we likely have to go there to fix it.

As a workaround you can add the __dontExport property to any overlay where you override gnome. And that should prevent that error from being triggered.

[benneti]

thank you very much, overlays were basically the only thing I did not consider (as I forgot I had one for nautilus and thereby gnome admittedly)!
I think some of these new paradigms are a bit weird (at least for me) as they are counterintuitive to the lazy evaluation of nix.

[Pacman99]

I think some of these new paradigms are a bit weird (at least for me) as they are counterintuitive to the lazy evaluation of nix.

Not sure if your talking about overlays or devos. One of the goals for devos is to try and intuitively export as many of your creations as possible. We tried to create a sharing model for overlays and packages for anyone that uses digga. So its a bit counter-intuitive in terms of lazy evaluation, but fits well with flakes and the sharing goal.

[benneti]

Yes thats what I meant.

[benneti]

I think there is another issue due to exporting everything, it seems like packages in overlays that are currently broken are tried to be built when entering the direnv currently.
An example for this is emacsPgtkGcc from the emacs-overlay which is built when entering my nixos config folder but is currently broken, making it unnecessary hard to enter the direnv.

[benneti]

Sorry for kinda spamming this thread, but I think there is another small problem.
I think overlays, that change custom pkgs are not applied anymore.
I guess this is not a hard problem but sometimes it is cleaner to have a package close to upstream and then apply some changes in an overlay.

[Pacman99]

I think there is another issue due to exporting everything, it seems like packages in overlays that are currently broken are tried to be built when entering the direnv currently.

This could be because we try to build all packages defined in your overlay when you enter the devShell. This follows the same rules as package exporting, so if you don't want a package to be built you can set __dontExport in the overlay or mark packages as broken. devos/digga can't actually tell which packages are broken unless you explicitly mark it as such and its also good practice to do so. Now if you have done either, set __dontExport or marked the package, that is probably a bug, they shouldn't be built when entering the devShell. Or if you disagree with the practice feel free to explain why and we can discuss how a change would look.

I think overlays, that change custom pkgs are not applied anymore.
I guess this is not a hard problem but sometimes it is cleaner to have a package close to upstream and then apply some changes in an overlay

Now this is an issue, all overlays should be applied to the channels they are passed to. Can you include more details about this? Which overlays weren't applied, and where did you use the package that didn't get changed?

Sorry for kinda spamming this thread, but I think there is another small problem.

Please keep spamming, the more information here the more stable devos is for all of us.

[benneti]

How would I add __dontExport to emacs-overlay (thats not part of my config but an external overlay from another flake, for me that is https://github.com/nix-community/emacs-overlay ).

The package that had the overlay not applied was schildichat and it is installed via homemanager, here the link to the commit where i move the contents of the overlay to the package which makes things work again, but is not that clean anymore https://cloud.tissot.de/gitea/benneti/nixos/commit/09b80c090dab64fb4f7636807eb07d4736d6a620

[Pacman99]

And I realized I disagree with my old self: #14

I don't think building all packages from overlays was the best idea.