Merge pull request #123682 from NixOS/haskell-updates

Haskell updates

pkgs/data/misc/hackage/pin.json

@@ -1,6 +1,6 @@
 {
-  "commit": "2295bd36e0d36af6e862dfdb7b0694fba2e7cb58",
-  "url": "https://github.com/commercialhaskell/all-cabal-hashes/archive/2295bd36e0d36af6e862dfdb7b0694fba2e7cb58.tar.gz",
-  "sha256": "1bzqy6kbw0i1ryg3ia5spg6m62zkc46xhhn0h76pfq7mfmm3fqf8",
-  "msg": "Update from Hackage at 2021-05-12T11:46:04Z"
+  "commit": "0fb7f9edea05a2b464b5667debe1e3ece585c185",
+  "url": "https://github.com/commercialhaskell/all-cabal-hashes/archive/0fb7f9edea05a2b464b5667debe1e3ece585c185.tar.gz",
+  "sha256": "01rzbda8g62gj2x3if46lglis9gqw3qfpqyiv2lrnm7alsg36ld9",
+  "msg": "Update from Hackage at 2021-05-19T07:17:55Z"
 }

pkgs/development/compilers/ghc/8.10.2-binary.nix

@@ -212,6 +212,9 @@ stdenv.mkDerivation rec {
   passthru = {
     targetPrefix = "";
     enableShared = true;
+
+    # Our Cabal compiler name
+    haskellCompilerName = "ghc-${version}";
   };
 
   meta = rec {

pkgs/development/compilers/ghc/8.6.5-binary.nix

@@ -171,6 +171,9 @@ stdenv.mkDerivation rec {
   passthru = {
     targetPrefix = "";
     enableShared = true;
+
+    # Our Cabal compiler name
+    haskellCompilerName = "ghc-${version}";
   };
 
   meta = rec {

pkgs/development/haskell-modules/HACKING.md

@@ -0,0 +1,270 @@
+
+## Maintainer Workflow
+
+The goal of the [@NixOS/haskell](https://github.com/orgs/NixOS/teams/haskell)
+team is to keep the Haskell packages in Nixpkgs up-to-date, while making sure
+there are no Haskell-related evaluation errors or build errors that get into
+the Nixpkgs `master` branch.
+
+We do this by periodically merging an updated set of Haskell packages on the
+`haskell-updates` branch into the `master` branch.  Each member of the team
+takes a two week period where they are in charge of merging the
+`haskell-updates` branch into `master`.  This is the documentation for this
+workflow.
+
+The workflow generally proceeds in three main steps:
+
+1. create the initial `haskell-updates` PR, and update Stackage and Hackage snapshots
+1. wait for contributors to fix newly broken Haskell packages
+1. merge `haskell-updates` into `master`
+
+Each of these steps is described in a separate section.
+
+### Initial `haskell-updates` PR
+
+In this section we create the PR for merging `haskell-updates` into `master`.
+
+1.  Make sure the `haskell-updates` branch is up-to-date with `master`.
+
+1.  Update the Stackage Nightly resolver used by Nixpkgs and create a commit:
+
+    ```console
+    $ ./maintainers/scripts/haskell/update-stackage.sh --do-commit
+    ```
+
+1.  Update the Hackage package set used by Nixpkgs and create a commit:
+
+    ```console
+    $ ./maintainers/scripts/haskell/update-hackage.sh --do-commit
+    ```
+
+1.  Regenerate the Haskell package set used in Nixpkgs and create a commit:
+
+    ```console
+    $ ./maintainers/scripts/haskell/regenerate-hackage-packages.sh --do-commit
+    ```
+
+1.  Push these commits to the Nixpkgs repository.
+
+1.  Open a PR on Nixpkgs merging `haskell-updates` into `master`.
+
+Use the following message body:
+
+```markdown
+### This Merge
+
+This PR is the regular merge of the `haskell-updates` branch into `master`.
+
+This branch is being continually built and tested by hydra at https://hydra.nixos.org/jobset/nixpkgs/haskell-updates.
+
+I will aim to merge this PR **by 2021-TODO-TODO**. If I can merge it earlier, there might be successor PRs in that time window. As part of our rotation @TODO will continue these merges from 2021-TODO-TODO to 2021-TODO-TODO.
+
+### haskellPackages Workflow Summary
+
+Our workflow is currently described in
+[`pkgs/development/haskell-modules/HACKING.md`](https://github.com/NixOS/nixpkgs/blob/haskell-updates/pkgs/development/haskell-modules/HACKING.md).
+
+The short version is this:
+* We regularly update the Stackage and Hackage pins on `haskell-updates` (normally at the beginning of a merge window).
+* The community fixes builds of Haskell packages on that branch.
+* We aim at at least one merge of `haskell-updates` into `master` every two weeks.
+* We only do the merge if the `mergeable` job is succeeding on hydra.
+* If a maintained package is still broken at the time of merge, we will only merge if the maintainer has been pinged 7 days in advance. (If you care about a Haskell package, become a maintainer!)
+
+---
+
+This is the follow-up to #TODO.
+```
+
+Make sure to replace all TODO with the actual values.
+
+### Notify Maintainers and Fix Broken Packages
+
+After you've done the previous steps, Hydra will start building the new and
+updated Haskell packages.  You can see the progress Hydra is making at
+https://hydra.nixos.org/jobset/nixpkgs/haskell-updates.  This Hydra jobset is
+defined in the file [release-haskell.nix](../../top-level/release-haskell.nix).
+
+#### Notify Maintainers
+
+When Hydra finishes building all the updated packages for the `haskell-updates`
+jobset, you should generate a build report to notify maintainers of their
+newly broken packages.  You can do that with the following commands:
+
+```console
+$ ./maintainers/scripts/haskell/hydra-report.hs get-report
+$ ./maintainers/scripts/haskell/hydra-report.hs ping-maintainers
+```
+
+The `hyda-report.hs ping-maintainers` command generates a Markdown document
+that you can paste in a GitHub comment on the PR opened above.  This
+comment describes which Haskell packages are now failing to build.  It also
+pings the maintainers so that they know to fix up their packages.
+
+It may be helpful to pipe `hydra-report.hs ping-maintainers` into `xclip`
+(XOrg) or `wl-copy` (Wayland) in order to post on GitHub.
+
+This build report can be fetched and re-generated for new Hydra evaluations.
+It may help contributors to try to keep the GitHub comment updated with the
+most recent build report.
+
+Maintainers should be given at least 7 days to fix up their packages when they
+break.  If maintainers don't fix up their packages with 7 days, then they
+may be marked broken before merging `haskell-updates` into `master`.
+
+#### Fix Broken Packages
+
+After getting the build report, you can see which packages and Hydra jobs are
+failing to build.  The most important jobs are the `maintained` and `mergeable`
+jobs. These are both defined in
+[`release-haskell.nix`](../../top-level/release-haskell.nix).
+
+`mergeable` is a set of the most important Haskell packages, including things
+like Pandoc and XMonad.  These packages are widely used.  We would like to
+always keep these building.
+
+`maintained` is a set of Haskell packages that have maintainers in Nixpkgs.
+We should be proactive in working with maintainers to keep their packages
+building.
+
+Steps to fix Haskell packages that are failing to build is out of scope for
+this document, but it usually requires fixing up dependencies that are now
+out-of-bounds.
+
+#### Mark Broken Packages
+
+Packages that do not get fixed can be marked broken with the following
+commands.  First check which packages are broken:
+
+```console
+$ ./maintainers/scripts/haskell/hydra-report.hs get-report
+$ ./maintainers/scripts/haskell/hydra-report.hs mark-broken-list
+```
+
+This shows a list of packages that reported a build failure on `x86_64-linux` on Hydra.
+
+Next, run the following command:
+
+```console
+$ ./maintainers/scripts/haskell/mark-broken.sh --do-commit
+```
+
+This first opens up an editor with the broken package list.  Some of these
+packages may have a maintainer in Nixpkgs.  If these maintainers have not been
+given 7 days to fix up their package, then make sure to remove those packages
+from the list before continuing.  After saving and exiting the editor, the
+following will happen:
+
+-   Packages from the list will be added to
+    [`configuration-hackage2nix/broken.yaml`](configuration-hackage2nix/broken.yaml).
+    This is a list of Haskell packages that are known to be broken.
+
+-   [`hackage-packages.nix`](hackage-packages.nix) will be regenerated.  This
+    will mark all Haskell pacakges in `configuration-hackage2nix/broken.yaml`
+    as `broken`.
+
+-   The
+    [`configuration-hackage2nix/transitive-broken.yaml`](configuration-hackage2nix/transitive-broken.yaml)
+    file will be updated.  This is a list of Haskell packages that
+    depend on a package in `configuration-hackage2nix/broken.yaml` or
+    `configuration-hackage2nix/transitive-broken.yaml`
+
+-   `hackage-packages.nix` will be regenerated again.  This will set
+    `hydraPlatforms = none` for all the packages in
+    `configuration-hackage2nix/transitive-broken.yaml`.  This makes
+    sure that Hydra does not try to build any of these packages.
+
+-   All updated files will be committed.
+
+#### Merge `master` into `haskell-updates`
+
+You should occasionally merge the `master` branch into the `haskell-updates`
+branch.
+
+In an ideal world, when we merge `haskell-updates` into `master`, it would
+cause few Hydra rebuilds on `master`.  Ideally, the `nixos-unstable` channel
+would never be prevented from progressing because of needing to wait for
+rebuilding Haskell packages.
+
+In order to make sure that there are a minimal number of rebuilds after merging
+`haskell-updates` into `master`, `master` should occasionally be merged into
+the `haskell-updates` branch.
+
+This is especially important after `staging-next` is merged into `master`,
+since there is a high chance that this will cause all the Haskell packages to
+rebuild.
+
+### Merge `haskell-updates` into `master`
+
+Now it is time to merge the `haskell-updates` PR you opened above.
+
+Before doing this, make sure of the following:
+
+-   All Haskell packages that fail to build are correctly marked broken or
+    transitively broken.
+
+-   The `maintained` and `mergeable` jobs are passing on Hydra.
+
+-   The maintainers for any maintained Haskell packages that are newly broken
+    have been pinged on GitHub and given at least a week to fix their packages.
+    This is especially important for widely-used packages like `cachix`.
+
+-   Make sure you first merge the `master` branch into `haskell-updates`.  Wait
+    for Hydra to evaluate the new `haskell-updates` jobset.  Make sure you only
+    merge `haskell-updates` into `master` when there are no evaluation errors.
+
+When you've double-checked these points, go ahead and merge the `haskell-updates` PR.
+After merging, **make sure not to delete the `haskell-updates` branch**, since it
+causes all currently open Haskell-related pull-requests to be automatically closed on GitHub.
+
+### Additional Info
+
+Here are some additional tips that didn't fit in above.
+
+-   Hydra tries to evalute the `haskell-updates` branch (in the
+    [`nixpkgs:haskell-updates`](https://hydra.nixos.org/jobset/nixpkgs/haskell-updates)
+    jobset) every 4 hours.  It is possible to force a new Hydra evaluation without
+    waiting 4 hours by the following steps:
+
+    1. Log into Hydra with your GitHub or Google account.
+    1. Go to the [nixpkgs:haskell-updates](https://hydra.nixos.org/jobset/nixpkgs/haskell-updates) jobset.
+    1. Click the `Actions` button.
+    1. Select `Evaluate this jobset`.
+    1. If you refresh the page, there should be a new `Evaluation running since:` line.
+    1. Evaluations take about 10 minutes to finish.
+
+-   It is sometimes helpful to update the version of
+    [`cabal2nix` / `hackage2nix`](https://github.com/NixOS/cabal2nix) that our
+    maintainer scripts use.  This can be done with the
+    [`maintainers/scripts/haskell/update-cabal2nix-unstable.sh`](../../../maintainers/scripts/haskell/update-cabal2nix-unstable.sh)
+    script.
+
+    You might want to do this if a user contributes a fix to `cabal2nix` that
+    will immediately fix a Haskell package in Nixpkgs.  First, merge in
+    the PR to `cabal2nix`, then run `update-cabal2nix-upstable.sh`.  Finally, run
+    [`regenerate-hackage-packages.sh`](../../../maintainers/scripts/haskell/regenerate-hackage-packages.sh)
+    to regenerate the Hackage package set with the updated version of `hackage2nix`.
+
+-   Make sure never to update the Hackage package hashes in
+    [`pkgs/data/misc/hackage/`](../../../pkgs/data/misc/hackage/), or the
+    pinned Stackage Nightly versions on the release branches (like
+    `release-21.05`).
+
+    This means that the
+    [`update-hackage.sh`](../../../maintainers/scripts/haskell/update-hackage.sh)
+    and
+    [`update-stackage.sh`](../../../maintainers/scripts/haskell/update-stackage.sh)
+    scripts should never be used on the release branches.
+
+    However, changing other files in `./.` and regenerating the package set is encouraged.
+    This can be done with
+    [`regenerate-hackage-packages.sh`](../../../maintainers/scripts/haskell/regenerate-hackage-packages.sh)
+    as described above.
+
+-   The Haskell team members generally hang out in the Matrix room
+    [#haskell:nixos.org](https://matrix.to/#/#haskell:nixos.org).
+
+## Contributor Workflow
+
+(TODO: this section is to describe the type of workflow for non-committers to
+contribute to `haskell-updates`)

pkgs/development/haskell-modules/cabal2nix-unstable.nix

@@ -8,10 +8,10 @@
 }:
 mkDerivation {
   pname = "cabal2nix";
-  version = "unstable-2021-05-06";
+  version = "unstable-2021-05-28";
   src = fetchzip {
-    url = "https://github.com/NixOS/cabal2nix/archive/b598bc4682b0827554b5780acdd6f948d320283b.tar.gz";
-    sha256 = "04afm56cyhj2l41cvq4z11k92jjchr21a8vg9pjaz438pma7jgw1";
+    url = "https://github.com/NixOS/cabal2nix/archive/5fb325e094af91328e02cc2ecfd211feaeb135a7.tar.gz";
+    sha256 = "1zbd336s99rgk24yjqlp012d0f66s5nf190sjmsl7mfhqx9j2y4l";
   };
   isLibrary = true;
   isExecutable = true;

pkgs/development/haskell-modules/configuration-arm.nix

@@ -86,6 +86,7 @@ self: super: {
   yesod-paginator = dontCheck super.yesod-paginator;
   grammatical-parsers = dontCheck super.grammatical-parsers;
   construct = dontCheck super.construct;
+  orbits = dontCheck super.orbits;
 
   # https://github.com/ekmett/half/issues/35
   half = dontCheck super.half;