Make Cabal agnostic about working directory

[sheaf]

Template Α: This PR modifies cabal behaviour

This PR tackles #9702, making the library functions in Cabal agnostic to the working directory. In practice, this means that we distinguish FilePaths from un-interpreted SymbolicPaths. The latter may be paths that are relative to the working directory, and need to be interpreted with respect to a passed-in argument specifying the working directory, instead of using the working directory of the current process.
See Note [Symbolic paths] in Distribution.Utils.Path.

In particular, paths in the package description now use the SymbolicPath abstraction, which allows specifying whether they are allowed to be absolute, and, if they are relative, what they are relative to.
For example, source files are relative to a source search directory, data files are relative to the data directory, and doc files are relative to the package root.

Review notes

For review, I would recommend starting by reading the notes and comments in Distribution.Utils.Path.

The main goal is to not set the working directory when calling the Cabal library internally when building a package in cabal-install, and instead passing it separately (currently via a global flag to the Setup executable).
To enable this, the entire Cabal library needs to be able to interpret file paths with respect to a passed-in working directory, instead of using the working directory of the current process. This means all interactions with the file system (e.g. doesFileExist, rewriteFile etc), as well as all invocations of external processes (such as ghc) need to be scrutinised for correctness.

---

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added.

TODO:

• More extensive testing, e.g. compiling stackage.

QA notes

The testing strategy for this patch is simply to build, test & benchmark packages. The main difference is that we no longer set the working directory when invoking the Cabal library internally (see internalSetupMethod), relying instead on passing -working-dir.

[phadej]

I'm very unhappy to see these kind of changes. You don't get any benefit. The SymbolicPath PackageDir LicenseFile was already relative, it's a (relative) path from package directory to the license file.

You also drop some type-safety, as LicenseFile was telling that it's not some file, but a license file (e.g. not a some file in extraSrcFiles.

Again, I'm very unhappy to see this. You should have asked before accepting 10k diff.

[phadej]

In particular, there shouldn't be any non-relative files in .cabal files. The offenders like extra-lib-dirs simply don't belong into package files; those should be configured by users externally (e.g. in cabal.project or cabal.config files). That's the historical mistake which should be cleaned up, not further cemented.

[geekosaur]

Forcing users who want to install a package to turn it into a project or otherwise hack their config to make it work on e.g. MacOS isn't really acceptable.

[phadej]

I didn't mean you need to remove the support for extra-lib-dirs right away, but having system specific hard-coded paths is not correct either. The absolute paths do not belong into package definitions. That's why there are build-type: Configure, (soon Hooks) and build-type: Custom.

[sheaf]

I'm very unhappy to see these kind of changes. You don't get any benefit. The SymbolicPath PackageDir LicenseFile was already relative, it's a (relative) path from package directory to the license file.

You also drop some type-safety, as LicenseFile was telling that it's not some file, but a license file (e.g. not a some file in extraSrcFiles.

Again, I'm very unhappy to see this. You should have asked before accepting 10k diff.

Oleg, I'm sorry but I spent a lot of time trying to come up with a design that was workable for the whole of Cabal. To do that, I did find it necessary to change the API in several ways. For instance, yes, we no longer specify the exact kind of file:

• it lead to a proliferation of definitions for all the different kinds of files,
• in my experience it was not useful to keep track of which specific file something was pointing to, it was extra book-keeping that didn't in practice avoid any bugs.

One other major point of departure came from the fact I realised we unfortunately would not be able to only use relative symbolic paths everywhere; there are a lot of parts in Cabal which use paths that are either relative or absolute, so I had to make that work with the API.

I understand if you think this "cemented" bad parts of the codebase, but I want to say that it is much easier now to change these things because it is often a case of doing a search/replace, whereas when most of the codebase was "untyped" (= using FilePath) it took a lot of effort to identify the places which needed to be changed.

I know, and can tell, that you put serious consideration into the design of the initial API. I honestly did try sticking to it as much as I could but I was also informed by looking at what was actually in use in the Cabal codebase. Maybe you think that's a mistake, and that I should have stuck to a design and tried harder to make Cabal adapt to that abstract ideal, but I don't think I would have managed to write that patch with that approach.

[phadej]

there are a lot of parts in Cabal which use paths

That is the main pain point.

The SymbolicPath was intended to be used as symbolic path, i.e. paths to be interpreted that appear in .cabal files. Not any real filepath which may appear on the filesystem. That should had different type. FilePath or OsPath or ... I.e. not as typed wrapper around filepath / directory. Maybe it's not the case, but the distinction is now very blurry. (In particular, CWD root doesn't make sense for SymbolicPaths as paths in .cabal file in my design).

TL;DR I hope that SymbolicPath is not the only abstraction over paths, as in my opinion, it simply won't work well. Too many concerns, too complicated API. (Example distinction: symbolic paths are text, files on disk should eventually use OsPath as representation).

[ulidtko]

Breaking change to Cabal library users. 💥

import Distribution.Simple.Setup (BuildFlags (buildVerbosity))

miniRepro :: BuildFlags
miniRepro = emptyBuildFlags {buildVerbosity = undefined}

— compiles against Cabal 3.12; yields compile error with 3.14:

src/Distribution/Extra/Doctest.hs:235:13: error: [GHC-16444]
    • non-bidirectional pattern synonym ‘Cabal-3.14.0.0:Distribution.Simple.Setup.Build.BuildCommonFlags’ used in an expression
    • In a record update at field ‘buildVerbosity’
      and with pattern synonym ‘Cabal-3.14.0.0:Distribution.Simple.Setup.Build.BuildCommonFlags’.
      In the expression: emptyBuildFlags {buildVerbosity = undefined}
      In an equation for ‘miniRepro’:
          miniRepro = emptyBuildFlags {buildVerbosity = undefined}
    |
235 | miniRepro = emptyBuildFlags {buildVerbosity = undefined}
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

(edit: on GHC 9.6.4)