Cannot reference internal libraries with cabal-version: 3.4

[noughtmare]

Describe the bug

When using cabal-version: 3.4 it seems like I cannot directly reference internal libraries anymore. I can only use the experimental package:internal-lib notation.

To Reproduce
Steps to reproduce the behavior:

1. Make a cabal file with these contents:

   cabal-version: 3.4
   name: my-lib
   version: 0.1.0.0
   
   library my-lib-internal
   
   library
     build-depends: my-lib-internal

2. cabal v2-build

   Resolving dependencies...
   cabal: Could not resolve dependencies:
   [__0] trying: my-lib-0.1.0.0 (user goal)
   [__1] unknown package: my-lib-internal (dependency of my-lib)
   [__1] fail (backjumping, conflict set: my-lib, my-lib-internal)
   After searching the rest of the dependency tree exhaustively, these were the
   goals I've had most trouble fulfilling: my-lib, my-lib-internal
   

Expected behavior

A successful build like if I use cabal-version: 3.0 or as if I use the experimental my-lib:my-lib-internal notation in the build-depends field.

System information

• Operating system: NixOS 21.05

• cabal, ghc versions

  cabal-install version 3.4.0.0
  compiled using version 3.4.0.0 of the Cabal library 
  

  The Glorious Glasgow Haskell Compilation System, version 8.10.4
  

Additional context

From this stackoverflow question: https://stackoverflow.com/q/68880126/15207568

[fgaz]

This was partially discussed in #7286 and related tickets. Unfortunately removing the experimental label is blocked on #7270

[noughtmare]

I guess I'm still confused why cabal can't find the internal library without the : notation. Wouldn't it be sensible to default to internal libraries when resolving names that do not use that : notation, or at most showing a specific error if a package with the same name exists externally? Currently it just acts as if it doesn't know that the library exists at all, which is confusing.

Edit: I guess the reason is that build-depends field expects package names and not library names. The error message also mentions unknown package which hints in that direction. However, in the end, and especially with multi-library packages, isn't it libraries that we want to specify in the build-depends field? My intuition with my-lib:my-lib-internal is that it is like a qualified name for a library, not a kind of "sub-selection" of a package.

[phadej]

See #6083

EDIT: TL;DR sub libraries shadowing package names would require adding a syntax for package name build depends. It made more sense (also implementation wise, as name resolution is then simpler) to always require qualified names for sublibraries.
EDIT2: With public sublibraries, their name is not an implementation detail, so there could be situation where they cannot be renamed, but they shadow needed dependency.

This issue is IMO wontfix, except for possibly better diagnostics from solver.

[kczulko]

Sorry, now I understand what I did (therefore I've removed my previous msg)... In this case cabal-version is the entry in the cabal file... So indeed, for cabal-version >= 3.4 it is still reproducible.

I'm just going through the issues related to internal libraries and I've found this one. Thx @phadej for the explanation.

[jneira]

Not sure if the issue is actionable: internal libs can be referenced for cabal-version 3.4 and it is documented
Additional improvements could be discussed in new issues.

[noughtmare]

Actions I would like to see happen:

• Update documentation such as this section about internal libraries to use the now required foo:foo-internal syntax. And this last part needs to be removed:

  Shadowing can be used to vendor an external dependency into a package and thus emulate private dependencies. Below is an example based on a real-world use case:
  ...

  Maybe there are more places that still use the old syntax.

• Improve the error message in the case that an unknown package name in a build-depends field happens to be a valid name of an internal library. Then the error message can suggest using the new syntax.

[jneira]

Many thanks for summarize needed changes in docs, it seems we have the sketch for a pull request updating documentation!