python3Packages.torch: "unbundle" git submodules

[ConnorBaker]

Describe the bug

Pytorch "vendors" a number of dependencies via git-submodules. The bundled libraries are complex enough that they suffer from the "cabal hell"/the "diamond problem"¹. The Nixpkgs' pytorch expression currently keeps the submodules as is, which means that:

• ...we cannot configure or consistently apply patches to torch's dependencies,
• cannot control the bootstrap process,
• and potentially suffer the increased build times (as we cannot cache the built dependencies as separate store paths)
• and increased closure sizes.

Pytorch's CMakeLists implement a custom mechanism, USE_SYSTEM_LIBS, that partially recovers the ability for dependency injection. Nonetheless, unbundling all of the vendored libraries apparently still takes substantial patching¹²³⁴.

The Guix-HPC project has, apparently, made great progress addressing this issue¹². Nixpkgs needs to catch up, but also we need to work with the upstream to render patching unnecessary.

Footnotes

1. "What's in a package": https://hpc.guix.info/blog/2021/09/whats-in-a-package/ ↩ ↩² ↩³

2. "[PATCH 00/31] Unbundle and update python-pytorch": https://issues.guix.gnu.org/69591 ↩ ↩²

3. "PyTorch with ROCm": https://lists.gnu.org/archive/html/guix-devel/2024-03/msg00242.html ↩

4. https://codeberg.org/dtelsing/Guix-HPC/commit/5da1afd389d5a0adcca83538d50c0a785f81eb29 ↩

[SomeoneSerge]

AFAIU, Guix-HPC has patched pytorch enough to disentangle the submodules:

• https://lists.gnu.org/archive/html/guix-devel/2024-03/msg00242.html
• https://issues.guix.gnu.org/69591
• https://hpc.guix.info/blog/2021/09/whats-in-a-package/

---

EDIT: @ConnorBaker I updated the description to reflect the fact that unvendoring is provably feasible