Release 2.5.6 (#21336)

* Remove Support For Deprecated Habana (#21327)

* remove habana

* changelog

* remove from install

* update docs

* update

* update

* pytest match

* Add name to accelerator interface (#21325)

* changelog + rebase

* version.info bumo

* fix: Checkout and Git LFS fetch issue for pytorch docs workflow (#21219)

* ci: `uv` for `docs-build` (#21206)

replace pip with uv

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* fix: failing ci linkcheck in markdown (#21222)

Update timeout and retry configuration
Increased timeout and adjusted retry settings.

* simplify lit CI config (#21220)

* simplify lit CI config

* Empty-Commit

* Empty-Commit

* Update

---------

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* fix:docs build errors (#21286)

* docs: add 'typing.Union' to nitpick_ignore_regex in Sphinx configuration

* docs: add Supermicro link to linkcheck_ignore due to 403 error

* [Hotfix] Remove reference to `torch.jit.export` (#21292)

update docs

* Docs: drop deprecated project (#21250)

* drop dumping wheels after UV switch (#21217)

* chore(`ci/litjob`): fix and optimize GPU related tests (#21290)

* fix: add unzip to dependencies in pytorch workflow
* fix: update variable interpolation for PACKAGE_NAME in YAML workflows
* fix: update variable interpolation for PACKAGE_NAME in fabric workflow
* fix: disable CUDA_LAUNCH_BLOCKING to speed up tests
* apply suggestion

* remove integrations from Dockerfile

* Update readme (#21263)

Update LitServe description for clarity

* Update heading from 'Why PyTorch Lightning?' to 'Convert from PyTorch to PyTorch Lightning?'

* Fix heading formatting in README.md

* Add UTM parameters to links in README

Updated links in README to include UTM parameters for tracking.

* readme

readme

* readme updates

readme updates

* update readme

Update readme

* update

* update

* update

* update

* resolve docker builds

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: Bhimraj Yadav <bhimrajyadav977@gmail.com>
Co-authored-by: Shion Matsumoto <smatsumoto97@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Deependu <deependujha21@gmail.com>
Co-authored-by: Nicki Skafte Detlefsen <skaftenicki@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>

Author: 7 people

.actions/assistant.py

@@ -459,6 +459,25 @@ def generate_docker_tags(
         tags = [f"{docker_project}:{tag}" for tag in tags]
         print(",".join(tags))
 
+    @staticmethod
+    def prune_pytest_as_errors(
+        pyproject_toml: str = "pyproject.toml", errors: tuple = ("FutureWarning", "DeprecationWarning")
+    ) -> None:
+        """Prune pytest warnings as errors from the pyproject.toml file."""
+        import tomlkit
+
+        with open(pyproject_toml, encoding="utf-8") as fopen:
+            content = fopen.read()
+        pyproject = tomlkit.parse(content)
+        filterwarnings = pyproject.get("tool", {}).get("pytest", {}).get("ini_options", {}).get("filterwarnings", [])
+        if not filterwarnings:
+            return
+        filterwarnings = [wrn for wrn in filterwarnings if not any(f"error::{err}" in wrn for err in errors)]
+        pyproject["tool"]["pytest"]["ini_options"]["filterwarnings"] = filterwarnings
+
+        with open(pyproject_toml, "w", encoding="utf-8") as fopen:
+            fopen.write(tomlkit.dumps(pyproject))
+
 
 if __name__ == "__main__":
     import jsonargparse

.azure/gpu-tests-fabric.yml

@@ -76,7 +76,7 @@ jobs:
           cuda_ver=$(python -c "import torch ; print(''.join(map(str, torch.version.cuda.split('.')[:2])))")
           echo "##vso[task.setvariable variable=CUDA_VERSION_MM]$cuda_ver"
           echo "##vso[task.setvariable variable=TORCH_URL]https://download.pytorch.org/whl/cu${cuda_ver}/torch_stable.html"
-          scope=$(python -c 'n = "$(PACKAGE_NAME)" ; print(dict(fabric="lightning_fabric").get(n, n))')
+          scope=$(python -c 'n = "${PACKAGE_NAME}" ; print(dict(fabric="lightning_fabric").get(n, n))')
           echo "##vso[task.setvariable variable=COVERAGE_SOURCE]$scope"
         displayName: "set env. vars"
       - bash: |
@@ -135,7 +135,7 @@ jobs:
 
       - bash: |
           set -e
-          extra=$(python -c "print({'lightning': 'fabric-'}.get('$(PACKAGE_NAME)', ''))")
+          extra=$(python -c "print({'lightning': 'fabric-'}.get('${PACKAGE_NAME}', ''))")
           pip install -e ".[${extra}dev]" -U --upgrade-strategy=eager --extra-index-url="${TORCH_URL}"
         displayName: "Install package & dependencies"
 

.azure/gpu-tests-pytorch.yml

@@ -80,7 +80,7 @@ jobs:
           cuda_ver=$(python -c "import torch ; print(''.join(map(str, torch.version.cuda.split('.')[:2])))")
           echo "##vso[task.setvariable variable=CUDA_VERSION_MM]$cuda_ver"
           echo "##vso[task.setvariable variable=TORCH_URL]https://download.pytorch.org/whl/cu${cuda_ver}/torch_stable.html"
-          scope=$(python -c 'n = "$(PACKAGE_NAME)" ; print(dict(pytorch="pytorch_lightning").get(n, n))')
+          scope=$(python -c 'n = "${PACKAGE_NAME}" ; print(dict(pytorch="pytorch_lightning").get(n, n))')
           echo "##vso[task.setvariable variable=COVERAGE_SOURCE]$scope"
         displayName: "set env. vars"
       - bash: |
@@ -139,7 +139,7 @@ jobs:
 
       - bash: |
           set -e
-          extra=$(python -c "print({'lightning': 'pytorch-'}.get('$(PACKAGE_NAME)', ''))")
+          extra=$(python -c "print({'lightning': 'pytorch-'}.get('${PACKAGE_NAME}', ''))")
           pip install -e ".[${extra}dev]" -U --upgrade-strategy=eager --extra-index-url="${TORCH_URL}"
         displayName: "Install package & dependencies"
 

.github/actions/pip-wheels/action.yml

@@ -21,10 +21,10 @@ subprojects:
     checks:
       - "pl-cpu-guardian" # aggregated check for all cases
 
-  - id: "pytorch_lightning: Azure GPU"
+  - id: "pytorch_lightning: lit GPU"
     paths:
       - ".actions/*"
-      - ".azure/gpu-tests-pytorch.yml"
+      - ".lightning/workflows/pytorch.yml"
       # only the azure GPU workflow runs the examples
       # all examples don't need to be added because they aren't used in CI, but these are
       - "examples/run_pl_examples.sh"
@@ -47,13 +47,13 @@ subprojects:
       - "!*.md"
       - "!**/*.md"
     checks:
-      - "pytorch-lightning (GPUs) (testing Lightning | latest)"
-      - "pytorch-lightning (GPUs) (testing PyTorch | oldest)"
-      - "pytorch-lightning (GPUs) (testing PyTorch | latest)"
+      - "pytorch.yml / Lit Job (nvidia/cuda:12.1.1-devel-ubuntu22.04, pytorch, 3.10)"
+      - "pytorch.yml / Lit Job (lightning, 3.12)"
+      - "pytorch.yml / Lit Job (pytorch, 3.12)"
 
-  - id: "pytorch_lightning: Benchmarks"
+  - id: "Benchmarks"
     paths:
-      - ".azure/gpu-benchmarks.yml"
+      - ".lightning/workflows/benchmark.yml"
       - "requirements/fabric/**"
       - "requirements/pytorch/**"
       - "src/lightning/fabric/**"
@@ -65,7 +65,8 @@ subprojects:
       - "!*.md"
       - "!**/*.md"
     checks:
-      - "lightning.Benchmarks"
+      - "benchmark.yml / Lit Job (fabric)"
+      - "benchmark.yml / Lit Job (pytorch)"
 
   # Temporarily disabled
   #  - id: "pytorch-lightning: TPU workflow"
@@ -111,34 +112,6 @@ subprojects:
       - "docs-make (pytorch, doctest)"
       - "docs-make (pytorch, html)"
 
-  - id: "pytorch_lightning: Docker"
-    paths:
-      - ".actions/*"
-      - ".github/workflows/docker-build.yml"
-      - "dockers/**"
-      - "requirements/pytorch/**"
-      - "requirements/fabric/**"
-      - "setup.py"
-      - "!requirements/*/docs.txt"
-      - "!*.md"
-      - "!**/*.md"
-    checks:
-      - "build-cuda (3.10, 2.1.2, 12.1.1)"
-      - "build-cuda (3.11, 2.2.2, 12.1.1)"
-      - "build-cuda (3.11, 2.3.1, 12.1.1)"
-      - "build-cuda (3.11, 2.4.1, 12.1.1)"
-      - "build-cuda (3.12, 2.5.1, 12.1.1)"
-      - "build-cuda (3.12, 2.6.0, 12.4.1)"
-      #- "build-NGC"
-      - "build-pl (3.10, 2.1, 12.1.1)"
-      - "build-pl (3.11, 2.2, 12.1.1)"
-      - "build-pl (3.11, 2.3, 12.1.1)"
-      - "build-pl (3.11, 2.4, 12.1.1)"
-      - "build-pl (3.12, 2.5, 12.1.1)"
-      - "build-pl (3.12, 2.6, 12.4.1)"
-      - "build-pl (3.12, 2.7, 12.6.3)"
-      - "build-pl (3.12, 2.8, 12.6.3, true)"
-
   # SECTION: lightning_fabric
 
   - id: "lightning_fabric: CPU workflow"
@@ -156,10 +129,10 @@ subprojects:
     checks:
       - "fabric-cpu-guardian" # aggregated check for all cases
 
-  - id: "lightning_fabric: Azure GPU"
+  - id: "lightning_fabric: lit GPU"
     paths:
       - ".actions/*"
-      - ".azure/gpu-tests-fabric.yml"
+      - ".lightning/workflows/fabric.yml"
       - "examples/fabric/**"
       - "examples/run_fabric_examples.sh"
       - "requirements/fabric/**"
@@ -175,9 +148,9 @@ subprojects:
       - "!*.md"
       - "!**/*.md"
     checks:
-      - "lightning-fabric (GPUs) (testing Fabric | oldest)"
-      - "lightning-fabric (GPUs) (testing Fabric | latest)"
-      - "lightning-fabric (GPUs) (testing Lightning | latest)"
+      - "fabric.yml / Lit Job (nvidia/cuda:12.1.1-devel-ubuntu22.04, fabric, 3.10)"
+      - "fabric.yml / Lit Job (fabric, 3.12)"
+      - "fabric.yml / Lit Job (lightning, 3.12)"
 
   # Temporarily disabled
   #  - id: "lightning_fabric: TPU workflow"

.github/checkgroup.yml

@@ -54,12 +54,6 @@ package:
           - "src/lightning_fabric/*"
           - "src/pytorch_lightning/*"
 
-dockers:
-  - changed-files:
-      - any-glob-to-any-file:
-          - "dockers/**"
-          - ".github/workflows/docker-build.yml"
-
 dependencies:
   - changed-files:
       - any-glob-to-any-file:

.github/label-change.yml

@@ -23,8 +23,8 @@
       }
     }
   ],
-  "timeout": "20s",
+  "timeout": "30s",
   "retryOn429": true,
-  "retryCount": 5,
-  "fallbackRetryDelay": "20s"
+  "retryCount": 10,
+  "fallbackRetryDelay": "10s"
 }

.github/markdown-links-config.json

@@ -26,7 +26,7 @@ jobs:
         pkg-name: ${{ fromJSON(inputs.pkg-names) }}
     steps:
       - uses: actions/checkout@v5
-      - uses: actions/setup-python@v5
+      - uses: actions/setup-python@v6
         with:
           python-version: "3.x"
 
@@ -41,7 +41,7 @@ jobs:
           mkdir -p pypi/${{ matrix.pkg-name }}
           cp dist/* pypi/${{ matrix.pkg-name }}/
 
-      - uses: actions/upload-artifact@v4
+      - uses: actions/upload-artifact@v5
         with:
           name: ${{ inputs.artifact-name }}-${{ matrix.pkg-name }}
           path: pypi
@@ -51,7 +51,7 @@ jobs:
     needs: build-packages
     runs-on: ubuntu-22.04
     steps:
-      - uses: actions/download-artifact@v5
+      - uses: actions/download-artifact@v6
         with: # download all build artifacts
           pattern: ${{ inputs.artifact-name }}-*
           merge-multiple: true
@@ -62,7 +62,7 @@ jobs:
 
       - name: Keep artifact
         run: python -c "print('DAYS=' + str(5 if '${{ github.event_name }}'.startswith('pull_request') else 0))" >> $GITHUB_ENV
-      - uses: actions/upload-artifact@v4
+      - uses: actions/upload-artifact@v5
         with:
           name: ${{ inputs.artifact-name }}
           path: pypi

.github/workflows/_build-packages.yml

@@ -58,7 +58,7 @@ jobs:
       - uses: actions/checkout@v5
 
       - name: Install uv and set Python version
-        uses: astral-sh/setup-uv@v6
+        uses: astral-sh/setup-uv@v7
         with:
           python-version: "3.9"
           # TODO: Avoid activating environment like this
@@ -108,7 +108,7 @@ jobs:
           python -c "print('AWS_RUN=' + str('' if '${{inputs.push_to_s3}}' == 'true' else '--dryrun'))" >> $GITHUB_ENV
 
       - name: Upload checkpoints to GitHub Actions artifact
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@v5
         with:
           name: checkpoints-${{ github.sha }}
           path: ${{ env.LEGACY_FOLDER }}/checkpoints/

.github/workflows/_legacy-checkpoints.yml

@@ -47,10 +47,10 @@ jobs:
         python-version: ["3.9", "3.11"]
     steps:
       - uses: actions/checkout@v5
-      - uses: actions/setup-python@v5
+      - uses: actions/setup-python@v6
         with:
           python-version: ${{ matrix.python-version }}
-      - uses: actions/download-artifact@v5
+      - uses: actions/download-artifact@v6
         with:
           name: dist-packages-${{ github.sha }}
           path: dist