Avoid setting the multiprocessing context when importing lightning #18177

awaelchli · 2023-07-27T12:31:58Z

What does this PR do?

Was this discussed/agreed via a GitHub issue? (not for typos and docs) Yes, approved by @lantiga offline
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @Borda @carmocca @justusschock @awaelchli

for more information, see https://pre-commit.ci

github-actions · 2023-07-27T12:59:18Z

🌩️ Required checks status: Pending 🟡

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11)	success	✅
pl-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	success	✅

These checks are required after the changes to tests/tests_pytorch/utilities/test_imports.py, pyproject.toml.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs)	success	✅

These checks are required after the changes to src/lightning/__init__.py, tests/tests_pytorch/utilities/test_imports.py, pyproject.toml.

🟢 fabric: Docs

Check ID	Status
make-doctest (fabric)	success	✅
make-html (fabric)	success	✅

These checks are required after the changes to pyproject.toml.

🟢 pytorch_lightning: Docs

Check ID	Status
make-doctest (pytorch)	success	✅
make-html (pytorch)	success	✅

These checks are required after the changes to pyproject.toml.

🟢 lightning_data: CPU workflow

Check ID	Status
data-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
data-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
data-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅

These checks are required after the changes to pyproject.toml.

🟢 lightning_fabric: CPU workflow

Check ID	Status
fabric-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
fabric-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11)	success	✅
fabric-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13)	success	✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13)	success	✅

These checks are required after the changes to tests/tests_fabric/utilities/test_imports.py, pyproject.toml.

🟢 lightning_fabric: Azure GPU

Check ID	Status
lightning-fabric (GPUs)	success	✅

These checks are required after the changes to src/lightning/__init__.py, tests/tests_fabric/utilities/test_imports.py, pyproject.toml.

🟢 lightning_app: Tests workflow

Check ID	Status
app-pytest (macOS-11, lightning, 3.8, latest)	success	✅
app-pytest (macOS-11, lightning, 3.8, oldest)	success	✅
app-pytest (macOS-11, app, 3.9, latest)	success	✅
app-pytest (ubuntu-20.04, lightning, 3.8, latest)	success	✅
app-pytest (ubuntu-20.04, lightning, 3.8, oldest)	success	✅
app-pytest (ubuntu-20.04, app, 3.9, latest)	success	✅
app-pytest (windows-2022, lightning, 3.8, latest)	success	✅
app-pytest (windows-2022, lightning, 3.8, oldest)	success	✅
app-pytest (windows-2022, app, 3.8, latest)	success	✅

These checks are required after the changes to src/lightning/app/__init__.py, src/lightning/app/pdb/__init__.py, src/lightning/app/pdb/pdb.py, src/lightning/app/perf/__init__.py.

🟢 lightning_app: Examples

Check ID	Status
app-examples (macOS-11, lightning, 3.9, latest)	success	✅
app-examples (macOS-11, lightning, 3.9, oldest)	success	✅
app-examples (macOS-11, app, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, lightning, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, lightning, 3.9, oldest)	success	✅
app-examples (ubuntu-20.04, app, 3.9, latest)	success	✅
app-examples (windows-2022, lightning, 3.9, latest)	success	✅
app-examples (windows-2022, lightning, 3.9, oldest)	success	✅
app-examples (windows-2022, app, 3.9, latest)	success	✅

These checks are required after the changes to src/lightning/app/__init__.py, src/lightning/app/pdb/__init__.py, src/lightning/app/pdb/pdb.py, src/lightning/app/perf/__init__.py.

🟡 lightning: Store

Check ID	Status
Store.cloud (test_store Linux)	no_status	❓
Store.cloud (test_store Mac)	no_status	❓
Store.cloud (test_store Windows)	no_status	❓

These checks are required after the changes to src/lightning/__init__.py.

🟢 lightning_app: Docs

Check ID	Status
make-doctest (app)	success	✅
make-html (app)	success	✅

These checks are required after the changes to src/lightning/app/__init__.py, src/lightning/app/pdb/__init__.py, src/lightning/app/pdb/pdb.py, src/lightning/app/perf/__init__.py, docs/source-app/levels/basic/scripts/toy_app_1_component_pdb.py, docs/source-app/levels/intermediate/debug_a_lightning_app.rst, docs/source-app/levels/intermediate/debug_app_scripts/toy_app_1_component_pdb.py, pyproject.toml.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/__init__.py, src/lightning/app/__init__.py, src/lightning/app/pdb/__init__.py, src/lightning/app/pdb/pdb.py, src/lightning/app/perf/__init__.py, pyproject.toml.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.10)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.10)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.10)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.10)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.10)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.10)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.10)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.10)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.10)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.10)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.10)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.10)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.10)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.10)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.10)	success	✅

These checks are required after the changes to src/lightning/__init__.py, src/lightning/app/__init__.py, src/lightning/app/pdb/__init__.py, src/lightning/app/pdb/pdb.py, src/lightning/app/perf/__init__.py.

🟢 link-check

Check ID	Status
check-md-links / markdown-link-check	success	✅

These checks are required after the changes to src/lightning/app/CHANGELOG.md, src/lightning/fabric/CHANGELOG.md, src/lightning/pytorch/CHANGELOG.md.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

lantiga

LGTM

.github/workflows/ci-tests-app.yml

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

codecov · 2023-07-27T17:06:37Z

Codecov Report

Merging #18177 (ee570b2) into master (4e30879) will increase coverage by 38%.
Report is 1 commits behind head on master.
The diff coverage is 80%.

❗ Current head ee570b2 differs from pull request most recent head 2e1f77f. Consider uploading reports for the commit 2e1f77f to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #18177      +/-   ##
==========================================
+ Coverage      45%      83%     +38%     
==========================================
  Files         424      432       +8     
  Lines       32243    32907     +664     
==========================================
+ Hits        14489    27363   +12874     
+ Misses      17754     5544   -12210

tchaton

LGTM !

…18177) * avoid import at top module * tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove comment * update docs * changelog * mypy * trigger app tests * can't import lightning on py 3.8 * Update .github/workflows/ci-tests-app.yml Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com> --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com> (cherry picked from commit 1db4713)

awaelchli added 2 commits July 27, 2023 08:14

avoid import at top module

7886cb6

tests

fc0d79c

github-actions bot added fabric lightning.fabric.Fabric app (removed) Generic label for Lightning App package pl Generic label for PyTorch Lightning package labels Jul 27, 2023

pre-commit-ci bot and others added 5 commits July 27, 2023 12:33

[pre-commit.ci] auto fixes from pre-commit.com hooks

9fa886c

for more information, see https://pre-commit.ci

remove comment

9d077ce

update docs

b0e8b59

changelog

7726d12

mypy

fe759a3

awaelchli marked this pull request as ready for review July 27, 2023 12:46

awaelchli requested review from tchaton, lantiga, ethanwharris, carmocca, justusschock, williamFalcon and Borda as code owners July 27, 2023 12:46

awaelchli added bug Something isn't working performance labels Jul 27, 2023

awaelchli added this to the 2.0.x milestone Jul 27, 2023

trigger app tests

32a58ba

github-actions bot added the ci Continuous Integration label Jul 27, 2023

can't import lightning on py 3.8

af6c295

lantiga approved these changes Jul 27, 2023

View reviewed changes

Borda approved these changes Jul 27, 2023

View reviewed changes

.github/workflows/ci-tests-app.yml Outdated Show resolved Hide resolved

mergify bot added the ready PRs ready to be merged label Jul 27, 2023

Update .github/workflows/ci-tests-app.yml

ee570b2

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

github-actions bot removed the ci Continuous Integration label Jul 27, 2023

mergify bot added has conflicts and removed ready PRs ready to be merged labels Jul 27, 2023

Merge branch 'master' into bugfix/multiprocessing-context

2e1f77f

mergify bot added ready PRs ready to be merged and removed has conflicts ready PRs ready to be merged labels Jul 28, 2023

tchaton approved these changes Jul 28, 2023

View reviewed changes

justusschock approved these changes Jul 31, 2023

View reviewed changes

Merge branch 'master' into bugfix/multiprocessing-context

d175637

lantiga merged commit 1db4713 into master Jul 31, 2023
120 of 121 checks passed

lantiga deleted the bugfix/multiprocessing-context branch July 31, 2023 16:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid setting the multiprocessing context when importing lightning #18177

Avoid setting the multiprocessing context when importing lightning #18177

awaelchli commented Jul 27, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Jul 27, 2023 •

edited

Loading

lantiga left a comment

codecov bot commented Jul 27, 2023 •

edited

Loading

tchaton left a comment

Avoid setting the multiprocessing context when importing lightning #18177

Avoid setting the multiprocessing context when importing lightning #18177

Conversation

awaelchli commented Jul 27, 2023 • edited by github-actions bot Loading

What does this PR do?

PR review

github-actions bot commented Jul 27, 2023 • edited Loading

🌩️ Required checks status: Pending 🟡

Groups summary

lantiga left a comment

Choose a reason for hiding this comment

codecov bot commented Jul 27, 2023 • edited Loading

Codecov Report

tchaton left a comment

Choose a reason for hiding this comment

awaelchli commented Jul 27, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Jul 27, 2023 •

edited

Loading

codecov bot commented Jul 27, 2023 •

edited

Loading