Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Adopt PEP 563, 585, and 604 for src/lightning/pytorch #17779

Closed
wants to merge 4 commits into from

Conversation

mauvilsa
Copy link
Contributor

@mauvilsa mauvilsa commented Jun 7, 2023

What does this PR do?

Try to pyupgrade src/lightning/pytorch to adopt PEP 563, 585, and 604. Ran the following and then manually fixed src/lightning/pytorch/loggers/neptune.py

isort -a "from __future__ import annotations" --append-only src/lightning/pytorch
pyupgrade --py37-plus $(find src/lightning/pytorch -name '*.py')

Part of #11205

Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Jun 7, 2023
@@ -11,9 +11,12 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from lightning.pytorch.loops.loop import _Loop # noqa: F401 isort: skip (avoids circular imports)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this is what is causing the circular import issues. I assume one of the tools did it accidentally? isort should be skipping it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran with isort version 5.10.1. Maybe that was the issue.

@@ -1,5 +1,7 @@
"""Root package info."""

from __future__ import annotations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd need to remove all these unused additions. Maybe we can write a script that processes the git diff and undoes it if this is the only change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for f in $(git diff --name-only master..HEAD); do
  if [ "$(git diff master..HEAD $f | sed -n '/^+++/d; /^+$/d; /^+/p')" = "+from __future__ import annotations" ]; then
    git checkout master -- $f
  fi
done

@mauvilsa
Copy link
Contributor Author

mauvilsa commented Jun 8, 2023

For a draft pull request, the test_cli.py tests are run? I couldn't find them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the doctests there is one failure, but not jsonargparse related. It fails for torch.jit.script.

https://github.com/mauvilsa/lightning/blob/e305cd8f7035b7c07062ce04a51448465c5648b4/src/lightning/pytorch/core/module.py#L1394

Copy link
Contributor Author

@mauvilsa mauvilsa Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a deal breaker since pytorch is central to lightning. The postponed annotations work correctly with jsonargparse 4.22.0. There are some unit tests in test_cli.py that fail but it is because of mocking. For now I think I will just close this pull request.

@@ -44,7 +46,7 @@ def clean_namespace(hparams: MutableMapping) -> None:
del hparams[k]


def parse_class_init_keys(cls: Type) -> Tuple[str, Optional[str], Optional[str]]:
def parse_class_init_keys(cls: type) -> tuple[str, str | None, str | None]:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy does not consider Type and type equivalent?

@carmocca carmocca changed the title Adopt PEP 563, 585, and 604 for src/lightning/pytorch [WIP] Adopt PEP 563, 585, and 604 for src/lightning/pytorch Jun 8, 2023
@carmocca carmocca marked this pull request as ready for review June 8, 2023 06:08
@carmocca
Copy link
Contributor

carmocca commented Jun 8, 2023

For a draft pull request, the test_cli.py tests are run? I couldn't find them.

They don't. I marked the pull request as ready and added WIP to the title.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
has conflicts pl Generic label for PyTorch Lightning package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants