Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added py.typed to the package #4154

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Added py.typed to the package #4154

wants to merge 2 commits into from

Conversation

vfdev-5
Copy link
Collaborator

@vfdev-5 vfdev-5 commented Jul 4, 2021

Description:

Adding this will enable to remove mypy errors on user code with torchvision:

import torch
import torchvision.transforms as T
torch.manual_seed(12)

tensor_image = torch.rand(3, 32, 32)
transforms = T.Compose([
    T.RandomCrop(224),
    T.RandomHorizontalFlip(p=0.3),
    T.ConvertImageDtype(torch.float),
    T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
out_image = transforms(tensor_image)

and mypy check_vision.py

- check_vision.py:2: error: Skipping analyzing "torchvision.transforms": found module but no type hints or library stubs
- check_vision.py:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
- check_vision.py:2: error: Skipping analyzing "torchvision": found module but no type hints or library stubs
- Found 2 errors in 1 file (checked 1 source file)
+ Success: no issues found in 1 source file

@pmeier
Copy link
Collaborator

pmeier commented Jul 5, 2021

@vfdev-5 In general, that was the plan. But looking at mypy.ini

vision/mypy.ini

Lines 7 to 33 in ef71159

[mypy-torchvision.io._video_opt.*]
ignore_errors = True
[mypy-torchvision.io.*]
ignore_errors = True
[mypy-torchvision.models.densenet.*]
ignore_errors=True
[mypy-torchvision.models.detection.*]
ignore_errors = True
[mypy-torchvision.models.quantization.*]
ignore_errors = True
[mypy-torchvision.ops.*]
ignore_errors = True
[mypy-torchvision.transforms.*]
ignore_errors = True

we have quite a few holes in our own type checking. IMO we should get annotations working in torchvision.io, .ops, and .transforms before we make them public.

@NicolasHug
Copy link
Member

I wonder if this py.typed file is necessary at all, I can't find it for e.g. pandas, and they introduced types a while ago.

Is mypy complaining the same for modules that are typed, e.g. the io module?
If it's not complaning, then we probably don't need to add the file?

@vfdev-5
Copy link
Collaborator Author

vfdev-5 commented Jul 5, 2021

@NicolasHug there are still discussions on this pep 561 compatibility in pandas:

According to the docs: https://mypy.readthedocs.io/en/stable/installed_packages.html#creating-pep-561-compatible-packages there are two ways of doing things:

If you want to create a stub-only package for an existing library, the simplest way is to contribute stubs to the typeshed repository, and a stub package will automatically be uploaded to PyPI.

If you would like to publish a library package to a package repository yourself (e.g. on PyPI) for either internal or external use in type checking, packages that supply type information via type comments or annotations in the code should put a py.typed file in their package directory.

See numpy as example :

@pmeier
Copy link
Collaborator

pmeier commented Jul 5, 2021

I wonder if this py.typed file is necessary at all, I can't find it for e.g. pandas, and they introduced types a while ago.

Is mypy complaining the same for modules that are typed, e.g. the io module?
If it's not complaning, then we probably don't need to add the file?

To add a little context: you only need the py.typed file if the package should be PEP561 compliant. If it is not present, mypy will ignore all annotations and assumes the package is completely untyped. If we only want to have the checks for ourselves we can ignore it. But since more and more projects are adopting static type checking, IMO it would be a good idea to add annotations to the remaining modules and add the file. PyTorch core does:

https://github.com/pytorch/pytorch/blob/master/torch/py.typed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants