Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ignoring a file with a single # type: ignore comment. #6830

Merged
merged 7 commits into from
May 16, 2019
Merged

Ignoring a file with a single # type: ignore comment. #6830

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
7a51cf2
Stop checking at bare top-level "# type: ignore" comments.
brandtbucher May 14, 2019
aa11089
Test new "# type: ignore" behavior.
brandtbucher May 14, 2019
ce54ab4
Appease mypy.
brandtbucher May 14, 2019
4567661
Add docs for new # type: ignore usage.
brandtbucher May 14, 2019
c95b98d
Code cleanup from GvR's first review.
brandtbucher May 14, 2019
07cc315
Break out a new section for this feature.
brandtbucher May 15, 2019
9937b1b
Back out Python 3.8 feature.
brandtbucher May 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 52 additions & 14 deletions mypy/fastparse.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import sys

from typing import (
Tuple, Union, TypeVar, Callable, Sequence, Optional, Any, Dict, cast, List, overload
Tuple, Union, TypeVar, Callable, Sequence, Optional, Any, Dict, cast, List, overload, Set
)
MYPY = False
if MYPY:
Expand Down Expand Up @@ -258,7 +258,7 @@ def __init__(self,
self.is_stub = is_stub
self.errors = errors

self.extra_type_ignores = [] # type: List[int]
self.type_ignores = set() # type: Set[int]

# Cache of visit_X methods keyed by type of visited object
self.visitor_cache = {} # type: Dict[type, Callable[[Optional[AST]], Any]]
Expand Down Expand Up @@ -294,11 +294,50 @@ def translate_expr_list(self, l: Sequence[AST]) -> List[Expression]:
res.append(exp)
return res

def translate_stmt_list(self, l: Sequence[AST]) -> List[Statement]:
def get_line(self, n: ast3.stmt) -> int:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rename to get_lineno? Without context, the name get_line vaguely makes me think that it would get the source code line somehow.

if (isinstance(n, (ast3.AsyncFunctionDef, ast3.ClassDef, ast3.FunctionDef))
and n.decorator_list):
return n.decorator_list[0].lineno
return n.lineno

def translate_stmt_list(self,
l: Sequence[ast3.stmt],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please choose a longer name for this argument, maybe stmts? (I know the original code uses one-letter names, but I don't want to encourage the practice for new code, and you've added a lot here.)

module: bool = False) -> List[Statement]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps rename to ismodule?


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd leave this blank line out, as well a the one below this block comment. (Ditto for other block comments below; in general you sprinkle more whitespace throughout the code than is mypy's tradition.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed a lot of my extra whitespace throughout.

# A "# type: ignore" comment before the first statement of a module
# ignores the whole module:

if module and l and self.type_ignores and min(self.type_ignores) < self.get_line(l[0]):
self.errors.used_ignored_lines[self.errors.file].add(min(self.type_ignores))
b = Block(self.fix_function_overloads(self.translate_stmt_list(l)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rename to block?

b.is_unreachable = True
return [b]

res = [] # type: List[Statement]
for e in l:
line = 0

for i, e in enumerate(l):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, I'd propose a longer name for e, e.g. stmt. (But, i for the index is traditional, so please keep that.)


if module: # and...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow it looks inefficient to check for module and sys.version_info each time through the loop. Also the comment here adds nothing. You did know and short-circuits, right? You can combine the two checks and save one level of indentation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was originally if module and sys.version_info >= (3, 8):, but the mypy self check complained when I accessed e.end_lineno on 336. As I understand it, the mypy's version checks don't work for compound tests like this? Splitting it up silences the error, and I believe the emitted bytecode is identical.

If it would be clearer, I can just # type: ignore line 336 instead!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, then the two if statements is optimal -- just put in a clearer comment explaining that this is to work around a mypy issue.

if sys.version_info >= (3, 8):

# In Python 3.8, a "# type: ignore" comment between statements at
# the top level of a module skips checking for everything else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most importantly, explain why: this check needs access to end_lineno which only exists in 3.8 and up.


ignores = set(range(line + 1, self.get_line(e))) & self.type_ignores

if ignores:
self.errors.used_ignored_lines[self.errors.file].add(min(ignores))
b = Block(self.fix_function_overloads(self.translate_stmt_list(l[i:])))
b.is_unreachable = True
res.append(b)
return res

line = e.end_lineno if e.end_lineno is not None else e.lineno

stmt = self.visit(e)
res.append(stmt)

return res

op_map = {
Expand Down Expand Up @@ -403,13 +442,12 @@ def translate_module_id(self, id: str) -> str:
return id

def visit_Module(self, mod: ast3.Module) -> MypyFile:
body = self.fix_function_overloads(self.translate_stmt_list(mod.body))
ignores = [ti.lineno for ti in mod.type_ignores]
ignores.extend(self.extra_type_ignores)
self.type_ignores = {ti.lineno for ti in mod.type_ignores}
body = self.fix_function_overloads(self.translate_stmt_list(mod.body, module=True))
return MypyFile(body,
self.imports,
False,
set(ignores),
set(self.type_ignores),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add a comment explaining why it's copied here. (If there is a good reason!)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this isn't needed. Removed.

)

# --- stmt ---
Expand Down Expand Up @@ -615,7 +653,7 @@ def make_argument(self, arg: ast3.arg, default: Optional[ast3.expr], kind: int,
elif type_comment is not None:
extra_ignore, arg_type = parse_type_comment(type_comment, arg.lineno, self.errors)
if extra_ignore:
self.extra_type_ignores.append(arg.lineno)
self.type_ignores.add(arg.lineno)

return Argument(Var(arg.arg), arg_type, self.visit(default), kind)

Expand Down Expand Up @@ -673,7 +711,7 @@ def visit_Assign(self, n: ast3.Assign) -> AssignmentStmt:
if n.type_comment is not None:
extra_ignore, typ = parse_type_comment(n.type_comment, n.lineno, self.errors)
if extra_ignore:
self.extra_type_ignores.append(n.lineno)
self.type_ignores.add(n.lineno)
else:
typ = None
s = AssignmentStmt(lvalues, rvalue, type=typ, new_syntax=False)
Expand Down Expand Up @@ -707,7 +745,7 @@ def visit_For(self, n: ast3.For) -> ForStmt:
if n.type_comment is not None:
extra_ignore, target_type = parse_type_comment(n.type_comment, n.lineno, self.errors)
if extra_ignore:
self.extra_type_ignores.append(n.lineno)
self.type_ignores.add(n.lineno)
else:
target_type = None
node = ForStmt(self.visit(n.target),
Expand All @@ -722,7 +760,7 @@ def visit_AsyncFor(self, n: ast3.AsyncFor) -> ForStmt:
if n.type_comment is not None:
extra_ignore, target_type = parse_type_comment(n.type_comment, n.lineno, self.errors)
if extra_ignore:
self.extra_type_ignores.append(n.lineno)
self.type_ignores.add(n.lineno)
else:
target_type = None
node = ForStmt(self.visit(n.target),
Expand Down Expand Up @@ -753,7 +791,7 @@ def visit_With(self, n: ast3.With) -> WithStmt:
if n.type_comment is not None:
extra_ignore, target_type = parse_type_comment(n.type_comment, n.lineno, self.errors)
if extra_ignore:
self.extra_type_ignores.append(n.lineno)
self.type_ignores.add(n.lineno)
else:
target_type = None
node = WithStmt([self.visit(i.context_expr) for i in n.items],
Expand All @@ -767,7 +805,7 @@ def visit_AsyncWith(self, n: ast3.AsyncWith) -> WithStmt:
if n.type_comment is not None:
extra_ignore, target_type = parse_type_comment(n.type_comment, n.lineno, self.errors)
if extra_ignore:
self.extra_type_ignores.append(n.lineno)
self.type_ignores.add(n.lineno)
else:
target_type = None
s = WithStmt([self.visit(i.context_expr) for i in n.items],
Expand Down
34 changes: 25 additions & 9 deletions mypy/fastparse2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"""
import sys

from typing import Tuple, Union, TypeVar, Callable, Sequence, Optional, Any, Dict, cast, List
from typing import Tuple, Union, TypeVar, Callable, Sequence, Optional, Any, Dict, cast, List, Set
MYPY = False
if MYPY:
import typing # for typing.Type, which conflicts with types.Type
Expand Down Expand Up @@ -163,7 +163,7 @@ def __init__(self,
# Cache of visit_X methods keyed by type of visited object
self.visitor_cache = {} # type: Dict[type, Callable[[Optional[AST]], Any]]

self.extra_type_ignores = [] # type: List[int]
self.type_ignores = set() # type: Set[int]

def fail(self, msg: str, line: int, column: int, blocker: bool = True) -> None:
if blocker or not self.options.ignore_errors:
Expand Down Expand Up @@ -193,7 +193,24 @@ def translate_expr_list(self, l: Sequence[AST]) -> List[Expression]:
res.append(exp)
return res

def translate_stmt_list(self, l: Sequence[AST]) -> List[Statement]:
def get_line(self, n: ast27.stmt) -> int:
if isinstance(n, (ast27.ClassDef, ast27.FunctionDef)) and n.decorator_list:
return n.decorator_list[0].lineno
return n.lineno

def translate_stmt_list(self,
l: Sequence[ast27.stmt],
module: bool = False) -> List[Statement]:

# A "# type: ignore" comment before the first statement of a module
# ignores the whole module:

if module and l and self.type_ignores and min(self.type_ignores) < self.get_line(l[0]):
self.errors.used_ignored_lines[self.errors.file].add(min(self.type_ignores))
b = Block(self.fix_function_overloads(self.translate_stmt_list(l)))
b.is_unreachable = True
return [b]

res = [] # type: List[Statement]
for e in l:
stmt = self.visit(e)
Expand Down Expand Up @@ -304,13 +321,12 @@ def translate_module_id(self, id: str) -> str:
return id

def visit_Module(self, mod: ast27.Module) -> MypyFile:
self.type_ignores = {ti.lineno for ti in mod.type_ignores}
body = self.fix_function_overloads(self.translate_stmt_list(mod.body))
ignores = [ti.lineno for ti in mod.type_ignores]
ignores.extend(self.extra_type_ignores)
return MypyFile(body,
self.imports,
False,
set(ignores),
set(self.type_ignores),
)

# --- stmt ---
Expand Down Expand Up @@ -558,7 +574,7 @@ def visit_Assign(self, n: ast27.Assign) -> AssignmentStmt:
extra_ignore, typ = parse_type_comment(n.type_comment, n.lineno, self.errors,
assume_str_is_unicode=self.unicode_literals)
if extra_ignore:
self.extra_type_ignores.append(n.lineno)
self.type_ignores.add(n.lineno)

stmt = AssignmentStmt(self.translate_expr_list(n.targets),
self.visit(n.value),
Expand All @@ -578,7 +594,7 @@ def visit_For(self, n: ast27.For) -> ForStmt:
extra_ignore, typ = parse_type_comment(n.type_comment, n.lineno, self.errors,
assume_str_is_unicode=self.unicode_literals)
if extra_ignore:
self.extra_type_ignores.append(n.lineno)
self.type_ignores.add(n.lineno)
else:
typ = None
stmt = ForStmt(self.visit(n.target),
Expand Down Expand Up @@ -608,7 +624,7 @@ def visit_With(self, n: ast27.With) -> WithStmt:
extra_ignore, typ = parse_type_comment(n.type_comment, n.lineno, self.errors,
assume_str_is_unicode=self.unicode_literals)
if extra_ignore:
self.extra_type_ignores.append(n.lineno)
self.type_ignores.add(n.lineno)
else:
typ = None
stmt = WithStmt([self.visit(n.context_expr)],
Expand Down
5 changes: 5 additions & 0 deletions test-data/unit/check-38.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,3 +106,8 @@ def g(x: int): ...
/
0 # type: ignore
) # type: ignore # E: unused 'type: ignore' comment

[case testIgnoreRestOfModule]
ERROR # E: Name 'ERROR' is not defined
# type: ignore
IGNORE
40 changes: 40 additions & 0 deletions test-data/unit/check-ignore.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,3 +218,43 @@ def f() -> None: pass

[case testCannotIgnoreBlockingError]
yield # type: ignore # E: 'yield' outside function

[case testIgnoreWholeModule1]
# flags: --warn-unused-ignores
# type: ignore
IGNORE # type: ignore # E: unused 'type: ignore' comment

[case testIgnoreWholeModule2]
# type: ignore
if True:
IGNORE

[case testIgnoreWholeModule3]
# type: ignore
@d
class C: ...
IGNORE

[case testIgnoreWholeModule4]
# type: ignore
@d

def f(): ...
IGNORE

[case testDontIgnoreWholeModule1]
if True:
# type: ignore
ERROR # E: Name 'ERROR' is not defined
ERROR # E: Name 'ERROR' is not defined

[case testDontIgnoreWholeModule2]
@d # type: ignore
class C: ...
ERROR # E: Name 'ERROR' is not defined

[case testDontIgnoreWholeModule3]
@d # type: ignore

def f(): ...
ERROR # E: Name 'ERROR' is not defined