Release branch 3.0.0

[Pierre-Sassoulas]

What's new in Pylint 3.0.0?

Release date: 2023-10-02

Breaking Changes

• Enabling or disabling individual messages will now take effect even if an
  --enable=all or disable=all follows in the same configuration file
  (or on the command line).

  This means for the following example, fixme messages will now be emitted:

      pylint my_module --enable=fixme --disable=all

  To regain the prior behavior, remove the superfluous earlier option.

  Closes enable/disable options in rcfile should not depend on the order with which they are specified #3696

• Remove support for launching pylint with Python 3.7.
  Code that supports Python 3.7 can still be linted with the --py-version=3.7 setting.

  Refs Cleanup after we drop support for Python 3.7 #6306

• Disables placed in a try block now apply to the except block.
  Previously, they only happened to do so in the presence of an else clause.

  Refs Use Try node #7767

• pyreverse now uses a new default color palette that is more colorblind friendly.
  The color scheme is taken from Paul Tol's Notes <https://personal.sron.nl/~pault/>_.
  If you prefer other colors, you can use the --color-palette option to specify custom colors.

  Closes Use better default colors for pyreverse #8251

• Everything related to the __implements__ construct was removed. It was based on PEP245
  that was proposed in 2001 and rejected in 2006.

  The capability from pyreverse to take __implements__ into account when generating diagrams
  was also removed.

  Refs [__implements__] Remove everything related to the rejected PEP245 #8404

• pyreverse: Support for the .vcg output format (Visualization of Compiler Graphs) has been dropped.

  Closes Deprecate / remove support for .vcg files in pyreverse #8416

• The warning when the now useless old pylint cache directory (pylint.d) was
  found was removed. The cache dir is documented in
  the FAQ <https://pylint.readthedocs.io/en/latest/faq.html#where-is-the-persistent-data-stored-to-compare-between-successive-runs>_.

  Refs [deprecation] Remove the warning about the old pylint home #8462

• Following a deprecation period, pylint.config.PYLINTRC was removed.
  Use the pylint.config.find_default_config_files generator instead.

  Closes Preferred way to find pylintrc in pylint 3 #8862

Changes requiring user actions

• The invalid-name message no longer checks for a minimum length of 3 characters by default.
  (This was an unadvertised commingling of concerns between casing
  and name length, and users regularly reported this to be surprising.)

  If checking for a minimum length is still desired, it can be regained in two ways:

  • If you are content with a disallowed-name message (instead of
    invalid-name), then simply add the option bad-names-rgxs="^..?$",
    which will fail 1-2 character-long names. (Ensure you enable
    disallowed-name.)

  • If you would prefer an invalid-name message to be emitted, or would
    prefer finer-grained control over the circumstances in which messages are
    emitted (classes vs. methods, etc.), then avail yourself of the regex
    options described
    here <https://pylint.readthedocs.io/en/stable/user_guide/configuration/all-options.html#main-checker>.
    (In particular, take note of the commented out options in the "example
    configuration" given at the bottom of the section.) The prior regexes can
    be found in the
    pull request <https://github.com/pylint-dev/pylint/pull/8813>
    that removed the length requirements.

  Closes Show why snake_case errors is emitted on single word variables (might be too small) #2018

• The compare to empty string checker (pylint.extensions.emptystring) and the compare to
  zero checker (pylint.extensions.compare-to-zero) have been removed and their checks are
  now part of the implicit booleaness checker:

  • compare-to-zero was renamed use-implicit-booleaness-not-comparison-to-zero and
    compare-to-empty-string was renamed use-implicit-booleaness-not-comparison-to-string
    and they now need to be enabled explicitly.

  • The pylint.extensions.emptystring and pylint.extensions.compare-to-zero extensions
    no longer exist and need to be removed from the load-plugins option.

  • Messages related to implicit booleaness were made more explicit and actionable.
    This permits to make their likeness explicit and will provide better performance as they
    share most of their conditions to be raised.

  Closes Uniformize compare-to-zero, compare-to-empty-string with the use-implicit-booleaness-x checks #6871

• epylint was removed. It now lives at: https://github.com/emacsorphanage/pylint.

  Refs Please consider maintaining the Emacs-Lisp files in a separate repository #7737

• The overgeneral-exceptions option now only takes fully qualified names
  into account (builtins.Exception not Exception). If you overrode
  this option, you need to use the fully qualified name now.

  There's still a warning, but it will be removed in 3.1.0.

  Refs [overgeneral-exceptions] Only handle qualified names #8411

• Following a deprecation period, it's no longer possible to use MASTER
  or master as configuration section in setup.cfg or tox.ini. It's bad practice
  to not start a section title with the tool name. Please use pylint.main instead.

  Refs [deprecation] setup.cfgand tox.ini sections need to start by 'pylint' #8465

• Package stats are now printed when running Pyreverse and a --verbose flag was added to get the original output with parsed modules. You might need to activate the verbose option if you want to keep the old output.

  Closes More useful printing for Pyreverse #8973

New Features

• A new json2 reporter has been added. It features a more enriched output that is
  easier to parse and provides more info.

  Compared to json the only changes are that messages are now under the "messages"
  key and that "message-id" now follows the camelCase convention and is renamed to
  "messageId".
  The new reporter also reports the "score" of the modules you linted as defined by the
  evaluation option and provides statistics about the modules you linted.

  We encourage users to use the new reporter as the json reporter will no longer
  be maintained.

  Closes Improve JSON output format with score #4741

• In Pyreverse package dependency diagrams, show when a module imports another only for type-checking.

  Closes Idea: Pyreverse should distinguish between functional imports and type checking imports #8112

• Add new option (--show-stdlib, -L) to pyreverse.
  This is similar to the behavior of --show-builtin in that standard library
  modules are now not included by default, and this option will include them.

  Closes pyreverse: Add option to exclude standard library #8181

• Add Pyreverse option to exclude standalone nodes from diagrams with --no-standalone.

  Closes Pyreverse: Add option to ignore classes without relationships #8476

New Checks

• Added DataclassChecker module and invalid-field-call checker to check for invalid dataclasses.field() usage.

  Refs Add dataclass tests #5159

• Add return-in-finally to emit a message if a return statement was found in a finally clause.

  Closes Prevent usage of return, yield in try/finally #8260

• Add a new checker kwarg-superseded-by-positional-arg to warn when a function is called with a keyword argument which shares a name with a positional-only parameter and the function contains a keyword variadic parameter dictionary. It may be surprising behaviour when the keyword argument is added to the keyword variadic parameter dictionary.

  Closes Lower severity message than redundant-keyword-arg for calling distinct positional-only arg and variadic keyword arg with same name #8558

Extensions

• Add new prefer-typing-namedtuple message to the CodeStyleChecker to suggest
  rewriting calls to collections.namedtuple as classes inheriting from typing.NamedTuple
  on Python 3.6+.

  Requires load-plugins=pylint.extensions.code_style and enable=prefer-typing-namedtuple to be raised.

  Closes Discourage collections.namedtuple in favor of typing.NamedTuple #8660

False Positives Fixed

• Extend concept of "function ambiguity" in safe_infer() from
  differing number of function arguments to differing set of argument names.

  Solves false positives in tensorflow.

  Closes Pylint gets TensorFlow's tf.split all wrong #3613

• Fix unused-argument false positive when __new__ does not use all the arguments of __init__.

  Closes unused-argument for required argument in __new__ #3670

• Fix a false positive for invalid-name when a type-annotated class variable in an enum.Enum class has no assigned value.

  Refs false positive for type-hinted Enum class instance variable #7402

• Fix unused-import false positive for usage of six.with_metaclass.

  Closes False positive of unused-import since v2.15.0 #7506

• Fix false negatives and false positives for too-many-try-statements,
  too-complex, and too-many-branches by correctly counting statements
  under a try.

  Refs Use Try node #7767

• When checking for unbalanced dict unpacking in for-loops, Pylint will now test whether the length of each value to be
  unpacked matches the number of unpacking targets. Previously, Pylint would test the number of values for the loop
  iteration, which would produce a false unbalanced-dict-unpacking warning.

  Closes False reports on W0644 unbalanced-dict-unpacking in pylint 2.16.0 #8156

• Fix false positive for used-before-assignment when usage and assignment
  are guarded by the same test in different statements.

  Closes False positive used-before-assignment when usage and assignment guarded by the same test in different statements #8167

• Adds asyncSetUp to the default defining-attr-methods list to silence
  attribute-defined-outside-init warning when using
  unittest.IsolatedAsyncioTestCase.

  Refs Add Python 3.8+ asyncSetUp to "defining-attr-methods" list #8403

• logging-not-lazy is not longer emitted for explicitly concatenated string arguments.

  Closes Pylint should not emit logging-not-lazy when using explicit string concatenation for the message #8410

• Fix false positive for isinstance-second-argument-not-valid-type when union types contains None.

  Closes isinstance-second-argument-not-valid-type false positive for union types with None #8424

• invalid-name now allows for integers in typealias names:

  • now valid: Good2Name, GoodName2.
  • still invalid: _1BadName.

  Closes invalid-name check for TypeAlias should allow for digits in names #8485

• No longer consider Union as type annotation as type alias for naming checks.

  Closes invalid-name type alias false-positive for union values #8487

• unnecessary-lambda no longer warns on lambdas which use its parameters in
  their body (other than the final arguments), e.g.
  lambda foo: (bar if foo else baz)(foo).

  Closes unnecessary-lambda false positive if parameter is used in lambda body #8496

• Fixed unused-import so that it observes the dummy-variables-rgx option.

  Closes unused-import should observe dummy-variables-rgx setting #8500

• Union typed variables without assignment are no longer treated as
  TypeAlias.

  Closes invalid-name type alias false-positive for union variables without assignment values #8540

• Allow parenthesized implicitly concatenated strings when check-str-concat-over-line-jumps is enabled.

  Closes implicit-str-concat: allow wrapping implicitly concatenated strings in parenthesis. #8552.

• Fix false positive for positional-only-arguments-expected when a function contains both a positional-only parameter that has a default value, and **kwargs.

  Closes False positive for positional-only-arguments-expected #8555

• Fix false positive for keyword-arg-before-vararg when a positional-only parameter with a default value precedes *args.

  Closes False positive for keyword-arg-before-vararg with positional-only parameter #8570

• Fix false positive for arguments-differ when overriding __init_subclass__.

  Closes False positive: arguments-differ for __init_subclass__ in generic #8919

• Fix a false positive for no-value-for-parameter when a staticmethod is called in a class body.

  Closes False positive E1120:no-value-for-parameter #9036

False Negatives Fixed

• Emit used-before-assignment when calling module-level functions before definition.

  Closes Fail to find NameError for undefined functions #1144

• Apply infer_kwarg_from_call() to more checks

  These mostly solve false negatives for various checks,
  save for one false positive for use-maxsplit-arg.

  Closes get_argument_from_call doesn't account for **kwargs #7761

• TypeAlias variables defined in functions are now checked for invalid-name errors.

  Closes TypeAlias defined inside functions doesn't respect the typealias naming style. #8536

• Fix false negative for no-value-for-parameter when a function, whose signature contains both a positional-only parameter name and also *kwargs, is called with a keyword-argument for name.

  Closes False negative no-value-for-parameter when positional-only argument name consumed by **kwargs #8559

• Fix a false negative for too-many-arguments by considering positional-only and keyword-only parameters.

  Closes R0913 excludes keyword- and positional-only arguments #8667

• Emit assignment-from-no-return for calls to builtin methods like dict.update().
  Calls to list.sort() now raise assignment-from-no-return
  rather than assignment-from-none for consistency.

  Closes False negative assigned-from-none for value = dict.update({}, {}) #8714
  Closes Emit assignment-from-no-return instead of assignment-from-none for implicitly returning stdlib calls #8810

• consider-using-augmented-assign is now applied to dicts and lists as well.

  Closes False negative for consider-using-augmented-assign for nested dicts #8959

Other Bug Fixes

• Support duplicate-code message when parallelizing with --jobs.

  Closes Different output with --jobs=1 and --jobs=2 #374

• Support cyclic-import message when parallelizing with --jobs.

  Closes No cyclic-import messages with jobs=0 #4171

• --jobs can now be used with --load-plugins.

  This had regressed in astroid 2.5.0.

  Closes Transform plugin not called when running with multiple jobs #4874

• docparams extension considers type comments as type documentation.

  Closes docparams should support missing type documentation when type comments are used #6287

• When parsing comma-separated lists of regular expressions in the config, ignore
  commas that are inside braces since those indicate quantifiers, not delineation
  between expressions.

  Closes bad-names-rgxs mangles regular expressions with commas #7229

• The ignored-modules option will now be correctly taken into account for no-name-in-module.

  Closes no-name-in-module does not respect patterns in ignored-modules #7578

• sys.argv is now always correctly considered as impossible to infer (instead of
  using the actual values given to pylint).

  Closes sys.argv is inferred wrongly as pylint's own values for sys.argv instead of Uninferrable #7710

• Avoid duplicative warnings for unqualified exception names in the overgeneral-exceptions
  setting when running with --jobs.

  Closes With --jobs 2, the open() call is executed for each opened file #7774

• Don't show class fields more than once in Pyreverse diagrams.

  Closes pyreverse should ignore class level attribute declarations #8189

• Fix used-before-assignment false negative when TYPE_CHECKING imports
  are used in multiple scopes.

  Closes used-before-assignment false negative with TYPE_CHECKING #8198

• --clear-cache-post-run now also clears LRU caches for pylint utilities
  holding references to AST nodes.

  Closes Clear pylint's LRU caches when using --clear-cache-post-run #8361

• Fix a crash when TYPE_CHECKING is used without importing it.

  Closes in_type_checking_block can crash if you haven't imported TYPE_CHECKING #8434

• Fix a used-before-assignment false positive when imports
  are made under the TYPE_CHECKING else if branch.

  Closes used-before-assignment false positive for import nodes under elif after TYPE_CHECKING #8437

• Fix a regression of preferred-modules where a partial match was used instead of the required full match.

  Closes v2.17.0 regression related to W0407: preferred-modules #8453

• Fix a crash in pyreverse when "/" characters are used in the output filename e.g pyreverse -o png -p name/ path/to/project.

  Closes [pyreverse] Setting the project name with a "/" at the end crashes pyreverse #8504

• Don't show arrows more than once in Pyreverse diagrams.

  Closes Pyreverse duplicate arrows #8522

• Improve output of consider-using-generator message for min() calls with default keyword.

  Closes consider-using-generator error message ignores default arguments #8563

• Fixed a crash when generating a configuration file: tomlkit.exceptions.TOMLKitError: Can't add a table to a dotted key
  caused by tomlkit v0.11.8.

  Closes TOMLKitError when generating TOML config with tomlkit 0.11.8 #8632

• Fix a line break error in Pyreverse dot output.

  Closes type hints in methods using PEP 604 result in horizontal line #8671

• Fix a false positive for method-hidden when using cached_property decorator.

  Closes False Positive: method-hidden with @functools.cached_property #8753

• Dunder methods defined in lambda do not trigger unnecessary-dunder-call anymore, if they cannot be replaced by the non-dunder call.

  Closes C2801 False positive when __setitem__ appears in a lambda #8769

• Don't show duplicate type annotations in Pyreverse diagrams.

  Closes Incorrect type reported by pyreverse for unions and generics #8888

• Fixing inconsistent hashing issue in BaseChecker causing some reports not being exported.

  Closes Dependency graphs are not exported to files #9001

• Don't add Optional to | annotations with None in Pyreverse diagrams.

  Closes Redundant Optional in Pyreverse #9014

• Pyreverse doesn't show multiple class association arrows anymore, but only the strongest one.

  Refs Pyreverse: composition / aggregation arrow strange behavior (and field annotation bug) #9045

• Prevented data loss in the linter stats for messages relating
  to the linter itself (e.g. unknown-option-value), fixing
  problems with score, fail-on, etc.

  Closes Score, fail-on, fail-under, etc ignores messages emitted by PyLinter itself #9059

• Fix crash in refactoring checker when unary operand used with variable in for loop.

  Closes Refactoring checker crashes when unary operator used with variable #9074

Other Changes

• Pylint now exposes its type annotations.

  Closes Expose types #5488 and Annotate pylint code with type annotations #2079

• Search for pyproject.toml recursively in parent directories up to a project or file system root.

  Refs Search for pyproject.toml config file in parent dirs #7163, Closes Looking for configuration files like pylintrc or pyproject.toml in parent directories like black or isort #3289

• All code related to the optparse config parsing has been removed.

  Refs Remove all old code related to optparse config parsing. #8405

• Pylint now supports python 3.12.

  Refs [ci] Add jobs for python 3.12-beta following its release #8718

• Add a CITATION.cff file to the root of the repository containing the necessary metadata to cite Pylint.

  Closes Add CITATION.cff #8760

• Renamed the "unneeded-not" error into "unnecessary_negation" to be clearer.

  Closes Refactor unneeded-not checker #8789

Internal Changes

• get_message_definition was removed from the base checker API. You can access
  message definitions through the MessageStore.

  Refs [deprecation] Remove get_message_definitions from BaseChecker #8401

• Everything related to the __implements__ construct was removed. It was based on PEP245
  that was proposed in 2001 and rejected in 2006.

  All the classes inheriting Interface in pylint.interfaces were removed.
  Checker should only inherit BaseChecker or any of the other checker types
  from pylint.checkers. Reporter should only inherit BaseReporter.

  Refs [__implements__] Remove everything related to the rejected PEP245 #8404

• modname and msg_store are now required to be given in FileState.
  collect_block_lines has also been removed. Pylinter.current_name
  cannot be null anymore.

  Refs [FileState] Deprecation enactment for 3.0.0 #8407

• Reporter.set_output was removed in favor of reporter.out = stream.

  Refs [Reporter] Remove 'set_output' in favor of reporter.out #8408

• A number of old utility functions and classes have been removed:

  MapReduceMixin: To make a checker reduce map data simply implement
  get_map_data and reduce_map_data.

  is_inside_lambda: Use utils.get_node_first_ancestor_of_type(x, nodes.Lambda)

  check_messages: Use utils.only_required_for_messages

  is_class_subscriptable_pep585_with_postponed_evaluation_enabled: Use
  is_postponed_evaluation_enabled(node) and is_node_in_type_annotation_context(node)

  get_python_path: assumption that there's always an init.py is not true since
  python 3.3 and is causing problems, particularly with PEP 420. Use discover_package_path
  and pass source root(s).

  fix_import_path: Use augmented_sys_path and pass additional sys.path
  entries as an argument obtained from discover_package_path.

  get_global_option: Use checker.linter.config to get all global options.

  Related private objects have been removed as well.

  Refs Remove deprecated functions and classes #8409

• colorize_ansi now only accepts a MessageStyle object.

  Refs [colorize_ansi] Remove the possibility to use anything else than a MessageStyle #8412

• Following a deprecation period, Pylinter.check now only works with sequences of strings, not strings.

  Refs [deprecation] 'Pylinter.check' now takes sequence of str only #8463

• Following a deprecation period, ColorizedTextReporter only accepts ColorMappingDict.

  Refs [deprecation] 'ColorizedTextReporter' only accepts 'ColorMappingDict' #8464

• Following a deprecation period, MessageTest's end_line and end_col_offset
  must be accurate in functional tests (for python 3.8 or above on cpython, and for
  python 3.9 or superior on pypy).

  Refs [deprecation] end_line and end_col_offset required in functional tests #8466

• Following a deprecation period, the do_exit argument of the Run class (and of the _Run
  class in testutils) were removed.

  Refs [deprecation] Remove 'do_exit' from the 'Run' constructor #8472

• Following a deprecation period, the py_version argument of the
  MessageDefinition.may_be_emitted function is now required. The most likely solution
  is to use 'linter.config.py_version' if you need to keep using this
  function, or to use 'MessageDefinition.is_message_enabled' instead.

  Refs [deprecation] Make 'py_version' required for 'MessageDefinition.may_be_emitted' #8473

• Following a deprecation period, the OutputLine class now requires
  the right number of argument all the time. The functional output can be
  regenerated automatically to achieve that easily.

  Refs [deprecation] Make 'OutputLine' require a fixed number of args #8474

• Following a deprecation period, is_typing_guard, is_node_in_typing_guarded_import_block and
  is_node_in_guarded_import_block from pylint.utils were removed: use a combination of
  is_sys_guard and in_type_checking_block instead.

  Refs [deprecation] Remove all duplicated typing guard check functions #8475

• Following a deprecation period, the location argument of the
  Message class must now be a MessageLocationTuple.

  Refs [deprecation] Message.location must be a 'MessageLocationTuple' #8477

• Following a deprecation period, the check_single_file function of the
  Pylinter is replaced by Pylinter.check_single_file_item.

  Refs [deprecation] 'check_single_file' replaced by 'check_single_file_item' #8478

Performance Improvements

• pylint runs (at least) ~5% faster after improvements to astroid
  that make better use of the inference cache.

  Refs Improving inference caching (unnecessary context clones?) astroid#529

• • Optimize is_trailing_comma().
  • Cache class_is_abstract().

  Refs Performance Benchmarks / Integration tests #1954

• Exit immediately if all messages are disabled.

  Closes Short circuit if all checks disabled #8715

Closes #7607

[Pierre-Sassoulas]

Blocked by #9092

[Pierre-Sassoulas]

Maybe not blocked, the github CI are not ready yet.

[codecov]

Codecov Report

Merging #9093 (c61edea) into main (3cba15b) will not change coverage.
The diff coverage is 100.00%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #9093   +/-   ##
=======================================
  Coverage   95.75%   95.75%           
=======================================
  Files         173      173           
  Lines       18663    18663           
=======================================
  Hits        17871    17871           
  Misses        792      792           

Files	Coverage Δ
pylint/__pkginfo__.py	100.00% <100.00%> (ø)

[Pierre-Sassoulas]

pyreverse also got a lot of love but hard to pinpoint a particular issues that was super important compared to the others.

[cdce8p]

Blocked by #9092

Maybe not blocked, the github CI are not ready yet.

It'll probably take a day or two until 3.12.0 is available for Github CI runs. You can follow the releases here https://github.com/actions/python-versions

[github-actions]

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit c61edea