Merge pull request #3869 from manderj/pytest_pylint_parallel_runs_warnings

lint multiprocessing pool shutdown

Author: hippo91

CONTRIBUTORS.txt

@@ -425,3 +425,5 @@ contributors:
 * Giuseppe Valente: contributor
 
 * Takashi Hirashima: contributor
+
+* Joffrey Mander: contributor

ChangeLog

@@ -15,6 +15,10 @@ What's New in Pylint 2.6.1?
 ===========================
 Release date: TBA
 
+* Fix linter multiprocessing pool shutdown (triggered warnings when runned in parallels with other pytest plugins)
+
+  Closes #3779
+
 * Fix a false-positive emission of `no-self-use` and `unused-argument` for methods
   of generic structural types (`Protocol[T]`)
 
@@ -23,7 +27,7 @@ Release date: TBA
 * Fix bug that lead to duplicate messages when using ``--jobs 2`` or more.
 
   Close #3584
- 
+
 * Adds option ``check-protected-access-in-special-methods`` in the ClassChecker to activate/deactivate
   ``protected-access`` message emission for single underscore prefixed attribute in special methods.
 

doc/whatsnew/2.6.rst

@@ -29,6 +29,8 @@ New checkers
 Other Changes
 =============
 
+* Fix linter multiprocessing pool shutdown which triggered warnings when runned in parallels with other pytest plugins.
+
 * Enums are now required to be named in UPPER_CASE by ``invalid-name``.
 
 * Fix bug that lead to duplicate messages when using ``--jobs 2`` or more.

pylint/lint/parallel.py

@@ -91,15 +91,16 @@ def check_parallel(linter, jobs, files, arguments=None):
     # is identical to the linter object here. This is requred so that
     # a custom PyLinter object can be used.
     initializer = functools.partial(_worker_initialize, arguments=arguments)
-    with multiprocessing.Pool(jobs, initializer=initializer, initargs=[linter]) as pool:
-        # ..and now when the workers have inherited the linter, the actual reporter
-        # can be set back here on the parent process so that results get stored into
-        # correct reporter
-        linter.set_reporter(original_reporter)
-        linter.open()
+    pool = multiprocessing.Pool(jobs, initializer=initializer, initargs=[linter])
+    # ..and now when the workers have inherited the linter, the actual reporter
+    # can be set back here on the parent process so that results get stored into
+    # correct reporter
+    linter.set_reporter(original_reporter)
+    linter.open()
 
-        all_stats = []
+    all_stats = []
 
+    try:
         for (
             module,
             file_path,
@@ -116,6 +117,9 @@ def check_parallel(linter, jobs, files, arguments=None):
 
             all_stats.append(stats)
             linter.msg_status |= msg_status
+    finally:
+        pool.close()
+        pool.join()
 
     linter.stats = _merge_stats(all_stats)