Skip to content

test_concurrent_futures: test_deadlock failed with timeout on ARM Raspbian 3.x #109702

New issue
New issue
Closed
Closed
test_concurrent_futures: test_deadlock failed with timeout on ARM Raspbian 3.x#109702
Labels
testsTests in the Lib/test dir
@vstinner

Description

@vstinner
vstinner
opened on Sep 22, 2023

Example:

FAIL: test_crash_during_func_exec_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolForkserverExecutorDeadlockTest.test_crash_during_func_exec_on_worker)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 128, in _check_crash
    res.result(timeout=self.TIMEOUT)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

build: https://buildbot.python.org/all/#/builders/424/builds/4978

  • FAIL: test_crash_during_func_exec_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolForkserverExecutorDeadlockTest.test_crash_during_func_exec_on_worker)
  • FAIL: test_crash_during_result_pickle_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolForkserverExecutorDeadlockTest.test_crash_during_result_pickle_on_worker)
  • FAIL: test_crash_at_task_unpickle (test.test_concurrent_futures.test_deadlock.ProcessPoolSpawnExecutorDeadlockTest.test_crash_at_task_unpickle)
  • FAIL: test_crash_during_func_exec_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolSpawnExecutorDeadlockTest.test_crash_during_func_exec_on_worker)
  • FAIL: test_crash_during_result_pickle_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolSpawnExecutorDeadlockTest.test_crash_during_result_pickle_on_worker)

These tests passed when re-run in verbose mode in a fresh process.

It's just a timeout issue because the machine was too busy.

Full logs:

======================================================================
FAIL: test_crash_during_func_exec_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolForkserverExecutorDeadlockTest.test_crash_during_func_exec_on_worker)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 128, in _check_crash
    res.result(timeout=self.TIMEOUT)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 154, in test_crash_during_func_exec_on_worker
    self._check_crash(BrokenProcessPool, _crash)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132, in _check_crash
    self._fail_on_deadlock(executor)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 109, in _fail_on_deadlock
    self.fail(f"Executor deadlock:\n\n{tb}")
AssertionError: Executor deadlock:

Thread 0xf5deb440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 341 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/queues.py", line 231 in _feed
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 996 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Thread 0xf53ff440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/selectors.py", line 398 in select
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 1136 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 414 in wait_result_broken_or_wakeup
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 341 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Current thread 0xf79af040 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 100 in _fail_on_deadlock
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132 in _check_crash
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 154 in test_crash_during_func_exec_on_worker
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 634 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 690 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/runner.py", line 240 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1152 in _run_suite
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1279 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 36 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 92 in test_func
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 48 in regrtest_runner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 95 in _load_run_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 138 in _runtest_env_changed_exc
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 223 in _runtest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 266 in run_single_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 82 in worker_process
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 105 in main
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 109 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main


======================================================================
FAIL: test_crash_during_result_pickle_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolForkserverExecutorDeadlockTest.test_crash_during_result_pickle_on_worker)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 128, in _check_crash
    res.result(timeout=self.TIMEOUT)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 167, in test_crash_during_result_pickle_on_worker
    self._check_crash(BrokenProcessPool, _return_instance, CrashAtPickle)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132, in _check_crash
    self._fail_on_deadlock(executor)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 109, in _fail_on_deadlock
    self.fail(f"Executor deadlock:\n\n{tb}")
AssertionError: Executor deadlock:

Thread 0xf5deb440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 341 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/queues.py", line 231 in _feed
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 996 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Thread 0xf53ff440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/selectors.py", line 398 in select
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 1136 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 414 in wait_result_broken_or_wakeup
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 341 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Current thread 0xf79af040 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 100 in _fail_on_deadlock
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132 in _check_crash
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 167 in test_crash_during_result_pickle_on_worker
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 634 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 690 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/runner.py", line 240 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1152 in _run_suite
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1279 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 36 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 92 in test_func
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 48 in regrtest_runner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 95 in _load_run_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 138 in _runtest_env_changed_exc
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 223 in _runtest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 266 in run_single_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 82 in worker_process
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 105 in main
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 109 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main


======================================================================
FAIL: test_crash_at_task_unpickle (test.test_concurrent_futures.test_deadlock.ProcessPoolSpawnExecutorDeadlockTest.test_crash_at_task_unpickle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 128, in _check_crash
    res.result(timeout=self.TIMEOUT)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 150, in test_crash_at_task_unpickle
    self._check_crash(BrokenProcessPool, id, CrashAtUnpickle())
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132, in _check_crash
    self._fail_on_deadlock(executor)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 109, in _fail_on_deadlock
    self.fail(f"Executor deadlock:\n\n{tb}")
AssertionError: Executor deadlock:

Thread 0xf5deb440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 341 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/queues.py", line 231 in _feed
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 996 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Thread 0xf53ff440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/selectors.py", line 398 in select
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 1136 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 414 in wait_result_broken_or_wakeup
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 341 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Current thread 0xf79af040 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 100 in _fail_on_deadlock
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132 in _check_crash
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 150 in test_crash_at_task_unpickle
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 634 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 690 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/runner.py", line 240 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1152 in _run_suite
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1279 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 36 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 92 in test_func
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 48 in regrtest_runner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 95 in _load_run_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 138 in _runtest_env_changed_exc
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 223 in _runtest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 266 in run_single_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 82 in worker_process
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 105 in main
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 109 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main


Stderr:
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/queues.py", line 250, in _feed
    send_bytes(obj)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 185, in send_bytes
    self._check_closed()
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 138, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

======================================================================
FAIL: test_crash_during_func_exec_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolSpawnExecutorDeadlockTest.test_crash_during_func_exec_on_worker)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 128, in _check_crash
    res.result(timeout=self.TIMEOUT)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 154, in test_crash_during_func_exec_on_worker
    self._check_crash(BrokenProcessPool, _crash)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132, in _check_crash
    self._fail_on_deadlock(executor)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 109, in _fail_on_deadlock
    self.fail(f"Executor deadlock:\n\n{tb}")
AssertionError: Executor deadlock:

Thread 0xf5deb440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 341 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/queues.py", line 231 in _feed
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 996 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Thread 0xf53ff440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/selectors.py", line 398 in select
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 1136 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 414 in wait_result_broken_or_wakeup
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 341 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Current thread 0xf79af040 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 100 in _fail_on_deadlock
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132 in _check_crash
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 154 in test_crash_during_func_exec_on_worker
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 634 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 690 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/runner.py", line 240 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1152 in _run_suite
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1279 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 36 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 92 in test_func
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 48 in regrtest_runner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 95 in _load_run_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 138 in _runtest_env_changed_exc
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 223 in _runtest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 266 in run_single_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 82 in worker_process
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 105 in main
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 109 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main


Stderr:
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/queues.py", line 250, in _feed
    send_bytes(obj)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 185, in send_bytes
    self._check_closed()
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 138, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

======================================================================
FAIL: test_crash_during_result_pickle_on_worker (test.test_concurrent_futures.test_deadlock.ProcessPoolSpawnExecutorDeadlockTest.test_crash_during_result_pickle_on_worker)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 128, in _check_crash
    res.result(timeout=self.TIMEOUT)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 167, in test_crash_during_result_pickle_on_worker
    self._check_crash(BrokenProcessPool, _return_instance, CrashAtPickle)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132, in _check_crash
    self._fail_on_deadlock(executor)
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 109, in _fail_on_deadlock
    self.fail(f"Executor deadlock:\n\n{tb}")
AssertionError: Executor deadlock:

Thread 0xf5deb440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 341 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/queues.py", line 231 in _feed
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 996 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Thread 0xf53ff440 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/selectors.py", line 398 in select
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/multiprocessing/connection.py", line 1136 in wait
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 414 in wait_result_broken_or_wakeup
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/concurrent/futures/process.py", line 341 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1059 in _bootstrap_inner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/threading.py", line 1016 in _bootstrap

Current thread 0xf79af040 (most recent call first):
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 100 in _fail_on_deadlock
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 132 in _check_crash
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/test_concurrent_futures/test_deadlock.py", line 167 in test_crash_during_result_pickle_on_worker
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 634 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/case.py", line 690 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 122 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/suite.py", line 84 in __call__
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/unittest/runner.py", line 240 in run
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1152 in _run_suite
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/__init__.py", line 1279 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 36 in run_unittest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 92 in test_func
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 48 in regrtest_runner
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 95 in _load_run_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 138 in _runtest_env_changed_exc
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 223 in _runtest
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/single.py", line 266 in run_single_test
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 82 in worker_process
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 105 in main
  File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/libregrtest/worker.py", line 109 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main


----------------------------------------------------------------------
Ran 45 tests in 491.342s

FAILED (failures=5)

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dir

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions