coverage.misc.CoverageException: Couldn't use data file '/path/to/.coverage': UNIQUE constraint failed: meta.key

[craigmaloney]

Summary

Trying to run pytest-cov in parallel but getting a unique key constraint on the .coverage SQLite3 database.

Expected vs actual result

Here's the traceback from the run:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 1048, in execute
INTERNALERROR>     return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.IntegrityError: UNIQUE constraint failed: meta.key
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/_pytest/main.py", line 247, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/pytest_cov/plugin.py", line 254, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/pytest_cov/engine.py", line 197, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/coverage/control.py", line 642, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/coverage/control.py", line 696, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(self.mapped_file_dict(self.data))
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 437, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 497, in _choose_lines_or_arcs
INTERNALERROR>     ('has_arcs', str(int(arcs)))
INTERNALERROR>   File "/home/craig/projects/epathermostat/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 1065, in execute
INTERNALERROR>     raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, msg))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/home/craig/projects/epathermostat/.coverage': UNIQUE constraint failed: meta.key

(Enclosed tox.ini and pytest.ini)

Reproducer

Versions

pytest version 5.4.1
pytest-cov-2.8.1
tox: 3.14.5

Make sure you include complete output of tox if you use it (it will show versions of various things).

ini_files.zip

[davvid]

You're probably running into something similar to #386

There's a way to avoid this issue. In your .coveragerc make sure you have:

[run]
parallel = true

In your tox.ini make sure you use --cov-append and use an explicit clean step, as documented in the pytest-cov examples.

Lastly, make sure you have your dependencies locked down to coverage<5.

Without these settings, I am able to reproduce this same issue pretty easily because it'll use coverage 5. With these settings active you won't run into it anymore.

It's a workaround until a coverage5-compatible approach is documented, but it works.

[edwelker]

It would seem that dropping back to pytest==5.3.5 also corrects this issue.

So the options are either coverage<5 or pytest<5.4. It's not clear to me what the incompatibility is between them.

(Using pytest-cov==2.8.1 and coverage==5.1)

[JulienPalard]

This can be related to nedbat/coveragepy#883 (comment)

[barseghyanartur]

You're probably running into something similar to #386

There's a way to avoid this issue. In your .coveragerc make sure you have:

[run]
parallel = true

In your tox.ini make sure you use --cov-append and use an explicit clean step, as documented in the pytest-cov examples.

Lastly, make sure you have your dependencies locked down to coverage<5.

Without these settings, I am able to reproduce this same issue pretty easily because it'll use coverage 5. With these settings active you won't run into it anymore.

It's a workaround until a coverage5-compatible approach is documented, but it works.

Brilliant! Much appreciated!

[silverjam]

In your tox.ini make sure you use --cov-append and use an explicit clean step, as documented in the pytest-cov examples.

I believe this is the documentation that's being referred to: https://pytest-cov.readthedocs.io/en/latest/tox.html?highlight=clean#tox