Simplify pytest-cov configuration.

[jdangerx]

Overview

I was investigating #3381 when I ran into a slight weirdness in our pytest-cov configuration in pyproject.toml.

It sort of seems like this would fix it? @zaneselvans I know you were the one who was messing with this in the first place, did you already try this and have it not work?

See docs for v4.1.0 at
https://github.com/pytest-dev/pytest-cov/blob/2c9f2170d8575b21bafb6402eb30ca7de31e20b9/docs/config.rst

• particularly:

If you use the --cov=something option (with a value) then coverage's
source option will also get overridden. If you have multiple sources it
might be easier to set those in .coveragerc and always use --cov
(without a value) instead of having a long command line with --cov=pkg1
--cov=pkg2 --cov=pkg3 ....

Testing

How did you make sure this worked? How can a reviewer verify this?

I removed .coverage, then ran make pytest-coverage and saw that the coverage denominator was still 13450 statements.

To-do list

Beta Give feedback

1. Ensure docs build, unit & integration tests, and test coverage pass locally with make pytest-coverage (otherwise the merge queue may reject your PR)

[zaneselvans]

I think you're probably right about swapping in a --cov here.

There are a couple of other places where we're calling pytest that we might want to check to see if we're repeatedly specifying the same args unnecessarily or in conflict: the nightly build script, the Makefile, and the pre-commit configuration. (the pytest workflow uses make)

[zaneselvans]

I don't remember exactly what was happening here, but it seemed to have to do with coverage behaving differently when run directly, vs. run through pytest. But maybe I just didn't realize that a blanket --cov would default to whatever sources were listed in the configuration file. Seems likely.

We also need to deal with -n auto crashing your machine when you try to run the validation tests since we end up with potentially several copies of our largest dataframes getting read into memory simultaneously which is too much. But this problem isn't limited to this line.

Did you look at the Makefile, .pre-commit-config.yaml and docker/gcp_pudl_etl.sh to see if the pytest / coverage commands being used in there need to be updated to work with these addopts and/or avoid duplicative specifications of the same arguments in multiple places?

[jdangerx]

@zaneselvans - went through the places you mentioned (Makefile, gcp_pudl_etl.sh, and pre-commit) and found that we were respecifying the gcs cache path everywhere. So I pulled that into the pyproject.toml.

Other than that, I removed the -n auto because of the memory issue, and everything else seems to be fine / non-duplicative.

[zaneselvans]

If there's a safe number of CPUs we could use by default (like 4?) it would nice to keep some amount of parallelism. It does speed things up. Maybe we can turn it back on after we kill PudlTabl and get all the validations into the ETL / Pandera setup.