Don't use docstrings in test methods

[brettcannon]

It makes tracking down failures among the 1393 tests in test_importlib in the stdlib really hard as the docstring obscures the name of the failing test.

[brettcannon]

I'm already fixing them in python/cpython#25119 if you can copy-and-paste the result back to here.

[jaraco]

Hi Brett. I find this request surprising. Under what conditions should methods not have docstrings? Is it just for importlib? For anything in stdlib? For anything running under unit test?

I'm not a fan of multi-line comments because they're a pain to edit.

Perhaps there's another approach that wouldn't require re-writing the whole importlib test suite? Even adding the name of the test function to every docstring would be preferable to these multi-line comments. Or what about if the test runner removed all of the docstrings before running the tests?

I'll oblige your request, but I'd prefer to find a less invasive and degrading solution, or at least have more context.

[jaraco]

Just occurred to me, based on the date, maybe this is a joke. 📆 🤡 If it is, it sure is well executed.

[brettcannon]

Just occurred to me, based on the date, maybe this is a joke. 📆 🤡 If it is, it sure is well executed.

Sorry, not a joke.

Under what conditions should methods not have docstrings?

When they are test methods executed by unittest and its test discovery system in the stdlib. If you look in the rest of the stdlib this is the common practice thanks to unittest's runner using the docstring as the test name in the output rather than the test itself.

For instance, if you run ./python -m test -v test_importlib in the stdlib, the importlib.metadata tests just print out a chunk of their docstring. There's no indication of which test failed, etc. to trace back to the failing test without looking at the traceback (which also doesn't help when a test is marked as skipped) So in my case, I was looking for DeprecationWarning output, but to tie it back to importlib.metadata I had to copy the (portion of the) docstring printed and then do a complete recursive directory search to find the offending test(s). That was a massive pain for me with no perceived benefit of having the docstring printed out by the test runner.

I'm not a fan of multi-line comments because they're a pain to edit.

I don't love them either, but it's an unfortunately side-effect of how unittest's runner was designed.

Perhaps there's another approach that wouldn't require re-writing the whole importlib test suite?

The vast majority of test_importlib is entirely fine. It's really just importlib-metadata which is breaking the common practice of no docstrings in tests in the stdlib.

Even adding the name of the test function to every docstring would be preferable to these multi-line comments.

Except now you can't rename or move the tests without breaking the docstring.

Or what about if the test runner removed all of the docstrings before running the tests?

People have proposed updating unittest before and it never went anywhere due to backwards-compatibility/it's-what-people-are-used-to concerns. It's a key reason I use pytest for my personal projects these days.

[jaraco]

Fixed in 63e65f3 (cpython branch only). I'll keep the docstrings here. I hope at some point to come up with a more general fix for unittest, but I'll kick that can down the road for now.

[jaraco]

I filed bpo-46126 to track the concern.