Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest-plugins / Sphinx extensions #10

Open
stefanv opened this issue Feb 27, 2023 · 15 comments
Open

pytest-plugins / Sphinx extensions #10

stefanv opened this issue Feb 27, 2023 · 15 comments

Comments

@stefanv
Copy link
Member

stefanv commented Feb 27, 2023

Hackmd link for the summit: https://hackmd.io/JL5slkxORA-q7VRN79v1sA

@stefanv stefanv changed the title pytest-plugins pytest-plugins / Sphinx extensions Feb 27, 2023
@tylerjereddy
Copy link

I haven't found a plugin that allows regex usage in i.e., pytest -k "test_name". That could be pretty useful, though not really exclusively useful to scientific projects.

This helps you avoid running:

  • pytest -k "test_name_1"
  • pytest -k "test_name_2"
  • etc.

if the tests are spread over multiple modules but are affected by changes under iteration.

@pllim
Copy link

pllim commented Feb 27, 2023

I am particularly interested to see if any of these can be moved upstream from Astropy. Or if Astropy could retire some of them and use something else that is better maintained and more mainstream.

pytest

Sphinx

@bsipocz
Copy link
Member

bsipocz commented Feb 28, 2023

Tyler's feature request rings a bell (not necessarily regex, but to accept wildcard), but now I can't seem to find it. Point is that it was certainly asked as a feature before and I'm sure would be appreciated by many libraries.

@bsipocz
Copy link
Member

bsipocz commented Feb 28, 2023

And I for one will have more incentive and thus time to maintain the plugins above (with the exception of the sphinx-theme, that is extremely old-fashioned and should have been retired a long ago) if: - they are move out from the astropy organization somewhere where infrastructure and its maintenance is valued. - have a wider adoption than a single domain.

@pllim
Copy link

pllim commented Feb 28, 2023

Re: #10 (comment) -- I agree that for wider adoption, they should be moved out of Astropy org since these are not strictly related to astronomy sub-field in science. I will bring this issue to Astropy infrastructure tag-up but I do not foresee any opposition. Thanks!

@pllim
Copy link

pllim commented Feb 28, 2023

Re: astropy theme -- I think Astropy would be really happy to maintain once less theme if we can get a new one to work. 😸 Please also see astropy/astropy-sphinx-theme#29

@bsipocz bsipocz assigned bsipocz and unassigned bsipocz Apr 13, 2023
@drammock
Copy link

drammock commented Apr 20, 2023

commenting because I can't self-assign. MNE-Python has a couple small sphinx extensions that might be of general interest:

  • this one turns :gh:\`1234\` into a link to the issue/PR
  • this one turns :newcontrib:\`Name Surname\` into a bold-formatted link and prepends "new contributor" to it
  • this one joins a number and its unit with a non-breaking space so they aren't separated across linebreaks.

@bsipocz
Copy link
Member

bsipocz commented Apr 28, 2023

Neither plugin nor extension, but numpydoc is also very much part of the shared infrastructure.

And fixing the single-vs-double backtick issues for parameters in numpydoc would be a nice outcome that helps with doctesting, too: numpy/numpydoc#419 (comment)

@rossbar
Copy link
Member

rossbar commented May 1, 2023

And fixing the single-vs-double backtick issues for parameters in numpydoc

For context, this often rears it's ugly head when sphinx's nitpicky mode is activated, as the parameters enclosed in single backticks (as recommended by the numpydoc style guide) then get wrapped with the default sphinx role (typically :py:obj:), creating broken links. This is definitely on my list of todo's for the summit!

@rossbar rossbar self-assigned this May 1, 2023
@mdhaber mdhaber self-assigned this May 1, 2023
@tylerjereddy
Copy link

FYI, I drafted a pytest-regex plugin in 2-3 hours or so, which already seems considerably more capable than the conventional -k. Of course there are no tests for the plugin itself yet, but I added examples on how to use it in the README over there, and it seems to do the right thing so far...

Anyway, just getting rid of the initial inertia on that ahead of the summit. FWIW, it sounds like the main pytest project may not be opposed to directly supporting it from one or two issues I scanned through, but prototying quickly externally at the summit that way may not hurt anyway, and perhaps they'd handle the absorption if they really wanted.

The other thing is that folks may want features that the main project wouldn't, so defining that first probably makes sense with a test in a very small external plugin codebase to start.

@seberg
Copy link

seberg commented May 8, 2023

One thing related that I would be interested in also is infrastructure similar to/tagged on top of pytest-doctestplus to modify docs based on actual output. (That is to enable mass changes, I suppose whatever that should be should have something like a new_str = func(old_str, evaluated_str) to hook that in.)

(That is related to the fact we need to change how NumPy scalars print.)

@bsipocz
Copy link
Member

bsipocz commented May 8, 2023

@seberg - There is nothing new under the sun, would be wonderful to implement this issue 😅 scientific-python/pytest-doctestplus#107

@lagru lagru self-assigned this May 9, 2023
@drammock
Copy link

Another "neither pytest nor sphinx" thing that could be standardized/shared across projects is the website banner saying "warning this is an old version / the development version of this package, please switch to stable docs". The pydata-sphinx-theme has added an announcement banner that is configured in conf.py but it's not easy to get it to contextually appear/change messages depending on docs version (esp. when older versions of your docs were built with other themes / before the announcement banner was part of the theme).

  • @jarrodmillman describes the networkx approach here
  • MNE-Python uses this javascript that was originally adapted from sklearn but has diverged somwhat over the years

What are other folks doing? Is it worth trying to standardize/centralize this for the ecosystem?

@stefanv
Copy link
Member Author

stefanv commented May 10, 2023

Yes, please! I think this should be a built-in feature in pydata-sphinx-theme. It's not too hard to implement, but there are a few subtleties as you'll see on that issue on skimage

@tylerjereddy
Copy link

I touched base with upstream about pytest-regex in the cross-linked issue above, and so far it sounds like they won't want to upstream it--they may pursue some kind of JSON parsing technology for test selection. It doesn't seem like it is worth arguing with them about that, it was so easy to implement maybe it should just be a tiny plugin then, not sure how much time it is worth spending on though, maybe just try to get a few tests and put on PyPI and see if anyone uses it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

10 participants