Another darglint replacement

[jsh9]

Hi @akaihola and @real-yfprojects:

I was a user of darglint. I found some aspects of darglint a dealbreaker for me, so a few weeks ago I wrote a new linter, pydoclint, from scratch (https://github.com/jsh9/pydoclint).

It was not until today that I found out about darglint2 that you started (also recently).

That's why I'm starting this discussion thread to let you know of my linter.

In any case, I plan to keep maintaining and improving pydoclint, and I have no intention to contribute to darglint or any of its derivatives, for two main reasons:

1. darglint doesn't support # noqa: ... with flake8 well
2. darglint is too slow; here's a benchmarking test I did:

	pydoclint	darglint
numpy	2.0 sec	49 min 9 sec (1,475x slower)
scikit-learn	2.4 sec	3 hr 5 min 33 sec (4,639x slower)

[real-yfprojects]

Thanks for pinging me. Your tool looks nice.

It was not until today that I found out about darglint2 that you started (also recently).

Yeah, we are working on being mentioned in the original darglint2 repository. We can definitely mention your project in our docs.

In contrast to pydoclint darglint2 isn't a new linter or a new tool but a fork of darglint. We basically took over maintaining the project since we found it valuable.

I also noticed that performance is almost a deal breaking issue with darglint (or darglint2) which we will definitely have to face when we decide to keep the project going. The usefulness of darglint2 has to be reevaluated now that there is a faster alternative. Can you maybe provide a comparison of supported lint rules? For that matter I would also be interested in the other features you have planned and the philosophy/vision of your tool.

For now I think there might be value in darglint2 because I have big plans on extending the linting capabilities to class and module docstrings.

pydoclint	darglint
numpy	2.0 sec	49 min 9 sec (1,475x slower)
scikit-learn	2.4 sec	3 hr 5 min 33 sec (4,639x slower)

What did you do differently to improve the speed? Which algorithms do you use?

[jsh9]

Can you maybe provide a comparison of supported lint rules?

This is the currently supported linting rules: https://github.com/jsh9/pydoclint/blob/a1fda04f8589382092f9cb88fded560d9ecfc325/README.md#3-style-violation-codes

They are, on a high level, not much different from what darglint offers: https://github.com/terrencepreilly/darglint#error-codes

because I have big plans on extending the linting capabilities to class and module docstrings

Darglint doesn't check class docstrings, which, in my opinion, is an omission. Because Sphinx, when rendering a documentation HTML, is expecting to read info from the class docstring rather than the __init__() docstring.

As to module docstrings, they are already checked by pydocstyle (error code: D100 - Missing docstring in public module).

In pydoclint, I don't plan to check any styles that pydocstyle already checks.

What did you do differently to improve the speed? Which algorithms do you use?

I didn't do anything magical.

I'm using numpydoc to parse numpy-style docstrings, and docstring_parser to parse Google-style docstrings. Sphinx-style docstrings can also be parsed by the latter parser.

And then I'm using Python's standard library ast to parse the Python code (the function signature including type hints, raise's, return's, yield's).

All these tasks can be achieved very fast, even in pure Python, so I'm somewhat confused as to why darglint is so slow.

[jsh9]

For that matter I would also be interested in the other features you have planned and the philosophy/vision of your tool.

As of now, I think I have implemented almost everything that I want to achieve in pydoclint, which includes:

• Ensure the the input args in the function signature match the "args" section in the docstring
• If there are return's, raise's, yield's in the function body (or return type annotation), check the docstring for corresponding sections

And specifically, I don't plan to implement any checks that pydocstyle already supports, because that would mean reinventing the wheels.

There may be other use cases that I haven't thought of yet. If users raise such requests, I'll take a look and implement them if I can.

[akaihola]

Hi @jsh9, thanks for reaching out!

The story behind the Darglint2 fork is:

• I needed to unify and check docstrings in a project of mine.
• Out of the linters I found at the time, only Darglint did what I needed.
• Darglint was inconvenient to configure for my needs, but there was a fix in the open PR Parse ignore regexes from config terrencepreilly/darglint#210
• @terrencepreilly had stopped maintenance of Darglint and archived the project.
• When asking Terrence about forking and taking over the PyPI entry, his preference was a renamed fork.
• Thus was born Darglint2, with the original intent to just publish it on PyPI with the "Parse ignore regexes from config" fix from @9dogs.
• @real-yfprojects stepped in and has been working hard on other improvements and clean-up on the project.

Regarding performance, Terrence was aware of the problem and had already experimented replacing the parser with an LR parser (see darglint#186 comments). There's also a utility for measuring the tool's performance characteristics in the integration tests.

I think it definitely makes sense for Darglint2 documentation to point to pydoclint (it already mentions pydocstyle) and warn about the current performance issue.

If you guys decide to join forces and make pydoclint fill all the needs for which people use Darglint, I think the value of Darglint2 becomes to be a maintained fork of Darglint for those existing users who need the bugfixes but don't want to migrate to a different tool. In that case we should naturally point to maintained tools at the top of the Darglint2 README (It's a bit unfortunate that we can't achieve that for the README of the original Darglint on PyPI nor on GitHub).

[jsh9]

If my understanding is correct, what @real-yfprojects proposed would be auto publishable too. I've used readthedocs.org before. I think it's a good platform, but if Github offers something built-in, it's better to use Github's (for convenience, mainly).

[real-yfprojects]

I think it's a good platform, but if Github offers something built-in, it's better to use Github's (for convenience, mainly)

• it saves the nonprofit? RTD money.

[jsh9]

I created a site based on GitHub pages: https://jsh9.github.io/pydoclint/

Right now this site is quite static: I manually added all the contents there (some of them are from README.md, so now README.md is less busy).

[real-yfprojects]

Looks neat!

Right now this site is quite static

What would you like to be dynamic? The commandline options could be generated from the source code and one could have a section detailing the software architecture that documents the important classes and functions.

[jsh9]

Actually there may not be much that needs to be made dynamic. A detailed API documentation may be dynamic, but this is not a Python library, so users don't really need to read the API docs of the internal functions.