Add script to list third-party package status

[srittau]

This script lists all third-party package versions in typeshed and on PyPI and the latest release date of the upstream package. This allows us to quickly spot outdated packages.

By default, it prints the output in a tabular text format, but there is an HTML output version. Ideally, we'd run a daily GitHub action to publish the HTML page somewhere.

Example output:

DateTimeRange                 1.2.*       1.2.0      2021-07-10
Deprecated                    1.2.*       1.2.13     2021-09-05
Flask-Cors                    3.0.*       3.0.10     2021-01-06
Flask-SQLAlchemy              2.5.*       2.5.1      2021-03-18
JACK-Client                   0.5.*       0.5.3      2020-10-21
Markdown                      3.3.*       3.3.6      2021-11-17
Pillow                        9.0.*       9.0.1      2022-02-04
PyMySQL                       1.0.*       1.0.2      2021-01-09
PyYAML                        6.0.*       6.0        2021-10-13
Pygments                      2.9.*       2.11.2     2022-01-06
SQLAlchemy                    1.4.*       1.4.32     2022-03-06
Send2Trash                    1.8.*       1.8.0      2021-08-09
aiofiles                      0.8.*       0.8.0      2021-11-27
annoy                         1.17.*      1.17.0     2020-09-18
appdirs                       1.4.*       1.4.4      2020-05-11
atomicwrites                  1.4.*       1.4.0      2020-04-28
[...]

The output will be colored if termcolor is installed.

[JelleZijlstra]

Thanks for creating this! I'd be hesitant to put it in this repo until we have a concrete plan for how to use it. Should we host a page somewhere that shows this table? What would people use it for?

[AlexWaygood]

Could we maybe do something like what we do with the "daily test"?

We could create a github action that checks every night to see if any packages meet some criteria of obsolescence. If any do, it could automatically create an issue, and then we could decide whether or not it should be removed.

It's not necessarily an either/or approach with this script — the two might work well together :)

[hauntsaninja]

My preferred automation here is something "dependabot" style. Where we automatically get PRs updating the version in METADATA.toml and then stubtest tells us if things are different.

[srittau]

This should be auto-generated nightly and put somewhere easily accessible. Unless we have a better idea, I could put it at typeshed.rittau.org at first. I also looked into GitHub Pages, but found no fast way to deploy this. (I also might have broken something, since now "This branch has not been deployed" shows up below PRs.)

I have no super concrete ideas how this will be used, although a few ideas include:

• See at a glance how old our third-party stubs are and how many are outdated. We could add a total to the web page.
• See the status of stubs without having to navigate to PyPI and search for the package.
• New contributors could start updating stubs for a package they are using themselves.
• Help us decide if a package is still worth keeping in typeshed. I don't think we can really automate this. For example, termcolor's last release was more than 10 years ago, but it's still working with Python 3.10.

I'm actually working on a second script that will show us the completion status of all stubs, i.e. how many fields are still unannotated. (#7535 will help with that.) This could also be used for new contributors or to push nearly complete stubs to finalization.

[Akuli]

Maybe the published list should go to a GitHub issue on this repo. Issues can be edited from github actions, and it would be very discoverable for contributors, especially if we pin the issue.

A potential downside: people would likely comment on the issue about package-specific things, and those comments would get outdated as the status of that package changes.

[AlexWaygood]

A potential downside: people would likely comment on the issue about package-specific things, and those comments would get outdated as the status of that package changes.

I think we can lock issues so that only collaborators can comment on them? (Does GitHub Actions count as a collaborator?)

[AlexWaygood]

@srittau, reckon we can close this now that we have https://alexwaygood.github.io/typeshed-stats/? :)