Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI check to compare docs to system tables #2949

Open
Blargian opened this issue Dec 23, 2024 · 0 comments
Open

Add CI check to compare docs to system tables #2949

Blargian opened this issue Dec 23, 2024 · 0 comments
Labels

Comments

@Blargian
Copy link
Member

Blargian commented Dec 23, 2024

Motivation

To prevent sections of the docs getting removed accidentally such as what happened in #2948.
Or #2724 where auto-generated setting script failed and settings page was empty.

Feature

An idea to implement some form of basic regression testing with Github actions to check docs content against system tables, and contribute to the goal of having documentation be complete.

As an idea for something rudimentary we might pull all top level (# header1) tags on some page and compare them to the system tables to check there is a corresponding docs entry for some feature. For instance pulling all top level tags on arithmetic-functions page can be used to generate a list of the documented functions. We can compare it to system.functions to ensure that we have a section of the docs corresponding to each function in the system table. If a sections of the docs get accidentally removed then the CI check will fail on the now missing entry.

This can also be extended in the future for more specific checks. eg. for checking that feature tiers (beta, experimental, production, obsolete) in the docs are up to date with those in the system tables. If a feature changes from beta to production in the system table but is not updated in the docs then the CI check will fail.

@Blargian Blargian added the P3 label Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant