-
-
Notifications
You must be signed in to change notification settings - Fork 800
Automation
The endoflife.date project runs a bit of automation on top of GitHub Actions to automate mundane tasks. This is primarily updating the latest version of each product, wherever possible. This page describes how the pipeline works.
The auto
key in the frontmatter for any product can be used to configure the automation of updates for that particular product. See CONTRIBUTING.md for details of the schema for this specific key. As of now, only dockerhub
, and git
URLs are supported.
Note that auto
is a list of configurations, each containing a regex and optionally - regex/template. This is required for cases where the product is maintained across multiple repositories for eg. If the same version is duplicated across multiple of these configurations, the latter will take precedence.
An optional regex
can be provided to parse the tag into a presentable version number. The regex can be used to parse out the major/minor/patch details from the upstream tag. So for eg, ruby tags its releases like v3_1_2
, this can be parsed with a relevant regex into the correct components. Ruby releases occasionally contain a tiny
component. To account for that, we also set a template
variable that uses Liquid Templating to render the relevant tag. In this case a simple template that checks for the tiny component works: {{major}}.{{minor}}.{{patch}}{%if tiny %}p{{tiny}}{%endif%}
For most projects, writing a regex/template is not required, as long as the project follows semantic versioning - the default regex/template will work.
Once a product page has been updated with a valid auto
configuration, the release-data
repository will automatically pick it up using this workflow. The code in update.rb will clone the repository (a minimal filter clone if possible) and update the corresponding files in the releases
subdirectory.
Post this generic update for all supported project, custom updates can be run for other projects. This can be a script that scrapes a known website. Currently, we have simple scripts for unrealircd and haproxy that scrape the upstream website for release information.
The release-data
repository maintains release information in a simple JSON file, with each release being paired with the date of release. This looks something like:
{
"1.0.0": "2017-10-16",
"1.0.1": "2017-11-20",
...
"1.9.16": "2022-02-28",
"1.9.17": "2022-04-14"
}
Note that relying on upstream tags might not always be an accurate way to calculate the release date for every software. In some cases, the tag might be created much before the announcement of the release. Unless a tag is annotated in Git, its datetime is the same as the time of the commit, which might cause further delay.
And while we pick up the correct timezone from the tag, this might sometimes be inaccurate as the release might be created elsewhere.
Such quirks, if any, should be documented in the corresponding talk page for the product.
The release-data
repository is included in the endoflife-date repository as a git submodule in the _data/release-data
subdirectory. This submodule is tracked by Dependabot, which will file a PR (at most once a day) to update the data.
On such PRs, the latest.py
script triggers with this workflow which then does the following:
- Update the
latest
field in every product with a corresponding file in therelease-data
repository. - Update the
latestReleaseDate
field in every product with a corresponding file in therelease-data
repository. - Update the
release
date field by checking the date of first release that matches each cycle and using the same. - Committing back this information to the repository.
- Marking the PR to be auto-merged. This only happens if the script runs successfully, and the Netlify deploy also passes.
- Leaving a comment on the PR in case there are new releases that need to be tracked. This information is not automatically updated because EoL/support dates for a new release cycle need a bit more review.
Here's some ideas on how we can build upon this automation tooling:
- Make use of
latestReleaseDate
field by showcasing it in the UI/API. This can also be used for RSS feed updates. - Find a sensible way to "pin" certain fields so not everything is automatically updated.
- Generate a list of "supported" vs "unsupported" releases that can be used for automation purposes.
- Allow changing of changelogTemplate easily within release-data to allow the automation of releases such as NVIDIA Driver who do not follow a set pattern for URLs.