[SIP-108] Superset Release Process

[michael-s-molina]

[SIP-108] Superset Release Process

Motivation

The Superset release process have been evolving over time as new versions are released. Many practices that have been introduced during release cycles are not clearly documented and transparently disclosed. Given that Superset releases are closely tied with the community, it's important that we establish and formalize a clear release process to allow anyone to understand, plan, and collaborate with new versions of Superset.

Proposed Change

This SIP proposes creating a Superset Wiki page to describe our release process with the following content:

How do we plan the releases

We use a combination of artifacts to plan Superset releases:

• The SIPs (Superset Improvement Proposals) kanban board is where we keep planned major improvements for Superset. The tickets inside this board are labeled with the release version number indicating in which release the improvement will likely land.
• Each major release has its own planning kanban board (3.0, 4.0) where we commit to release items from the SIP board. Major releases have their own board because they follow a specific process to gather consensus given the breaking change nature of the proposals.
• We keep a schedule that contains information about all the releases such as their planned dates, support information, and links to their boards.

The combination of the above artifacts give us our roadmap by clearly defining the planned epics, when they will be delivered and in which version.

Our releases follow Semver conventions, and they are categorized as major, minor, and patch.

Major releases

Major releases typically include major new features, significant enhancements, and improvements. They are denoted by a change in the first digit of the version number (e.g., from version 2.0 to version 3.0), often introduce significant changes to the software's functionality or user interface, and may require users to adapt to new workflows or processes. They may also involve significant changes to the underlying architecture, database schema, or APIs, which can result in compatibility issues with previous versions.

To create a major release of Superset we follow a three step process:

Gather consensus around proposed changes

To determine the proposed changes eligibility, we follow a simple process:

• We create a new GitHub project kanban board under the Apache organization for the next major release (3.0 example).
• Committers and PMC members place proposed changes on the board, ready to be proposed for consensus.
• An email with a batch of these proposed changes (numbered for convenience of reference) is sent for lazy consensus to the dev@superset.apache.org (dev@) list.
• The dev@ list has three days to object to any of these proposed changes individually (i.e. the batch, or “wave” of them is not struck down in its entirety).
• After three days pass, if there is no objection, the changes are moved into a “consensus reached” state.

Implement the proposed changes using a breaking window

Breaking windows are time-boxed periods during which we allow the introduction of breaking changes to Superset's codebase. Breaking changes are modifications or updates that have the potential to disrupt or break the existing functionality. These changes typically involve modifications to interfaces, APIs, data formats, dependencies, or behavior, which can lead to compatibility issues with other software systems, applications, or integrations that rely on the previous version of the software.

The average duration of a breaking window is one month and during that time committers tend to focus on merging agreed breaking changes. The following steps constitute a breaking window:

• We formally declare the opening of a breaking window by sending an email to the dev@ list for lazy consensus with the start and end dates of the window.
• Contributors track the state of approved breaking changes on the board until merged. It's important that the pull requests that materialize the breaking changes have an atomic nature, in the sense that once merged we don't end up in an intermediate state if the window closes.
• Any items on the board that have not been proposed for consensus, or completed during the breaking change window, will be punted to the next major release.
• When we reach the end of the breaking window, we send another email to the dev@ list confirming that no more breaking changes are allowed.

After the window is closed, Superset's codebase is reverted to normal Semver practices and breaking changes are not allowed anymore.

Generate the release

The next phase is to generate a release with the breaking changes. At this point, we implement a series of practices to accelerate the release process:

• We add messages to the #release-announcements and #community-announcements Slack channels stating that we're in a stabilization phase and asking help from the community to keep the focus on fixes and testing.
• During our daily PR review meeting, we mark feature or refactor related PRs with the review-after-release label to preserve the focus on the release. This will not restrict people from reviewing and merging a PR, but just indicate that the community will not be able to prioritize the review.
• We engage with the community through discussions (example) for each release candidate to ensure the release is as stable as possible.

To successfully validate the release, we follow Apache's voting process.

Minor releases

Minor releases, also known as incremental releases, typically introduce new features, enhancements, bug fixes, or security updates. They are denoted by a change in the number to the right of the decimal point in the version number (e.g., from version 1.0 to 1.1) and aim to improve the software's functionality, stability, and security without introducing any significant disruptions or breaking changes.

We use the SIPs (Superset Improvement Proposals) board to track which features are planned for each minor version. Since minor releases don't introduce breaking changes, their management process is simpler than major releases. They don't have specific boards to gather consensus and no concept of breaking windows. Once we make the master cut to create the release, we follow the same steps outlined in Generate the release.

Patch releases

Patch releases, also referred to as maintenance releases or bug fix releases, are updates that primarily address specific issues or bugs found in the previous version. They are denoted by a change in the number to the rightmost segment of the version number (e.g., from version 1.0.1 to 1.0.2) and typically involve fixing critical security vulnerabilities, resolving software defects, and improving the overall stability and performance of the software. Patch releases don't introduce new features or make significant changes to Superset functionality. They are assembled from time to time or when a critical bug fix is needed and the only mandatory step to create the release is to follow Apache's voting process.

When a fix is merged to master, if it's applicable to the supported versions, we cherry-pick it into the corresponding version branch, increasing the release stability.

Testing a Release Candidate (RC)

Testing a release candidate is a crucial phase to ensure the stability, functionality, and quality of the upcoming release. The primary goal is to identify and rectify any bugs, issues, or discrepancies that might affect the user experience or disrupt the intended functionality. By testing the release candidate, we can gain confidence in its reliability and make informed decisions on whether it is ready for deployment or requires further refinement.

When creating a major or minor release candidate, we always create an associated GitHub discussion (example) where the community can give feedback about the release. Patch releases don't have an associated GitHub discussion due to their simpler process and the fact that they don't contain new features or breaking changes.

The best practice to report an issue with the release candidate is to open a new GitHub issue and add the issue URL to the discussion thread. It's not a good practice to describe your problem in the release candidate discussion thread as it can grow exponentially.

When reporting an issue, it is important to follow good practices to ensure clear and effective communication. Here are some best practices for reporting an issue:

• Provide a clear and concise description: Clearly describe the issue you are facing, including any error messages or unexpected behaviors you are encountering. Be specific and provide relevant details such as steps to reproduce the issue, expected results, and actual results.
• Include necessary context: Include relevant information about your environment, such as the Superset version, platform, operating system, browser version, and any other software or hardware configurations that may be relevant to the issue.
• Include screenshots or recordings: Whenever possible, include screenshots or recordings that visually demonstrate the issue. This can help developers better understand the problem and speed up the troubleshooting process.
• Follow up and provide additional information: If the recipient requests additional information or clarifications, respond promptly and provide the requested details.

Remember, the more detailed and informative your issue report is, the better the chances of a swift resolution. Providing clear information and following these best practices can greatly assist developers and support teams in understanding and addressing the issue effectively.

Each release candidate follows the Apache's voting process and it's the responsibility of PMCs and Committers to determine if a release candidate can be released or if we need another version with additional fixes.

If a bug reported during the RC test phase is not considered a blocker for the release, we'll probably fix it in subsequent patch releases.

Distribution platforms

Formalizing the distribution platforms of Superset is important as the community can easily identify which distributions are officially supported. It also allows the community to align their efforts, support resources, and documentation around the officially supported distributions, fostering a cohesive and collaborative ecosystem.

Currently, we officially support the following distribution platforms:

• Apache Website
• PyPi
• Docker

We also officially support the distribution of superset-ui packages on NPM.

In addition to formalizing the distribution platforms that are officially supported, it is crucial to address the existence of unsupported distribution platforms within the community. Where users rely on a currently-unsupported platform - for example, the Kubernetes Helm chart - It is essential that community members assume responsibility for these platforms, providing resources, documentation, and support through community-driven channels. A new platform may be considered to be officially supported when community members consistently maintain its necessary infrastructure, document its usage, and test releases on it.

Schedule

Superset releases will follow a time-based schedule to provide predictability and a regular cadence of updates. Keep in mind that we may delay or accelerate a release, or even alter its scope, due to special circumstances such as critical security fixes.

Major or Minor Version	Current Patch Version	State	First Release	EOL/Terminated
5.0	TBD	Planned	Sep 15, 2024	Mar 15, 2025
4.1	TBD	Planned	Jun 15, 2024	Dec 15, 2024
4.0	TBD	Planned	Mar 15, 2024	Sep 15, 2024
3.1	TBD	Planned	Dec 15, 2023	Jul 15, 2024
3.0	3.0.1	Supported	Sep 18, 2023	Mar 15, 2024
2.1	2.1.1	Supported	Aug 29, 2023	Dec 15, 2023
2.0	2.0.1	EOL	Jul 14, 2022	Apr 5, 2023
1.5	1.5.3	EOL	Apr 28, 2022	Apr 5, 2023
1.4	1.4.2	EOL	Jan 19, 2022	Apr 28, 2022

Apache Superset publishes the security vulnerabilities (CVEs) associated with each of its releases.

Due to resource constraints we only support two versions (major or minor) at a time. This means that when a major or minor is released, we stop creating patches for a previous version. For example, if we are supporting 3.0 and 2.1, when 3.1 arrives, we'll drop support for 2.1 and only provide patches for 3.1 and 3.0. When 4.0 arrives, we'll drop support for 3.0 and so on.

The above statement does not prevent someone from the community from voluntarily creating a patch for an old release.

New or Changed Public Interfaces

N/A

New dependencies

N/A

Migration Plan and Compatibility

N/A

Rejected Alternatives

Continue to operate without a defined process with clear definitions.

Acknowledgements

The content described above was created by a group of people during our Release Strategy meetings. Big thanks to @eschutho @john-bodley @justinpark @michael-s-molina @rusackas @sfirke for all the work on this document.

[michael-s-molina]

I created the Release Process Wiki page with the contents of this SIP 🎉