Proposal: Experimental features interface / API

[loujaybee]

This issue is a proposal to implement for a product-wide approach for experimental features in Gitpod.

Context / Motivations

We have the concept of "VS Code Insiders", an experimental pre-release designed for power-users to be able to test out features in VS Code before they're rolled out to the full user base (see below) [1].

But the IDE interface is now starting to get cluttered, and the distinction between pre-release versions isn't very clear (it already was not entirely clear what the insider's version is).

We are now considering applying the same approach (insiders/early access IDE's) for EAP [1] versions of JetBrains IDE's, where we have a pre-release version and a stable.

Additionally, we've had challenges in the past of users not knowing when a feature is a beta / pre-release vs a stable one (e.g. #8783). This causes potential to upset or confusion for users and causes additional support burden due to increased GitHub tickets, questions from users, etc. The docs labels help somewhat, but do not guarantee that a user understands fully the implications of an experimental feature:

We have beta labels in-app, but again, do not ensure that a user is entirely aware of implications of an experimental feature:

The same challenge applies also for experimental commands within gp cli

Solution

• A clear description of the experimental feature (to the user) e.g. what the feature does, what are the impacts
• A way to toggle on/off the feature

From a technical design perspective, the solution needs to be able to be easily plugged into other aspects of the application, so a logical API and/or interface is essential. The solution should also play nicely with any existing feature flag behaviours, and potentially could be used for A/B testing in the future. We'd also need to consider how this might look when integrated with other API's such as gp CLI.

Related: #6826 (comment)
Related internal Slack discussion.

CC: @jakobhero

[loujaybee]

Relates to #7081

[loujaybee]

CC @mikenikles — it would be useful if this could also apply to changes in the docs, as I'm thinking about how we ensure docs updates are synced also with feature updates, I believe you were thinking about moving docs into the main repo somehow so that we can review both at the same time?

[loujaybee]

Note: It has been discussed elsewhere to give community heroes some forms of early access to beta features, maybe this feature can help facilitate that, also.

[pawlean]

Note: It has been discussed elsewhere to give community heroes some forms of early access to beta features, maybe this feature can help facilitate that, also.

This sounds awesome!

[mikenikles]

Yes, I have an experimental PR at https://github.com/gitpod-io/website/pull/1367.

The technical limitation we had in April no longer exists. There are a few tasks listed in the PR that need to happen first though.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.