Pluggable Workspace Cluster: Admission Constraints

[csweichel]

This PR adds "admission constraints" to pluggable workspace cluster. With this change one can limit the set of workspace clusters a workspace can land on. To this end, each cluster registration can have a set of "admission constraints" associated with it. For the time being there are two constraints available:

• has-feature-preview makes the cluster available if the user who is starting the workspace has "feature preview" enabled
• has-permission=<permission> makes the cluster available if the user has the <permission>

In addition this PR

• updates the bash completion support for gpctl (gpctl completion --help)
• makes gpctl clusters list show all clusters, static and dynamic. There's a static flag that differentiates the two.
• introduces gpctl clusters swap to change over state and score for two clusters

This enables a set of use-cases:

• register a new cluster and try it out:

  # assuming your user has the `new-workspace-cluster` permission
  
  # register the cluster
  gpctl clusters register --name demo --hint-cordoned ...
  
  # add the admission constraint
  gpctl clusters update --name demo admission-constraint add has-permission=new-workspace-cluster
  
  # uncordon and increase score to ensure this is the preferred cluster for everyone with the permission
  gpctl clusters uncordon --name demo
  gpctl clusters update --name demo score 10000

• make a cluster available to users with feature-preview:

  # register the cluster
  gpctl clusters register --name demo --hint-preferability dont-schedule ...
  
  # add the admission constraint
  gpctl clusters update --name demo admission-constraint add has-feature-preview
  
  # increase score to start some workspaces for users with feature-preview on the new cluster
  gpctl clusters update --name demo score 10

• change over to a "fallback cluster"

  $ gpctl clusters list
  NAME        URL                    STATIC        STATE            SCORE        GOVERNED        ADMISSION CONSTRAINTS
  prod         ws-manager:8080        false          AVAILABLE       50          true            []
  fallback     ws-manager:8082        false          CORDONED        0           true            []
  
  # swap prod and fallback
  gpctl clusters swap prod fallback
  
  $ gpctl clusters list
  NAME        URL                    STATIC        STATE            SCORE        GOVERNED        ADMISSION CONSTRAINTS
  prod         ws-manager:8080        false          CORDONED       0          true            []
  fallback     ws-manager:8082        false          AVAILABLE        50           true            []
  

[corneliusludmann]

Code looks good. I have not found anything to complain about. 👍

[geropl]

• c.govern

👍