[EPIC] Scope connection definitions by workspace

[cgardens]

Tell us about the problem you're trying to solve

We want to allow different workspaces to do a few different things:

1. Use different versions of the same connector definitions. (e.g. workspace 1 uses postgres 0.11.1 and workspace 2 uses 0.12.0)
2. Use different connectors entirely (e.g. workspace 1 has access to custom connector A and workspace 2 doesn't OR workspace 1 has access to alpha connector X and workspace 2 doesn't)
3. A workspace admin is able to change the connector versions and add custom connectors
4. An instance admin is able to change connector versions and add connectors for any workspace.
5. Ability to mark connectors that by default are provisioned to all workspaces
6. Ability to mark connectors that not provisioned by default but can be provisioned by an instance admin

Describe the solution you’d like

Tech Spec for API changes

Implemented in:

• Define APIs for scoped connector definitions #11244
• Migrations for scoped connectors #11305
• Implement CRUD routes for custom actor definitions #11330
• List actor definitions for workspace #11336
• Grant/Revoke workspace definition grants #11338
• List private definitions endpoint #11339

New endpoints added

oss	cloud
/v1/source_definitions/list_private	✅ admin only
/v1/source_definitions/list_for_workspace	✅ workspace reader
/v1/source_definitions/get_for_workspace	✅ workspace reader
/v1/source_definitions/create_custom	⛔ unimplemented
/v1/source_definitions/update_custom	⛔ unimplemented
/v1/source_definitions/delete_custom	⛔ unimplemented
/v1/source_definitions/grant_definition	✅ admin only
/v1/source_definitions/revoke_definition	✅ admin only
/v1/destination_definitions/list_private	✅ admin only
/v1/destination_definitions/list_for_workspace	✅ workspace reader
/v1/destination_definitions/get_for_workspace	✅ workspace reader
/v1/destination_definitions/create_custom	⛔ unimplemented
/v1/destination_definitions/update_custom	⛔ unimplemented
/v1/destination_definitions/delete_custom	⛔ unimplemented
/v1/destination_definitions/grant_definition	✅ admin only
/v1/destination_definitions/revoke_definition	✅ admin only

Remaining Tasks

Here are the remaining steps to get scoped connection definitions working in cloud (that should be done in order):

• uptake the changes in cloud and allow cloud users to use the new get/list endpoints for workspace scoped definitions https://github.com/airbytehq/airbyte-cloud/pull/1311
• update the webapp to use the new workspace scoped endpoints for CRUD/list of connector definitions #11606
• create a migration to convert all existing custom connector definitions in oss deployments to be custom definitions #11607 (should be done together with the previous step)
• https://github.com/airbytehq/airbyte-cloud/issues/1335
• ensure admins can access the endpoints for listing private connector definitions and granting/revoking connector definitions to workspaces https://github.com/airbytehq/airbyte-cloud/pull/1312
• https://github.com/airbytehq/airbyte-cloud/issues/1336
• airbytehq/airbyte-cloud#1869

[cgardens]

@pmossman I updated this issue to be a bit less vague. lmk if you have questions.

[git-phu]

After all of the above PRs are merged, we should have all the schema and API changes necessary for supporting scoped connectors.
Here are the remaining steps (that should be done in order):

• uptake the changes in cloud and allow cloud users to use the new get/list endpoints for workspace scoped definitions https://github.com/airbytehq/airbyte-cloud/pull/1311
• update the webapp to use the new workspace scoped endpoints for CRUD/list of connector definitions #11606
• create a migration to convert all existing custom connector definitions in oss deployments to be custom definitions #11607 (should be done together with the previous step)
• https://github.com/airbytehq/airbyte-cloud/issues/1335
• ensure admins can access the endpoints for listing private connector definitions and granting/revoking connector definitions to workspaces https://github.com/airbytehq/airbyte-cloud/pull/1312
• https://github.com/airbytehq/airbyte-cloud/issues/1336

[cgardens]

de-prioritized for Q2. not blocked.

[pmossman]

Ongoing work towards custom connectors is now captured in a new epic: https://github.com/airbytehq/airbyte-cloud/issues/3079