Support for multi-repo projects #7785

svenefftinge · 2022-01-23T12:49:47Z

This PR introduces the possibility to describe dev environments that span multiple repositories. Such a project consists of a main repository, that contains the configuration for the project's dev environment using the .gitpod.yml as usual. The additional repositories don't have a configuration on their own but all point to the main repository's configuration. Here's a loom video showing this in action.

The pointing is done bi-directionally in the repositories .gitpod.yml.
The main repository's .gitpod.yml contains a list of additionalRepositories.

How to test (example contexts)

A team with projects is set up: Join the team

Repo	Context
main repo	default
main repo	pull request
sub repo	default
sub repo	issue
sub repo	pull request

fixes #7608

Things that knowingly are not perfect yet

Multi-repo projects don't have a great representation in the UI (dashboard) yet. One still has to install webhooks/GitHub App on all participating repositories to have the webhook trigger pushevents. Webhooks coming from subrepos are treated as events for the project that contains the main configuration. That's correct but i should be visually clearer, that the project spans multiple repos and also the detail information on prebuilds should list all participating commits.

The idea is to ship a first preview version to get more feedback from users and complete the feature based on what we learn from that.

Documentation

TODO: Add Draft PR for Docs

Release Notes

Support for multi-repository setups

roboquat · 2022-01-23T12:49:51Z

@svenefftinge: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codecov · 2022-01-23T12:58:12Z

Codecov Report

Merging #7785 (36ee357) into main (3ef1e86) will decrease coverage by 1.13%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #7785      +/-   ##
==========================================
- Coverage   12.31%   11.17%   -1.14%     
==========================================
  Files          20       18       -2     
  Lines        1161      993     -168     
==========================================
- Hits          143      111      -32     
+ Misses       1014      880     -134     
+ Partials        4        2       -2

Flag	Coverage Δ
components-gitpod-cli-app	`11.17% <ø> (ø)`
components-local-app-app-darwin-amd64	`?`
components-local-app-app-darwin-arm64	`?`
components-local-app-app-linux-amd64	`?`
components-local-app-app-linux-arm64	`?`
components-local-app-app-windows-386	`?`
components-local-app-app-windows-amd64	`?`
components-local-app-app-windows-arm64	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
components/local-app/pkg/auth/auth.go
components/local-app/pkg/auth/pkce.go

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ef1e86...36ee357. Read the comment docs.

roboquat · 2022-01-26T16:41:27Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign csweichel, geropl after the PR has been reviewed.
You can assign the PR to them by writing /assign @csweichel @geropl in a comment when ready.

Associated issue: #7608

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

components/content-service/OWNERS
components/content-service-api/OWNERS
components/gitpod-db/OWNERS
components/gitpod-protocol/OWNERS
components/server/OWNERS
components/ws-daemon/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

svenefftinge · 2022-01-26T16:51:03Z

/werft run

👍 started the job as gitpod-build-se-multi-repo.9

svenefftinge · 2022-02-11T08:10:11Z

/werft run with-clean-slate-deployment

👍 started the job as gitpod-build-se-multi-repo.17

jldec

lgtm

svenefftinge · 2022-03-09T10:59:50Z

/werft run

👍 started the job as gitpod-build-se-multi-repo.43

svenefftinge · 2022-03-09T12:19:56Z

/werft run clean-slate-deployment

👍 started the job as gitpod-build-se-multi-repo.44

svenefftinge · 2022-03-09T12:48:14Z

/werft run with-clean-slate-deployment

👍 started the job as gitpod-build-se-multi-repo.45

jldec · 2022-03-10T10:07:29Z

Tested prebuilds and projects with a 2-repo setup as per draft docs

fullstack-frontend - main configuration
fullstack-backend
created projects on a team for both repos
pushed to branch on backend - saw prebuild in frontend project - no prebuilds visible in backend project
correctly opened workspace using branch and PR contexts from backend repo
pushed to matching branches on both repos
correctly opened contexts with matching branches from both frontend and backend

project list - note backend branch name in frontend project tile

all prebuilds on main repo project

no prebuilds on sub-repo project

matching branches

jldec

LGTM based on testing in dev environment.

jankeromnes · 2022-03-10T11:04:16Z

components/gitpod-db/src/typeorm/migration/1646803519382-PrebuildUpdatableSHA.ts

+
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        if (!(await columnExists(queryRunner, "d_b_prebuilt_workspace_updatable", "commitSHA"))) {
+            await queryRunner.query("ALTER TABLE d_b_prebuilt_workspace_updatable ADD COLUMN commitSHA varchar(255) NOT NULL");


What happens if you add a NOT NULL column to an existing table, without a default value? 🤔

Maybe this should be NULL, since commitSHA can be undefined in DBPrebuiltWorkspaceUpdatable.

addressed in #8711

roboquat added do-not-merge/work-in-progress do-not-merge/release-note-label-needed team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team size/XXL labels Jan 23, 2022

svenefftinge changed the title ~~Se-multi-repo~~ Support for multi-repo projects Jan 23, 2022

svenefftinge force-pushed the se-multi-repo branch from ebab111 to 8b36266 Compare January 23, 2022 13:23

svenefftinge mentioned this pull request Jan 23, 2022

Support for multi-repo projects #7615

Closed

2 tasks

svenefftinge force-pushed the se-multi-repo branch 3 times, most recently from 739ab95 to 711a613 Compare January 26, 2022 16:35

roboquat added size/XL and removed size/XXL labels Jan 26, 2022

roboquat added size/XXL and removed size/XL labels Jan 26, 2022

svenefftinge force-pushed the se-multi-repo branch from 089704c to 6a47917 Compare January 26, 2022 16:56

svenefftinge force-pushed the se-multi-repo branch 6 times, most recently from 4a05700 to 75f6232 Compare February 10, 2022 13:08

svenefftinge force-pushed the se-multi-repo branch 2 times, most recently from 872e82a to 04766eb Compare February 14, 2022 08:25

svenefftinge marked this pull request as ready for review February 14, 2022 08:28

roboquat added the do-not-merge/hold label Mar 6, 2022

jldec previously approved these changes Mar 6, 2022

View reviewed changes

svenefftinge dismissed stale reviews from jldec and csweichel via e695cf8 March 8, 2022 11:42

svenefftinge force-pushed the se-multi-repo branch 5 times, most recently from b9e17e3 to 8fd641d Compare March 8, 2022 16:58

[multi-repos] Allow specifying subrepositories

d1728f4

svenefftinge force-pushed the se-multi-repo branch from 8fd641d to e3e6b02 Compare March 9, 2022 10:25

[multi-repo] Support prebuilds on multi-repo setup

36ee357

svenefftinge force-pushed the se-multi-repo branch from e3e6b02 to 36ee357 Compare March 9, 2022 17:46

svenefftinge removed do-not-merge/hold do-not-merge/release-note-label-needed labels Mar 10, 2022

csweichel approved these changes Mar 10, 2022

View reviewed changes

jldec approved these changes Mar 10, 2022

View reviewed changes

roboquat merged commit cc2f3b3 into main Mar 10, 2022

roboquat deleted the se-multi-repo branch March 10, 2022 10:16

jankeromnes reviewed Mar 10, 2022

View reviewed changes

svenefftinge mentioned this pull request Mar 10, 2022

[server] fix compile error #8711

Merged

roboquat added deployed: workspace Workspace team change is running in production deployed: webapp Meta team change is running in production deployed Change is completely running in production labels Mar 10, 2022

This was referenced Mar 16, 2022

[server] always use . for image builder checkoutLocation #8833

Closed

imagebuilds broken by wrong checkoutlocation #8851

Closed

Support for multi-repo projects #7785

Support for multi-repo projects #7785

Uh oh!

Conversation

svenefftinge commented Jan 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to test (example contexts)

Things that knowingly are not perfect yet

Documentation

Release Notes

Uh oh!

roboquat commented Jan 23, 2022

Uh oh!

codecov bot commented Jan 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

roboquat commented Jan 26, 2022

Uh oh!

svenefftinge commented Jan 26, 2022 • edited by werft-gitpod-dev-com bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

svenefftinge commented Feb 11, 2022 • edited by werft-gitpod-dev-com bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jldec left a comment

Choose a reason for hiding this comment

Uh oh!

svenefftinge commented Mar 9, 2022 • edited by werft-gitpod-dev-com bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

svenefftinge commented Mar 9, 2022 • edited by werft-gitpod-dev-com bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

svenefftinge commented Mar 9, 2022 • edited by werft-gitpod-dev-com bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jldec commented Mar 10, 2022

project list - note backend branch name in frontend project tile

all prebuilds on main repo project

no prebuilds on sub-repo project

matching branches

Uh oh!

jldec left a comment

Choose a reason for hiding this comment

Uh oh!

jankeromnes Mar 10, 2022

Choose a reason for hiding this comment

Uh oh!

jldec Mar 10, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

svenefftinge commented Jan 23, 2022 •

edited

Loading

codecov bot commented Jan 23, 2022 •

edited

Loading

svenefftinge commented Jan 26, 2022 •

edited by werft-gitpod-dev-com bot

Loading

svenefftinge commented Feb 11, 2022 •

edited by werft-gitpod-dev-com bot

Loading

svenefftinge commented Mar 9, 2022 •

edited by werft-gitpod-dev-com bot

Loading

svenefftinge commented Mar 9, 2022 •

edited by werft-gitpod-dev-com bot

Loading

svenefftinge commented Mar 9, 2022 •

edited by werft-gitpod-dev-com bot

Loading