server: rate limit prebuilds by cloneURL #8568

andrew-farries · 2022-03-03T12:52:07Z

Description

Introduce a rate-limit for prebuilds.

Prebuilds are rate limited by cloneURL in the last 1 minute to X. X either comes from Config, or defaults to 50.

Related Issue(s)

Fixes #5176

How to test

Same instructions as #8504

To have an easier time, you can modify the server-configmap.yaml on your preview and a special rate limit for your clone URL
```
 "https://gitlab.com/<USER>/test-gitpod-prebuilds-rate-limit.git": 10,
```

Tests

With a rate limit of 10:

Release Notes

Rate-limit workspace prebuilds to 50 per minute (rolling-window) by default, configurable through config.

Documentation

/werft with-clean-slate-deployment

codecov · 2022-03-03T14:12:03Z

Codecov Report

Merging #8568 (ec57d20) into main (f4826d3) will decrease coverage by 1.13%.
The diff coverage is n/a.

❗ Current head ec57d20 differs from pull request most recent head 519dbe9. Consider uploading reports for the commit 519dbe9 to get more accurate results

@@            Coverage Diff             @@
##             main    #8568      +/-   ##
==========================================
- Coverage   12.31%   11.17%   -1.14%     
==========================================
  Files          20       18       -2     
  Lines        1161      993     -168     
==========================================
- Hits          143      111      -32     
+ Misses       1014      880     -134     
+ Partials        4        2       -2

Flag	Coverage Δ
components-gitpod-cli-app	`11.17% <ø> (ø)`
components-local-app-app-darwin-amd64	`?`
components-local-app-app-darwin-arm64	`?`
components-local-app-app-linux-amd64	`?`
components-local-app-app-linux-arm64	`?`
components-local-app-app-windows-386	`?`
components-local-app-app-windows-amd64	`?`
components-local-app-app-windows-arm64	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
components/local-app/pkg/auth/pkce.go
components/local-app/pkg/auth/auth.go

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f4826d3...519dbe9. Read the comment docs.

easyCZ · 2022-03-07T07:57:12Z

/werft run

jankeromnes

Nice! 👏

Two comments in passing. 😇

EDIT: Also, I love that this limit is configurable per cloneUrl. Thanks a lot for doing that. 💯

components/server/ee/src/prebuilds/prebuild-manager.ts

chart/values.yaml

components/gitpod-db/src/workspace-db.spec.db.ts

components/server/ee/src/prebuilds/prebuild-manager.ts

components/gitpod-db/src/typeorm/workspace-db-impl.ts

components/gitpod-db/src/workspace-db.spec.db.ts

components/server/ee/src/prebuilds/prebuild-manager.ts

jankeromnes · 2022-03-07T18:18:04Z

Reminder before this gets merged: Please try to squash your commits into one or a few logical commits with a descriptive commit message. 🙏

easyCZ · 2022-03-07T20:09:37Z

Squashed commits.

easyCZ · 2022-03-07T20:21:58Z

I suppose this PR will also need documentation - around configuration of the prebuild rate limits.

easyCZ · 2022-03-08T09:01:05Z

/werft run

👍 started the job as gitpod-build-af-rate-limits.40

components/server/src/config.ts

components/gitpod-db/src/typeorm/workspace-db-impl.ts

geropl · 2022-03-08T10:40:52Z

@andrew-farries Alright, thanks a lot for tackling this! 🙏 I expected I would manage to finish the review quicker... but anyway, I'm through now.

@easyCZ thx for the adjustments and squashing (we should really think about allowing for this/make it the default).

The only thing left I see here is the migration for an index.

Prebuilds are rate-limited to N in the last S seconds on a rolling window basis. By default, 50 prebuilds are allowed in a 1 minute window. A configuration property `prebuildLimiter` is added which controls default rate limit but allows for explicit overrides by cloneURL.

geropl

Awesome, thanks to both of you! 🙏

geropl · 2022-03-08T14:10:09Z

/unhold

roboquat added do-not-merge/work-in-progress do-not-merge/release-note-label-needed size/XS size/S and removed size/XS labels Mar 3, 2022

roboquat added size/M and removed size/S labels Mar 4, 2022

easyCZ changed the title ~~Add logging around starting prebuilds~~ server: rate limit prebuilds by cloneURL Mar 4, 2022

easyCZ force-pushed the af/rate-limits branch from 01e2338 to 4917af5 Compare March 4, 2022 14:45

roboquat added release-note size/L and removed do-not-merge/release-note-label-needed size/M labels Mar 4, 2022

easyCZ force-pushed the af/rate-limits branch 2 times, most recently from 4e8ac63 to de567c4 Compare March 5, 2022 06:54

roboquat added size/M size/L and removed size/L size/M labels Mar 5, 2022

easyCZ force-pushed the af/rate-limits branch from 9f5b93a to ca2b546 Compare March 7, 2022 07:32

jankeromnes reviewed Mar 7, 2022

View reviewed changes

components/server/ee/src/prebuilds/prebuild-manager.ts Show resolved Hide resolved

chart/values.yaml Outdated Show resolved Hide resolved

easyCZ marked this pull request as ready for review March 7, 2022 16:20

easyCZ requested a review from a team March 7, 2022 16:20

roboquat removed the do-not-merge/work-in-progress label Mar 7, 2022

github-actions bot added the team: webapp Issue belongs to the WebApp team label Mar 7, 2022

geropl reviewed Mar 7, 2022

View reviewed changes

components/gitpod-db/src/workspace-db.spec.db.ts Outdated Show resolved Hide resolved

geropl reviewed Mar 7, 2022

View reviewed changes

components/server/ee/src/prebuilds/prebuild-manager.ts Show resolved Hide resolved

easyCZ reviewed Mar 7, 2022

View reviewed changes

components/gitpod-db/src/typeorm/workspace-db-impl.ts Show resolved Hide resolved

components/gitpod-db/src/workspace-db.spec.db.ts Outdated Show resolved Hide resolved

components/server/ee/src/prebuilds/prebuild-manager.ts Show resolved Hide resolved

geropl reviewed Mar 7, 2022

View reviewed changes

components/server/ee/src/prebuilds/prebuild-manager.ts Outdated Show resolved Hide resolved

geropl reviewed Mar 7, 2022

View reviewed changes

components/server/ee/src/prebuilds/prebuild-manager.ts Outdated Show resolved Hide resolved

easyCZ added the do-not-merge/hold label Mar 7, 2022

easyCZ force-pushed the af/rate-limits branch from 9ed43ef to c2c328e Compare March 7, 2022 20:08

easyCZ mentioned this pull request Mar 7, 2022

Namespace prebuild tracing tags #8648

Closed

easyCZ force-pushed the af/rate-limits branch from c2c328e to eef9bd1 Compare March 8, 2022 09:49

easyCZ requested review from geropl and jankeromnes March 8, 2022 09:49

geropl reviewed Mar 8, 2022

View reviewed changes

components/server/src/config.ts Outdated Show resolved Hide resolved

geropl reviewed Mar 8, 2022

View reviewed changes

components/gitpod-db/src/typeorm/workspace-db-impl.ts Show resolved Hide resolved

easyCZ force-pushed the af/rate-limits branch from 5abcbe1 to 519dbe9 Compare March 8, 2022 11:51

easyCZ requested a review from geropl March 8, 2022 11:51

geropl approved these changes Mar 8, 2022

View reviewed changes

roboquat removed the do-not-merge/hold label Mar 8, 2022

roboquat merged commit 54a765d into main Mar 8, 2022

roboquat deleted the af/rate-limits branch March 8, 2022 14:10

roboquat added deployed: webapp Meta team change is running in production deployed Change is completely running in production labels Mar 15, 2022

jldec added type: epic and removed release-note deployed Change is completely running in production deployed: webapp Meta team change is running in production labels Apr 8, 2022

jldec mentioned this pull request May 24, 2022

Repo with incremental prebuilds temporarily stopped triggering prebuilds #10209

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

server: rate limit prebuilds by cloneURL #8568

server: rate limit prebuilds by cloneURL #8568

andrew-farries commented Mar 3, 2022 •

edited by easyCZ

Loading

codecov bot commented Mar 3, 2022 •

edited

Loading

easyCZ commented Mar 7, 2022

jankeromnes left a comment •

edited

Loading

jankeromnes commented Mar 7, 2022 •

edited by werft-gitpod-dev-com bot

Loading

easyCZ commented Mar 7, 2022

easyCZ commented Mar 7, 2022

easyCZ commented Mar 8, 2022 •

edited by werft-gitpod-dev-com bot

Loading

geropl commented Mar 8, 2022

geropl left a comment

geropl commented Mar 8, 2022

server: rate limit prebuilds by cloneURL #8568

server: rate limit prebuilds by cloneURL #8568

Conversation

andrew-farries commented Mar 3, 2022 • edited by easyCZ Loading

Description

Related Issue(s)

How to test

Tests

Release Notes

Documentation

codecov bot commented Mar 3, 2022 • edited Loading

Codecov Report

easyCZ commented Mar 7, 2022

jankeromnes left a comment • edited Loading

Choose a reason for hiding this comment

jankeromnes commented Mar 7, 2022 • edited by werft-gitpod-dev-com bot Loading

easyCZ commented Mar 7, 2022

easyCZ commented Mar 7, 2022

easyCZ commented Mar 8, 2022 • edited by werft-gitpod-dev-com bot Loading

geropl commented Mar 8, 2022

geropl left a comment

Choose a reason for hiding this comment

geropl commented Mar 8, 2022

andrew-farries commented Mar 3, 2022 •

edited by easyCZ

Loading

codecov bot commented Mar 3, 2022 •

edited

Loading

jankeromnes left a comment •

edited

Loading

jankeromnes commented Mar 7, 2022 •

edited by werft-gitpod-dev-com bot

Loading

easyCZ commented Mar 8, 2022 •

edited by werft-gitpod-dev-com bot

Loading