Scaling Roadmap 2019 #3040

josephburnett · 2019-01-30T17:21:03Z

This is what we hope to accomplish in 2019.

docs/roadmap/scaling-2019.md

josephburnett · 2019-02-01T20:28:56Z

/assign mattmoor

Hey @mattmoor, I would love your input on this draft 2019 scaling roadmap.

greghaynes · 2019-02-06T20:41:54Z

/lgtm

docs/roadmap/scaling-2019.md

Co-Authored-By: josephburnett <josephburnett79@gmail.com>

docs/roadmap/scaling-2019.md

mattmoor · 2019-02-12T20:44:36Z

docs/roadmap/scaling-2019.md

+
+The goal of Overload Handling is to enqueue requests at a revision-level. Scale-from-zero should not overload if autoscaling can react in a reasonable amount of time to provide additional pods. When new pods come online, they should be able to take load for the existing pods. Even when scaled above zero, brief spikes of overload should be handled by enqueuing requests at a revision-level. The depth of the revision-level queue should also be configurable because even the Revision as a whole needs to guard against overload.
+
+The overall problem touches on both Networking and Autoscaling, two different working groups. Much of the overload handling will be implemented in the Activator, which is a part of ingress. So this project is shared jointly between the two working groups.


cc @tcnghia

docs/roadmap/scaling-2019.md

josephburnett · 2019-03-27T16:23:00Z

@mattmoor this is ready for another look.

bbrowning · 2019-03-28T16:01:10Z

@josephburnett thanks for the ping - added a PR to your fork & branch proposing something around migrating K8s Deployments to Knative.

Propose section on migration K8s Deployments

googlebot · 2019-03-28T19:21:27Z

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of all the commit author(s), set the cla label to yes (if enabled on your project), and then merge this pull request when appropriate.

ℹ️ Googlers: Go here for more info.

bbrowning · 2019-03-28T19:45:13Z

Sorry I messed up the CLA bot. I confirm that I am ok with my commits being contributed to this project.

And, feel free to rebase and take my wording without my actual commit if that makes it easier.

googlebot · 2019-03-28T21:53:26Z

A Googler has manually verified that the CLAs look good.

(Googler, please make sure the reason for overriding the CLA status is clearly documented in these comments.)

ℹ️ Googlers: Go here for more info.

markusthoemmes

A few comments, I feel like some of the points are narrowed down to much in a specific direction.

docs/roadmap/scaling-2019.md

- Unified some wording (capitalization mostly). - Removed prescriptive key steps. These should be captured by the respective projects, which will be more dynamically changeable than this document.

Reworked parts of the Scaling roadmap.

googlebot · 2019-05-29T12:33:42Z

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of all the commit author(s), set the cla label to yes (if enabled on your project), and then merge this pull request when appropriate.

ℹ️ Googlers: Go here for more info.

josephburnett · 2019-05-29T12:39:28Z

@evankanderson can you manually add the cla:yes label please? @markusthoemmes just added some edits to my branch.

evankanderson · 2019-05-31T23:54:17Z

/cla yes

googlebot · 2019-05-31T23:55:07Z

A Googler has manually verified that the CLAs look good.

(Googler, please make sure the reason for overriding the CLA status is clearly documented in these comments.)

ℹ️ Googlers: Go here for more info.

josephburnett · 2019-06-03T06:53:12Z

@mattmoor can you lgtm?

evankanderson

/lgtm

evankanderson · 2019-06-04T00:10:36Z

/approve

knative-prow-robot · 2019-06-04T00:10:47Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: evankanderson, josephburnett

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [evankanderson]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Move to corev1.PodSpec now that vN-1 supports the containers field. (knative#4221) Previously we defined our own partial PodSpec because the corev1 version lacks `omitempty` and appears as `containers: null` in requests from generated clients, even if unspecified, which would have broken webhook validation. Now that the field has been out for a release, we can switch to the common PodSpec. Scaling Roadmap 2019 (knative#3040) * Scaling 2019 roadmap stub. * Descriptions for all 2019 goals. * Goals, POCs and Github projects for each. * Remove recap (will do later). * Remove indent. * Add Pluggability and HPA line item. * Yanwei as POC for layering. * Update docs/roadmap/scaling-2019.md Co-Authored-By: josephburnett <josephburnett79@gmail.com> * Update docs/roadmap/scaling-2019.md Co-Authored-By: josephburnett <josephburnett79@gmail.com> * Clarify overload handling for 0 and non-0 cases. * Refactor cold-start goal. * Remove POC. * Autoscaler scalability. * More edits. * HPA Interation. * Minor edits. * Propose section on migration K8s Deployments * Reworked parts of the Scaling roadmap. - Unified some wording (capitalization mostly). - Removed prescriptive key steps. These should be captured by the respective projects, which will be more dynamically changeable than this document. Enable setting the resource request/limits via annotations for queue-proxy side-car container (knative#4151) * Enable setting the resource request/limits via annotations for queue-proxy side-car container * Last PR comments * more * added integration tests * more * testfix * integrationtest * comments * integration test fix * PR comments * more * final * more pr comments * added error ErrInvalidValue * code coverage of queue.go Remove unused constants. (knative#4238) Update DEVELOPMENT.md (knative#4230) Auto TLS landed in v0.6, so this documentation is out of date golang format tools (knative#4241) Produced via: `gofmt -s -w $(find -path './vendor' -prune -o -type f -name '*.go' -print))` `goimports -w $(find -name '*.go' | grep -v vendor)` Move Metric interfaces into the general autoscaling package. (knative#4236) * Move Metric interfaces into the general autoscaling package. This used to be KPA specific but will soon be needed to be used by HPA resources as well to trigger metric collection. Decider interfaces and types stay KPA specific. * Move the Metrics resource interface next to the metric implementation. * Move Deciders interface for consistency. Apply various fixes pointed out by staticcheck. (knative#4242) * Transform string(buf.Bytes()) to buf.String(). * Remove a bunch of unused code. * Fix error capitalization. * Fix issue with error overlapping. * Fix deprecated usage of Apps without version. * Fix file permission resolution. * Fix comparison to boolean. * Fix issue with variable never being used. * Remove unused conditionsets. * Fix error checks after fixing capitalization. * Remove unused values in performance tests. * Remove some more unused code. steadier state Format markdown (knative#4240) Produced via: `prettier --write --prose-wrap=always $(find -name '*.md' | grep -v vendor | grep -v .github)` Drop DeprecatedName from service_test.go (knative#4243) some junk things work

* Scaling 2019 roadmap stub. * Descriptions for all 2019 goals. * Goals, POCs and Github projects for each. * Remove recap (will do later). * Remove indent. * Add Pluggability and HPA line item. * Yanwei as POC for layering. * Update docs/roadmap/scaling-2019.md Co-Authored-By: josephburnett <josephburnett79@gmail.com> * Update docs/roadmap/scaling-2019.md Co-Authored-By: josephburnett <josephburnett79@gmail.com> * Clarify overload handling for 0 and non-0 cases. * Refactor cold-start goal. * Remove POC. * Autoscaler scalability. * More edits. * HPA Interation. * Minor edits. * Propose section on migration K8s Deployments * Reworked parts of the Scaling roadmap. - Unified some wording (capitalization mostly). - Removed prescriptive key steps. These should be captured by the respective projects, which will be more dynamically changeable than this document.

josephburnett added 4 commits January 15, 2019 21:31

Scaling 2019 roadmap stub.

c111ca0

Descriptions for all 2019 goals.

15a93c2

Goals, POCs and Github projects for each.

03bbda8

Remove recap (will do later).

b853f33

josephburnett added the area/autoscale label Jan 30, 2019

knative-prow-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 30, 2019

knative-prow-robot requested review from evankanderson and mattmoor January 30, 2019 17:21

Remove indent.

ca07b8a

markusthoemmes reviewed Jan 31, 2019

View reviewed changes

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

josephburnett added 2 commits February 1, 2019 20:24

Add Pluggability and HPA line item.

e9c4c1a

Yanwei as POC for layering.

4a3ca3f

knative-prow-robot assigned mattmoor Feb 1, 2019

josephburnett assigned markusthoemmes Feb 1, 2019

knative-prow-robot assigned greghaynes Feb 6, 2019

knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 6, 2019

mattmoor-sockpuppet reviewed Feb 6, 2019

View reviewed changes

vvraskin reviewed Feb 8, 2019

View reviewed changes

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

Update docs/roadmap/scaling-2019.md

a128125

Co-Authored-By: josephburnett <josephburnett79@gmail.com>

knative-prow-robot removed the lgtm Indicates that a PR is ready to be merged. label Feb 8, 2019

Update docs/roadmap/scaling-2019.md

6937081

Co-Authored-By: josephburnett <josephburnett79@gmail.com>

mattmoor-sockpuppet reviewed Feb 8, 2019

View reviewed changes

Clarify overload handling for 0 and non-0 cases.

f9ef30a

mattmoor reviewed Feb 12, 2019

View reviewed changes

dprotaso reviewed Feb 19, 2019

View reviewed changes

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

dprotaso reviewed Feb 19, 2019

View reviewed changes

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

dprotaso reviewed Feb 19, 2019

View reviewed changes

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

Propose section on migration K8s Deployments

e34c056

Merge pull request #4 from bbrowning/migration-roadmap2019

4806ce0

Propose section on migration K8s Deployments

evankanderson added the cla: yes Indicates the PR's author has signed the CLA. label Mar 28, 2019

markusthoemmes reviewed Apr 3, 2019

View reviewed changes

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

mattmoor reviewed Apr 3, 2019

View reviewed changes

docs/roadmap/scaling-2019.md Outdated Show resolved Hide resolved

markusthoemmes and others added 2 commits May 29, 2019 14:09

Reworked parts of the Scaling roadmap.

df43dc3

- Unified some wording (capitalization mostly). - Removed prescriptive key steps. These should be captured by the respective projects, which will be more dynamically changeable than this document.

Merge pull request #5 from markusthoemmes/roadmap2019

ecb59d7

Reworked parts of the Scaling roadmap.

googlebot added cla: no Indicates the PR's author has not signed the CLA. and removed cla: yes Indicates the PR's author has signed the CLA. labels May 29, 2019

evankanderson added cla: yes Indicates the PR's author has signed the CLA. and removed cla: no Indicates the PR's author has not signed the CLA. labels May 31, 2019

evankanderson reviewed Jun 4, 2019

View reviewed changes

knative-prow-robot assigned evankanderson Jun 4, 2019

knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 4, 2019

knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 4, 2019

knative-prow-robot merged commit aa302e9 into knative:master Jun 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scaling Roadmap 2019 #3040

Scaling Roadmap 2019 #3040

josephburnett commented Jan 30, 2019

josephburnett commented Feb 1, 2019 •

edited

Loading

greghaynes commented Feb 6, 2019

mattmoor Feb 12, 2019

josephburnett commented Mar 27, 2019

bbrowning commented Mar 28, 2019

googlebot commented Mar 28, 2019

bbrowning commented Mar 28, 2019

googlebot commented Mar 28, 2019

markusthoemmes left a comment

googlebot commented May 29, 2019

josephburnett commented May 29, 2019

evankanderson commented May 31, 2019

googlebot commented May 31, 2019

josephburnett commented Jun 3, 2019

evankanderson left a comment

evankanderson commented Jun 4, 2019

knative-prow-robot commented Jun 4, 2019


		The goal of Overload Handling is to enqueue requests at a revision-level. Scale-from-zero should not overload if autoscaling can react in a reasonable amount of time to provide additional pods. When new pods come online, they should be able to take load for the existing pods. Even when scaled above zero, brief spikes of overload should be handled by enqueuing requests at a revision-level. The depth of the revision-level queue should also be configurable because even the Revision as a whole needs to guard against overload.

		The overall problem touches on both Networking and Autoscaling, two different working groups. Much of the overload handling will be implemented in the Activator, which is a part of ingress. So this project is shared jointly between the two working groups.

Scaling Roadmap 2019 #3040

Scaling Roadmap 2019 #3040

Conversation

josephburnett commented Jan 30, 2019

josephburnett commented Feb 1, 2019 • edited Loading

greghaynes commented Feb 6, 2019

mattmoor Feb 12, 2019

Choose a reason for hiding this comment

josephburnett commented Mar 27, 2019

bbrowning commented Mar 28, 2019

googlebot commented Mar 28, 2019

bbrowning commented Mar 28, 2019

googlebot commented Mar 28, 2019

markusthoemmes left a comment

Choose a reason for hiding this comment

googlebot commented May 29, 2019

josephburnett commented May 29, 2019

evankanderson commented May 31, 2019

googlebot commented May 31, 2019

josephburnett commented Jun 3, 2019

evankanderson left a comment

Choose a reason for hiding this comment

evankanderson commented Jun 4, 2019

knative-prow-robot commented Jun 4, 2019

josephburnett commented Feb 1, 2019 •

edited

Loading