Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update DNS programming latency SLI #7756

Merged
merged 3 commits into from
Mar 14, 2024
Merged

update DNS programming latency SLI #7756

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
434c37c
update DNS programming latency SLI
aojea Mar 13, 2024
f9b6920
Update sig-scalability/slos/dns_programming_latency.md
aojea Mar 13, 2024
9e43c29
Update sig-scalability/slos/dns_programming_latency.md
aojea Mar 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion sig-scalability/slos/dns_programming_latency.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@ from all of them.
DNS will start resolving service name to its newly started backends.
- As a user of vanilla Kubernetes, I want some guarantee how quickly in-cluster
DNS will stop resolving service name to its removed (or unhealthy) backends.
- As a user of vanilla Kubernetes, I wasn some guarantee how quickly newly
- As a user of vanilla Kubernetes, I want some guarantee how quickly newly
create services will be resolvable via in-cluster DNS.
- As a user of vanilla Kubernetes, I want some guarantee how quickly in-cluster
DNS will start resolving headless service hostnames to its newly started backends.

### Other notes
- We are consciously focusing on in-cluster DNS for the purpose of this SLI,
Expand All @@ -37,6 +39,10 @@ The reason for doing it this way is feasibility for efficiently computing that:
in 99% of programmers (e.g. iptables). That requires tracking metrics on
per-change base (which we can't do efficiently).

- The SLI is expected to remain constant independently of the number of records, per
Copy link
Member

@thockin thockin Mar 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the implication of this? Scheduler has a finite throughput. Nodes have finite bandwidth.

If I start a 5 pod headless-service, it's reasonable to expect that DNS for the 5th pod is very very soon after DNS for the 1st:
t0: scale RS to 5
t1: RS controller creates pod 1
t2: scheduler schedules pod 1
t3: kubelet downloads image
t4: kubelet runs pod
t5: runtime assigns an IP
t6: kubelet reports the IP
t7: endpointslice controller observes IP and updates EPSlices
t8: DNS observes EPSlices and updates DNS

t1 - t8 happen 5 times, roughly concurrently, and is likely bounded by image download time.

Change that to 5000 and now you are bounded by the scheduler's throughput. Is DNS not allowed to publish the 1st IP until the last pod is started? How does it know which one is last?

Edit:

Or did you mean something like "The time between pod-started-and-IP-assigned and availble-in-DNS should not be significantly different for the 1st vs. last pod" ? That must be what you meant...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"The time between pod-started-and-IP-assigned and availble-in-DNS should not be significantly different for the 1st vs. last pod" ? That must be what you meant...

this, if you give me the right sentence in english so this is more clear please add it as a suggestion

example, in a headless service with thousands of pods the SLI for the first and the
last Pod should not exhibit a statistically significant difference.

### How to measure the SLI.
There [network programming latency](./network_programming_latency.md) is
formulated in almost exactly the same way. As a result, the methodology for
Expand Down