From f9e82d177d51501c35d0c01b140af600b13651ee Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Fri, 10 Aug 2018 12:07:03 -0700 Subject: [PATCH 1/2] Add details about supporting qps based sampler. --- trace/Sampling.md | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/trace/Sampling.md b/trace/Sampling.md index 3ff0c81..00aabcc 100644 --- a/trace/Sampling.md +++ b/trace/Sampling.md @@ -12,7 +12,11 @@ The Sampling bit is always set only at the start of a Span, using a `Sampler` * `AlwaysSample` - sampler that makes a "yes" decision every time. * `NeverSample` - sampler that makes a "no" decision every time. * `Probability` - sampler that tries to uniformly sample traces with a given probability. When -applied to a child `Span` of a **sampled** parent `Span`, the child `Span` keeps the sampling decision. +applied to a child `Span` of a **sampled** parent `Span`, the child `Span` keeps the sampling +decision. +* `QPS` - sampler that tries to sample traces based on QPS (e.g. 1 trace/second). When applied to a +child `Span` of a **sampled** parent `Span`, the child `Span` keeps the sampling decision. For +implementation details see [this](#qps-sampler-implementation-details) ### How can users control the Sampler that is used for sampling? There are 2 ways to control the `Sampler` used when the library samples: @@ -33,3 +37,26 @@ The OpenCensus library samples based on the following rules: 3. If the span is a child of a local `Span` the sampling decision will be: * If a "span-scoped" `Sampler` is provided, use it to determine the sampling decision. * Else keep the sampling decision from the parent. + +### QPS sampler implementation details +The problem we are trying to solve is: +1. Getting QPS based sampling. +2. Providing real sampling probabilities. +3. Minimal overhead. + +Idea is to store the time that we last made a QPS based sampling decision in an atomic. Then we can +use the elapsed time Z since the coin flip to weight our current coin flip. We choose our +probability function P(Z) such that we get the desired sample QPS. We want P(Z) to be very +cheap to compute. + +Let X be the desired QPS. Let Z be the elapsed time since the last sampling decision in seconds. +``` +P(Z) = min(Z * X, 1) +``` + +To see that this is approximately correct, consider the case where we have perfectly distributed +time intervals. Specifically, let X = 1 and Z = 1/N. Then we would have N coin flips per second, +each with probability 1/N, for an expectation of 1 sample per second. + +This will under-sample: consider the case where X = 1 and Z alternates between 0.5 and 1.5. It is +possible to get about 1 QPS by always sampling, but this algorithm only gets 0.75 QPS. From 3b0b7c3ef56bd526f7c59cde32caad9d38d7e336 Mon Sep 17 00:00:00 2001 From: Bogdan Drutu Date: Tue, 14 Aug 2018 17:10:57 -0700 Subject: [PATCH 2/2] Rename the qps to RateLimiting --- trace/Sampling.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/trace/Sampling.md b/trace/Sampling.md index 00aabcc..363e13f 100644 --- a/trace/Sampling.md +++ b/trace/Sampling.md @@ -14,9 +14,9 @@ The Sampling bit is always set only at the start of a Span, using a `Sampler` * `Probability` - sampler that tries to uniformly sample traces with a given probability. When applied to a child `Span` of a **sampled** parent `Span`, the child `Span` keeps the sampling decision. -* `QPS` - sampler that tries to sample traces based on QPS (e.g. 1 trace/second). When applied to a -child `Span` of a **sampled** parent `Span`, the child `Span` keeps the sampling decision. For -implementation details see [this](#qps-sampler-implementation-details) +* `RateLimiting` - sampler that tries to sample with a rate per time window (0.1 traces/second). +When applied to a child `Span` of a **sampled** parent `Span`, the child `Span` keeps the sampling +decision. For implementation details see [this](#ratelimiting-sampler-implementation-details) ### How can users control the Sampler that is used for sampling? There are 2 ways to control the `Sampler` used when the library samples: @@ -38,7 +38,7 @@ The OpenCensus library samples based on the following rules: * If a "span-scoped" `Sampler` is provided, use it to determine the sampling decision. * Else keep the sampling decision from the parent. -### QPS sampler implementation details +### RateLimiting sampler implementation details The problem we are trying to solve is: 1. Getting QPS based sampling. 2. Providing real sampling probabilities.