rate limit auto-tuning #1957
rate limit auto-tuning #1957
Conversation
|
@ajkr updated the pull request - view changes |
|
@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
@ajkr updated the pull request - view changes - changes since last import |
|
@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
Is it ready for code review? |
|
Thanks for checking. I think let's deprioritize this one as I don't have a use case yet or intent to benchmark it in the near future. We can focus on #2027 instead as it seems more likely to help write stall issues. |
|
@siying, do you mind looking at the API for configuring auto-tuning objects and giving any feedback? I plan to play with the rate limit calculation a bit, but the other stuff in this diff might be good to review earlier. |
|
@ajkr updated the pull request - view changes - changes since last import |
|
@ajkr updated the pull request - view changes - changes since last import |
|
@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
@ajkr updated the pull request - view changes - changes since last import |
|
@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
| // Copyright (c) 2017-present, Facebook, Inc. All rights reserved. | ||
| // This source code is licensed under the BSD-style license found in the | ||
| // LICENSE file in the root directory of this source tree. An additional grant | ||
| // of patent rights can be found in the PATENTS file in the same directory. |
There was a problem hiding this comment.
You might also need to add a few more lines here like the ones added in #2226 .
|
@ajkr updated the pull request - view changes - changes since last import |
|
@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
|
@ajkr updated the pull request - view changes - changes since last import |
|
replaced with #2899 |
try a simple demand-based dynamic rate limiter. as commented, it adjusts rate limit upwards when demand is consistently high, and downwards when demand is consistently low. what we really should do is estimate the rate needed to prevent L0 from filling up, but let's try something even simpler for the first stab.
Test Plan: tried in db_bench and watched it increase/decrease rate limit. need help coming up with an evaluation plan