make message rate configurable, including burst mode #908
Conversation
This adds several knobs that control how fast Sopel will send multiple messages to the same recipient: - `bucket_burst_tokens` -- this controls how many messages may be sent in "burst mode", i.e., without any inter-message delay. - `bucket_refill_rate` -- once exhausted, burst tokens are refilled at a rate of `bucket_refill_rate` tokens/second. - `bucket_empty_wait` -- how long to wait between sending messages when not in burst mode. This permis a bit to return a few lines of information quickly, while still engaging rate limiting to prevent flood protection from kicking in. How it works: When sending to a new recipient, we initialize a token counter to `bucket_burst_tokens`. Every time we send a message, we decrement this by 1. If the token counter reaches 0, we engage the rate limiting behavior (which is identical to the existing code, except that the minimum wait of 0.7 seconds is now configurable). When sending a new message to an existing recipient, we check if the token counter is exhausted. If it is, we refill it based on the elapsed time since the last message and the value of `bucket_refill_rate`, and then proceed as described above.
|
Using this change on my test bot -- works very well, have yet to encounter any issues. |
|
If you're still active, could you resolve the new merge conflicts that have popped up? +1 |
|
I'll take a look. It looks as if in this case "merge conflict" means "all the code my patch touched has been relocated". |
|
I'm really hesitant to touch the rate limiting code without doing some serious testing on it first. This is getting discussed in #952. It looks like this is a replacement of the existing PRIVMSG limit system following that model, which is good. When I get the chance, I'd like to run some stress tests with it on a bunch of networks to see how it holds up in practice. I'm definitely okay with merging this system for just PRIVMSG now, and then figuring out how to pull it in for everything else later. However, I don't really like having configs for these internals. There's no way for a user to know what to put for them, whether they've set it right, and whether their configuration will lead to unexpected and confusing errors later on. Moreover, I think it's highly unlikely that (assuming the stress tests work out with the defaults), anyone would ever really change these (let alone to something that wouldn't be quietly broken). |
|
I'm closing this old PR. Cheers! |
|
@larsks Would you like to continue working on this, or should someone else take over? This commit (f58fa9f) could be @deathbybandaid's starting point for #1342, if you're done with it. (Anything built on this will include the existing commit & authorship info, I'll make sure of it. 😸) If you're not done with it, feel free to reopen. I took over maintaining Sopel a few months ago, so if you closed this because it stalled, it can be un-stalled now. |
|
I'm going to reopen this just so we don't lose track of it, since closed PRs basically disappear into a black hole unless you know exactly what you're looking for. If someone wants to replace/extend it, we'll label this appropriately and close it again at that time. |
Original change description by @larsks (from sopel-irc#908, slightly edited): This adds several knobs that control how fast the bot sends multiple messages to the same recipient: - bucket_burst_tokens -- this controls how many messages may be sent in "burst mode", i.e., without any inter-message delay. - bucket_refill_rate -- once exhausted, burst tokens are refilled at a rate of bucket_refill_rate tokens/second. - bucket_empty_wait -- how long to wait between sending messages when not in burst mode. This permits the bot to return a few lines of information quickly and not trigger flood protection. How it works: When sending to a new recipient, we initialize a token counter to bucket_burst_tokens. Every time we send a message, we decrement this by 1. If the token counter reaches 0, we engage the rate limiting behavior (which is identical to the existing code, except that the minimum wait of 0.7 seconds is now configurable). When sending a new message to an existing recipient, we check if the token counter is exhausted. If it is, we refill it based on the elapsed time since the last message and the value of bucket_refill_rate, and then proceed as described above. ---- Adapted to current Sopel code and rebased by these users, respectively: Co-authored-by: kwaaak <kwaaak@users.noreply.github.com> Co-authored-by: dgw <dgw@technobabbl.es>
|
I think this one could be closed since #1518 takes care of this. |
|
Definitely. I just forgot to close it and add the "Replaced" label. |
Original change description by @larsks (from sopel-irc#908, slightly edited): This adds several knobs that control how fast the bot sends multiple messages to the same recipient: - bucket_burst_tokens -- this controls how many messages may be sent in "burst mode", i.e., without any inter-message delay. - bucket_refill_rate -- once exhausted, burst tokens are refilled at a rate of bucket_refill_rate tokens/second. - bucket_empty_wait -- how long to wait between sending messages when not in burst mode. This permits the bot to return a few lines of information quickly and not trigger flood protection. How it works: When sending to a new recipient, we initialize a token counter to bucket_burst_tokens. Every time we send a message, we decrement this by 1. If the token counter reaches 0, we engage the rate limiting behavior (which is identical to the existing code, except that the minimum wait of 0.7 seconds is now configurable). When sending a new message to an existing recipient, we check if the token counter is exhausted. If it is, we refill it based on the elapsed time since the last message and the value of bucket_refill_rate, and then proceed as described above. ---- Adapted to current Sopel code and rebased by these users, respectively: Co-authored-by: kwaaak <kwaaak@users.noreply.github.com> Co-authored-by: dgw <dgw@technobabbl.es>
Original change description by @larsks (from sopel-irc#908, slightly edited): This adds several knobs that control how fast the bot sends multiple messages to the same recipient: - bucket_burst_tokens -- this controls how many messages may be sent in "burst mode", i.e., without any inter-message delay. - bucket_refill_rate -- once exhausted, burst tokens are refilled at a rate of bucket_refill_rate tokens/second. - bucket_empty_wait -- how long to wait between sending messages when not in burst mode. This permits the bot to return a few lines of information quickly and not trigger flood protection. How it works: When sending to a new recipient, we initialize a token counter to bucket_burst_tokens. Every time we send a message, we decrement this by 1. If the token counter reaches 0, we engage the rate limiting behavior (which is identical to the existing code, except that the minimum wait of 0.7 seconds is now configurable). When sending a new message to an existing recipient, we check if the token counter is exhausted. If it is, we refill it based on the elapsed time since the last message and the value of bucket_refill_rate, and then proceed as described above. ---- Adapted to current Sopel code and rebased by these users, respectively: Co-authored-by: kwaaak <kwaaak@users.noreply.github.com> Co-authored-by: dgw <dgw@technobabbl.es>
This adds several knobs that control how fast Sopel will send multiple
messages to the same recipient:
bucket_burst_tokens-- this controls how many messages may be sentin "burst mode", i.e., without any inter-message delay.
bucket_refill_rate-- once exhausted, burst tokens are refilled at arate of
bucket_refill_ratetokens/second.bucket_empty_wait-- how long to wait between sending messages whennot in burst mode.
This permits a bot to return a few lines of information quickly, while
still engaging rate limiting to prevent flood protection from kicking
in.
How it works:
When sending to a new recipient, we initialize a token counter to
bucket_burst_tokens. Every time we send a message, we decrement thisby 1. If the token counter reaches 0, we engage the rate limiting
behavior (which is identical to the existing code, except that the
minimum wait of 0.7 seconds is now configurable).
When sending a new message to an existing recipient, we check if the
token counter is exhausted. If it is, we refill it based on the elapsed
time since the last message and the value of
bucket_refill_rate, andthen proceed as described above.