Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement max_concurrency support #2650

Closed
wants to merge 4 commits into from

Conversation

GnomedDev
Copy link
Member

@GnomedDev GnomedDev commented Dec 5, 2023

Implements support for the max concurrency system. This allows bots on very large sharding to start up multiple shards in parallel to cut startup times down quite a bit. I've personally tested this using my bot with 256 shards and a max_concurrency of 16 and it worked flawlessly.

@github-actions github-actions bot added model Related to the `model` module. client Related to the `client` module. gateway Related to the `gateway` module. labels Dec 5, 2023
@mkrasnitski
Copy link
Collaborator

Might be better to separate concerns to make sure the rework is reviewed separately.

@GnomedDev GnomedDev marked this pull request as draft December 5, 2023 10:18
@GnomedDev
Copy link
Member Author

Done, and drafted until that PR is merged.

@GnomedDev GnomedDev force-pushed the max-concurrency branch 2 times, most recently from f031098 to f2d32d2 Compare December 5, 2023 10:27
@mkrasnitski
Copy link
Collaborator

This was discussed in the discord, but basically: When creating a batch of shards, to ensure both maximum batch size and avoid being ratelimited, their ratelimit keys should all be unique (calculated via shard_id % max_concurrency). That means that if a shard fails to start, when including it in the next batch for retry, it should replace a shard with the same rate limit key, in order to make sure the batch doesn't include any shards which are ratelimited together.

@GnomedDev
Copy link
Member Author

@mkrasnitski is going to take over this effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client Related to the `client` module. gateway Related to the `gateway` module. model Related to the `model` module.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants