Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change MaximumClientWritesPending to 64 #461

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

lneoe
Copy link

@lneoe lneoe commented Mar 10, 2025

Adjust MaximumClientWritesPending default value: from 1024*8 to 64

  • Current default value is too high, causing significant memory overhead when maintaining a large number of connections (tens of thousands)
  • Testing shows that reducing this value significantly decreases memory usage, optimizing resource utilization

Technical Impact

  • Minimal impact on small-scale deployments
  • Significantly improves memory usage in high-concurrency scenarios
  • No effect on core functionality and performance

@thedevop
Copy link
Collaborator

@lneoe, for my use case (also large number of connections), I also set this to a much smaller value (128) than the default. However, many do not necessary use in environments with large number of connections. For those use cases, it is preferable to have a higher value to prevent possible loss of delivery.

@lneoe
Copy link
Author

lneoe commented Mar 13, 2025

@lneoe, for my use case (also large number of connections), I also set this to a much smaller value (128) than the default. However, many do not necessary use in environments with large number of connections. For those use cases, it is preferable to have a higher value to prevent possible loss of delivery.

Yes, larger values can prevent message loss, but this consideration is primarily necessary in scenarios with very frequent message transmission and relatively slow consumption rates.
In my actual application scenario, the server maintains approximately 33,000 simultaneous connections, consuming about 5GB of memory that never gets garbage collected. Memory consumption became a bottleneck, preventing the server from creating additional connections. After reducing this value, we now support 50,000 simultaneous connections while consuming only about 2.8GB of memory. Of course, 64 isn't appropriate for every scenario, but the original default value was indeed too large and easily led to excessive memory growth.

@thedevop
Copy link
Collaborator

Are you using the broker in embedded mode? if so, you can consult #453 for additional server settings that can reduce memory usage.

@lneoe
Copy link
Author

lneoe commented Mar 14, 2025

Are you using the broker in embedded mode? if so, you can consult #453 for additional server settings that can reduce memory usage.

Yes, I'm using embedded mode, and I've also configured it through setting my own mqtt.Options. The main reason for submitting this PR is considering that such a large default value isn't necessary for most scenarios, and we often use default settings because they typically represent a more universal solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants