Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Channel that doesn't remove the oldest messages when receiver buffer is full #73

Closed
ela-kotulska-frequenz opened this issue Feb 17, 2023 · 2 comments
Labels
part:channels Affects channels implementation resolution:duplicate This issue or pull request already exists type:enhancement New feature or enhancement visitble to users

Comments

@ela-kotulska-frequenz
Copy link
Contributor

ela-kotulska-frequenz commented Feb 17, 2023

What's needed?

Now when receiver buffer size is full and new message comes, then the oldest message is removed.
But in some cases it is important that each request is received and processed.
One such use-case is when we request to subscribe for metric data.
If we have many components/metrics and we subscribe for all data at the startup, then default receiver buffer size is not enough. The oldest messages are removed and the application doesn't work properly because we don't receive data.

Proposed solution

There are two possible solutions:

  1. We could have special kind of blocking channel.
    If sender tries to send message, but the receiver buffer size is full, then we should block sender until the receiver buffer size has free space. For implementation simplicity it is enough if channel have many senders and only one receiver. (this covers our all current use-cases)

  2. We could have option to dynamically increase the buffer size - but in that case there might be more and more requests, and receiver is unable to process them in time and the memory might increase too much.

Use cases

For now the only use-case was subscribing for the component metric data.

Alternatives and workarounds

No response

Additional context

No response

@ela-kotulska-frequenz ela-kotulska-frequenz added priority:❓ We need to figure out how soon this should be addressed type:enhancement New feature or enhancement visitble to users part:channels Affects channels implementation labels Feb 17, 2023
@ela-kotulska-frequenz ela-kotulska-frequenz changed the title Channel that doesn't blocks when receiver buffer is full. Channel that doesn't remove the oldest messages when receiver buffer is full. Feb 17, 2023
@ela-kotulska-frequenz ela-kotulska-frequenz changed the title Channel that doesn't remove the oldest messages when receiver buffer is full. Channel that doesn't remove the oldest messages when receiver buffer is full Feb 17, 2023
@leandro-lucarella-frequenz
Copy link
Contributor

leandro-lucarella-frequenz commented Feb 17, 2023

Option 1 is #20, right?

Option 2 IMHO doesn't make a lot of sense, if you need a bigger buffer, just use a bigger buffer. If you mean an infinite buffer, I also don't think is a good idea.

@llucax
Copy link
Contributor

llucax commented Aug 7, 2023

I will close this as a duplicate of #20 as I don't think option 2 is realistic.

@llucax llucax closed this as not planned Won't fix, can't repro, duplicate, stale Aug 7, 2023
@github-project-automation github-project-automation bot moved this from To do to Done in Python SDK Roadmap Aug 7, 2023
@llucax llucax added resolution:duplicate This issue or pull request already exists and removed priority:❓ We need to figure out how soon this should be addressed labels Aug 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
part:channels Affects channels implementation resolution:duplicate This issue or pull request already exists type:enhancement New feature or enhancement visitble to users
Projects
Development

No branches or pull requests

3 participants