Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ingest storage: proper shutdown of partitionCommitter #9436

Merged

Commits on Sep 26, 2024

  1. ingest storage: proper shutdown of partitionCommitter

    This `partitionCommitter` would be shut down via the services manager as soon as the service context is cancelled. This means that they shut down in parallel with the `PartitionReader`. The race comes when the `partitionCommitter` has already shut down while the `PartitionReader` is still processing some records. Then when the `PartitionReader` tries to `enqueueCommit`, that sets the atomic, but does not send this to Kafka.
    
    As a result we may not always persist the latest commit to Kafka.
    
    Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
    dimitarvdimitrov committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    01d32ac View commit details
    Browse the repository at this point in the history