Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Remove flush loop and queue from Ingester RF-1 #13538

Merged
merged 1 commit into from
Jul 17, 2024

Conversation

grobinson-grafana
Copy link
Contributor

@grobinson-grafana grobinson-grafana commented Jul 16, 2024

What this PR does / why we need it:

This commit removes the flush loop and flush queue from Ingester RF-1. This code is from the original ingester code, and is no longer needed since we have the WAL Manager.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@grobinson-grafana grobinson-grafana self-assigned this Jul 16, 2024
@grobinson-grafana grobinson-grafana requested a review from a team as a code owner July 16, 2024 11:37
@@ -141,7 +145,6 @@ func NewManager(cfg Config, metrics *Metrics) (*Manager, error) {
metrics: metrics.ManagerMetrics,
available: list.New(),
pending: list.New(),
shutdown: make(chan struct{}),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed as it wasn't used.

m.mu.Lock()
defer m.mu.Unlock()
m.closed = true
if m.available.Len() > 0 {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is duplicate code. I will refactor it in a follow up PR. But I wanted to show how it works here.

@grobinson-grafana grobinson-grafana force-pushed the grobinson/remove-loop-and-queue branch from 414c32e to a338260 Compare July 16, 2024 12:09
This commit removes the flush loop and flush queue from Ingester RF-1.
This code is from the original ingester code, and is no longer needed
since we have the WAL Manager.
@grobinson-grafana grobinson-grafana force-pushed the grobinson/remove-loop-and-queue branch from a338260 to cfaa574 Compare July 16, 2024 12:30
op := o.(*flushOp)

if it == nil {
// TODO: Do something more clever here instead.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was concerned this would increase CPU usage, but having tested it in dev it doesn't have much effect at all.

Copy link
Contributor

@cyriltovena cyriltovena Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think next pending should be blocking instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about it too! I'll take a look at how to use sync.Cond to achieve this.

Copy link
Contributor

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

Copy link
Contributor

@benclive benclive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, LGTM!

@grobinson-grafana grobinson-grafana merged commit 8ca03a2 into main Jul 17, 2024
60 checks passed
@grobinson-grafana grobinson-grafana deleted the grobinson/remove-loop-and-queue branch July 17, 2024 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants