Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(db): only enable postgres cleanup timer on nodes that have admin listeners #10405

Merged
merged 1 commit into from
Mar 1, 2023

Conversation

bungle
Copy link
Member

@bungle bungle commented Mar 1, 2023

Summary

PR #10389 and #10331 pursued different ways to improve our situation with cleanup timers.

This PR makes it so that it only happens on nodes that have admin listeners. Fairly simple but may already cover the majority of problem space.

@bungle
Copy link
Member Author

bungle commented Mar 1, 2023

Perhaps alternative to #10389 and #10331 is this PR. We can create additional PR that reduces the deleted row count to say 50000 like proposed in mentioned PRs.

… listeners

### Summary

PR #10389 and #10331 pursued different ways to improve our situation with cleanup timers.

This PR makes it so that it only happens on nodes that have admin listeners. Fairly simple
but may already cover the majority of problem space.
@hanshuebner hanshuebner merged commit 5527939 into master Mar 1, 2023
@hanshuebner hanshuebner deleted the chore/pg-cleanup-timer branch March 1, 2023 12:21
bungle added a commit that referenced this pull request Mar 1, 2023
### Summary

The PR #10405 changed cleanup to only happen on nodes that have Admin API listeners.

This PR makes deletion to happen in maximum of 50.000 row batches.

Inspired from #10331 and #10389.
@@ -315,8 +315,7 @@ end


function _mt:init_worker(strategies)
if ngx.worker.id() == 0 then

if ngx.worker.id() == 0 and #kong.configuration.admin_listeners > 0 then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Emmm, should this not instead check if the node doesn't have proxy listeners?
Thinking traditional mode.

cc @bungle @hanshuebner

hanshuebner pushed a commit that referenced this pull request Mar 3, 2023
* feat(db): batch cleanup expired rows from postgres

### Summary

The PR #10405 changed cleanup to only happen on nodes that have Admin API listeners.

This PR makes deletion to happen in maximum of 50.000 row batches.

Inspired from #10331 and #10389.

* change to batch delete on every table

* update changelog

* add test for ttl cleanup

---------

Co-authored-by: windmgc <windmgc@gmail.com>
@bungle bungle changed the title chore(db): only enable posgres cleanup timer on nodes that have admin listeners chore(db): only enable postgres cleanup timer on nodes that have admin listeners Mar 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants