Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(pageserver) remove task_mgr for most global tasks #8449

Merged
merged 6 commits into from
Jul 22, 2024

Conversation

problame
Copy link
Contributor

@problame problame commented Jul 21, 2024

Motivation & Context

We want to move away from task_mgr towards explicit tracking of child tasks.

This PR is extracted from #8339 where I refactor PageRequestHandler to not depend on task_mgr anymore.

Changes

This PR refactors all global tasks but PageRequestHandler to use some combination of JoinHandle/JoinSet + CancellationToken.

The task_mgr::spawn(.., shutdown_process_on_error) functionality is preserved through the new exit_on_panic_or_error wrapper.
Some global tasks were not using it before, but as of this PR, they are.
The rationale is that all global tasks are relevant for correct operation of the overall Neon system in one way or another.

Future Work

After #8339, we can make task_mgr::spawn require a TenantId instead of an Option<TenantId> which concludes this step of cleanup work and will help discourage future usage of task_mgr for global tasks.

Notes for Reviewers

The individual commits are mostly orthogonal but it's too painful to stack them in a series of PRs.
I would recommend reviewing commit-by-commit.

Copy link

github-actions bot commented Jul 21, 2024

3144 tests run: 3023 passed, 0 failed, 121 skipped (full report)


Code coverage* (full report)

  • functions: 32.6% (7003 of 21475 functions)
  • lines: 50.0% (55249 of 110491 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
fbf320b at 2024-07-22T11:35:20.953Z :recycle:

@problame problame requested review from jcsp and koivunej July 21, 2024 19:52
@problame problame marked this pull request as ready for review July 21, 2024 19:53
@problame problame requested a review from a team as a code owner July 21, 2024 19:53
@koivunej koivunej mentioned this pull request Jul 22, 2024
@problame problame merged commit e852301 into main Jul 22, 2024
65 checks passed
@problame problame deleted the problame/less-task-mgr branch July 22, 2024 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants