Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(common): LocalSystemParamManager for worker node #8153

Merged
merged 7 commits into from
Feb 24, 2023

Conversation

Gun9niR
Copy link
Contributor

@Gun9niR Gun9niR commented Feb 23, 2023

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

System params on worker nodes have 2 read patterns:

  • Simply read the latest value
  • Need to be notified of the change and do other logic, such as reset tick in loop, etc.

LocalSystemParamManager provides 2 interfaces for them respectively

  • get_params
  • subscribe_params

Checklist For Contributors

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • All checks passed in ./risedev check (or alias, ./risedev c)

Checklist For Reviewers

  • I have requested macro/micro-benchmarks as this PR can affect performance substantially, and the results are shown.

Documentation

  • My PR DOES NOT contain user-facing changes.
Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

  • Installation and deployment
  • Connector (sources & sinks)
  • SQL commands, functions, and operators
  • RisingWave cluster configuration changes
  • Other (please specify in the release note below)

Release note

@codecov
Copy link

codecov bot commented Feb 23, 2023

Codecov Report

Merging #8153 (c0538f1) into main (3735a4f) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #8153   +/-   ##
=======================================
  Coverage   71.63%   71.64%           
=======================================
  Files        1132     1133    +1     
  Lines      182213   182249   +36     
=======================================
+ Hits       130536   130575   +39     
+ Misses      51677    51674    -3     
Flag Coverage Δ
rust 71.64% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/common/src/system_param/mod.rs 96.42% <ø> (ø)
src/common/src/system_param/local_manager.rs 100.00% <100.00%> (ø)
src/common/src/system_param/reader.rs 57.89% <100.00%> (+2.63%) ⬆️
src/stream/src/executor/aggregation/minput.rs 96.35% <0.00%> (+0.10%) ⬆️
src/storage/src/hummock/compactor/mod.rs 80.88% <0.00%> (+0.19%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more


impl LocalSystemParamManager {
pub fn new(params: SystemParamsReader) -> Self {
let params = Arc::new(ArcSwap::from_pointee(params));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using Arc alone should be enough here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the latest params arrive at the worker though notification service, the local manager needs a thread-safe way to update params, but Arc does not provide that functionality 🤔

Copy link
Member

@BugenZhao BugenZhao Feb 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will the subscribers get the param updates, through the watch channel to get the updates or simply hold a Arc<ArcSwap>? For the latter way, this sounds reasonable to me.

Oh, the PR body has elaborated on this. 😄

src/common/src/system_param/local_manager.rs Outdated Show resolved Hide resolved

impl LocalSystemParamManager {
pub fn new(params: SystemParamsReader) -> Self {
let params = Arc::new(ArcSwap::from_pointee(params));
Copy link
Member

@BugenZhao BugenZhao Feb 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will the subscribers get the param updates, through the watch channel to get the updates or simply hold a Arc<ArcSwap>? For the latter way, this sounds reasonable to me.

Oh, the PR body has elaborated on this. 😄

@mergify mergify bot merged commit b640cbd into main Feb 24, 2023
@mergify mergify bot deleted the zhidong/local-param-manager branch February 24, 2023 06:30
@Gun9niR Gun9niR mentioned this pull request Mar 3, 2023
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants