Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add timeout to requests to nodes and add metric for iter cnts #883

Merged
merged 10 commits into from
Oct 10, 2024

Conversation

ppca
Copy link
Contributor

@ppca ppca commented Oct 10, 2024

No description provided.

@volovyks volovyks linked an issue Oct 10, 2024 that may be closed by this pull request
Copy link
Collaborator

@volovyks volovyks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good finding!
What is the expected behaviour now? From what I see, instead of being stuck, we will start having relatively big delayes. Especially in /msg endpoint. Can we aford that? One unresponsive node can slow things down.

@@ -398,6 +398,15 @@ pub(crate) static SIGNATURE_PUBLISH_RESPONSE_ERRORS: Lazy<CounterVec> = Lazy::ne
.unwrap()
});

pub(crate) static PROTOCOL_LATENCY_ITER_CNT: Lazy<CounterVec> = Lazy::new(|| {
try_create_counter_vec(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to measure latency or count iterations? Let's rename this accordingly and use counter if it's the later one.

ChaoticTempest
ChaoticTempest previously approved these changes Oct 10, 2024
@ppca
Copy link
Contributor Author

ppca commented Oct 10, 2024

Good finding! What is the expected behaviour now? From what I see, instead of being stuck, we will start having relatively big delayes. Especially in /msg endpoint. Can we aford that? One unresponsive node can slow things down.

That is a timeout, if the request finish before 5s, it will return much sooner than 5s. So it unstuck when /msg is not available, and it will be as fast if /msg is working properly.

@ppca ppca merged commit c08f6f0 into develop Oct 10, 2024
3 checks passed
@ppca ppca deleted the xiangyi/verify_log branch October 10, 2024 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Protocol stuck debug
4 participants