Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[server][dvc] Add Blob Transfer Related Metrics #1352

Merged
merged 11 commits into from
Dec 4, 2024

Conversation

jingy-li
Copy link
Contributor

@jingy-li jingy-li commented Nov 27, 2024

[server][dvc] Add Blob Transfer Related Metrics

This PR introduces 5 metrics to track blob transfer activity as a client side:

As a client side:

  • blob_transfer_total_num_responses: The total number of responses received.
  • blob_transfer_successful_num_responses: The number of successful responses received, which also indicates how many partitions were successfully bootstrapped via blob transfer.
  • blob_transfer_failed_num_responses: The number of failed responses, which also indicates how many partitions failed to bootstrap via blob transfer.
  • blob_transfer_file_receive_throughput: The file receive rate, measured in MB/sec.
  • blob_transfer_time: The total file transfer time per partition, measured in seconds.

How was this PR tested?

Testing with private build at testing hosts.

Does this PR introduce any user-facing changes?

  • No. You can skip the rest of this section.
  • Yes. Make sure to explain your proposed changes and call out the behavior change.

Copy link
Contributor

@sixpluszero sixpluszero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks ok, left one comment and will need to fix the coverage issue. and also synced offline, we agreed that due to the short period of blob transfer in some cases, it is ok to use gauge to set the throughput and count for other metrics, otherwise it may get averaged out or missed.

@jingy-li jingy-li requested a review from sixpluszero December 4, 2024 18:03
Copy link
Contributor

@sixpluszero sixpluszero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you!

@jingy-li jingy-li merged commit 9a03934 into linkedin:main Dec 4, 2024
51 checks passed
@jingy-li jingy-li deleted the add-blob-transfer-related-metrics branch December 4, 2024 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants