Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic metrics in storage service #872

Merged
merged 6 commits into from
Oct 12, 2019
Merged

Conversation

critical27
Copy link
Contributor

@critical27 critical27 commented Aug 30, 2019

  1. Add basic metrics for data operation in storage service
  2. Add a storage integrity test tool in storage-perf.

close #287

Now we support following metrics: get_bound, bound_stats, vertex_props, edge_props, add_vertex, add_edge, del_vertex, update_vertex, update_edge.

We can get qps or latency from these metrics (add suffix), for example: vertex_props_qps or vertex_props_latency.

You can get a single metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats=vertex_props_qps.rate.60

If you want get multiple metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats=vertex_props_qps.rate.60,vertex_props_latency.avg.60,vertex_props_latency.p99.60&returnjson"
We can get qps, avg latency and p99 latency in a minutue.

You can get all metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats"


The storage integrity tool is used for verifying data integrity in storage. It only works for single replica for now. You can use it like this:
./storage_integrity --meta_server_addrs=192.168.8.5:60001 --first_vertex_id=100 --width=1000 --height=1000

meta_server_addrs is the meta server addresss.
first_vertex_id is the first vertex id which we inserts.
width and height is the width and height of the big matrix.

@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing failed.

@critical27
Copy link
Contributor Author

Jenkins go, please don't abort - -!

@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

Copy link
Contributor

@laura-ding laura-ding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how to get the result of statsmanger? I have no see, will use Http?

src/storage/StorageStats.h Outdated Show resolved Hide resolved
@critical27
Copy link
Contributor Author

how to get the result of statsmanger? I have no see, will use Http?

Yeah, u r right

@nebula-community-bot
Copy link
Member

Unit testing passed.

@dangleptr
Copy link
Contributor

How to get the metrics? Could you describe it in the summary?

@nebula-community-bot
Copy link
Member

Unit testing passed.

dangleptr
dangleptr previously approved these changes Oct 10, 2019
Copy link
Contributor

@dangleptr dangleptr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the usage about StorageIntegrityTool

@nebula-community-bot
Copy link
Member

Unit testing failed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

Copy link
Contributor

@darionyaphet darionyaphet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Job 🎉

rebase on upstream/master

address @darionyaphet's comments
@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

Copy link
Contributor

@dangleptr dangleptr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dangleptr dangleptr merged commit 8436287 into vesoft-inc:master Oct 12, 2019
@critical27 critical27 deleted the stats branch October 12, 2019 08:40
whitewum pushed a commit to whitewum/nebula that referenced this pull request Oct 16, 2019
1. Add basic metrics for data operation in storage service
2. Add a storage integrity test tool in storage-perf.

close vesoft-inc#287

Now we support following metrics: get_bound, bound_stats, vertex_props, edge_props, add_vertex, add_edge, del_vertex, update_vertex, update_edge.

We can get qps or latency from these metrics (add suffix), for example: vertex_props_qps or vertex_props_latency.

You can get a single metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats=vertex_props_qps.rate.60

If you want get multiple metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats=vertex_props_qps.rate.60,vertex_props_latency.avg.60,vertex_props_latency.p99.60&returnjson"
We can get qps, avg latency and p99 latency in a minutue.

You can get all metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats"

The storage integrity tool is used for verifying data integrity in storage. It only works for single replica for now. You can use it like this:
./storage_integrity --meta_server_addrs=192.168.8.5:60001 --first_vertex_id=100 --width=1000 --height=1000

meta_server_addrs is the meta server addresss.
first_vertex_id is the first vertex id which we inserts.
width and height is the width and height of the big matrix.
@jude-zhu
Copy link
Contributor

relate to #185

yixinglu pushed a commit to yixinglu/nebula that referenced this pull request Feb 16, 2020
1. Add basic metrics for data operation in storage service
2. Add a storage integrity test tool in storage-perf.

close vesoft-inc#287

Now we support following metrics: get_bound, bound_stats, vertex_props, edge_props, add_vertex, add_edge, del_vertex, update_vertex, update_edge.

We can get qps or latency from these metrics (add suffix), for example: vertex_props_qps or vertex_props_latency.

You can get a single metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats=vertex_props_qps.rate.60

If you want get multiple metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats=vertex_props_qps.rate.60,vertex_props_latency.avg.60,vertex_props_latency.p99.60&returnjson"
We can get qps, avg latency and p99 latency in a minutue.

You can get all metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats"

The storage integrity tool is used for verifying data integrity in storage. It only works for single replica for now. You can use it like this:
./storage_integrity --meta_server_addrs=192.168.8.5:60001 --first_vertex_id=100 --width=1000 --height=1000

meta_server_addrs is the meta server addresss.
first_vertex_id is the first vertex id which we inserts.
width and height is the width and height of the big matrix.
tong-hao pushed a commit to tong-hao/nebula that referenced this pull request Jun 1, 2021
1. Add basic metrics for data operation in storage service
2. Add a storage integrity test tool in storage-perf.

close vesoft-inc#287

Now we support following metrics: get_bound, bound_stats, vertex_props, edge_props, add_vertex, add_edge, del_vertex, update_vertex, update_edge.

We can get qps or latency from these metrics (add suffix), for example: vertex_props_qps or vertex_props_latency.

You can get a single metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats=vertex_props_qps.rate.60

If you want get multiple metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats=vertex_props_qps.rate.60,vertex_props_latency.avg.60,vertex_props_latency.p99.60&returnjson"
We can get qps, avg latency and p99 latency in a minutue.

You can get all metrics like this:
curl -G "http://127.0.0.1:50005/get_stats?stats"

The storage integrity tool is used for verifying data integrity in storage. It only works for single replica for now. You can use it like this:
./storage_integrity --meta_server_addrs=192.168.8.5:60001 --first_vertex_id=100 --width=1000 --height=1000

meta_server_addrs is the meta server addresss.
first_vertex_id is the first vertex id which we inserts.
width and height is the width and height of the big matrix.
yixinglu pushed a commit to yixinglu/nebula that referenced this pull request Jan 31, 2023
Co-authored-by: Sophie <84560950+Sophie-Xie@users.noreply.github.com>

Co-authored-by: Romi <romi.chen@vesoft.com>
Co-authored-by: Sophie <84560950+Sophie-Xie@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add some metrics in storage service.
7 participants