Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark: add cmd to generate a meta report #396

Merged
merged 2 commits into from
Sep 16, 2021
Merged

benchmark: add cmd to generate a meta report #396

merged 2 commits into from
Sep 16, 2021

Conversation

eh-am
Copy link
Collaborator

@eh-am eh-am commented Sep 15, 2021

as previously discussed in slack, added a command to generate a meta (using meta here since table is already taken) to post metadata

this will make it easier to visualize different executions with different profiles

since there's a concern on revealing sensitive data, we decided to add an allowlist
which currently is somewhat silly, since the code that generates the allowlist and the code that uses the allowlist is in the same codebase
but it will make sense, where in the future we distribute the cli separately from the code

currently allowing the following fields

"BENCH_RUN_FOR",
"PYROBENCH_RAND_SEED",
"PYROBENCH_PROFILE_WIDTH",
"PYROBENCH_PROFILE_DEPTH",
"PYROBENCH_PROFILE_SYMBOL_LENGTH",
"PYROBENCH_APPS",
"PYROBENCH_CLIENTS",
"PYROBENCH_REQUESTS",

@codecov
Copy link

codecov bot commented Sep 15, 2021

Codecov Report

Merging #396 (527774b) into main (8993204) will increase coverage by 0.35%.
The diff coverage is 87.24%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #396      +/-   ##
==========================================
+ Coverage   49.52%   49.87%   +0.35%     
==========================================
  Files         121      122       +1     
  Lines        5499     5545      +46     
==========================================
+ Hits         2723     2765      +42     
- Misses       2508     2509       +1     
- Partials      268      271       +3     
Impacted Files Coverage Δ
benchmark/internal/cireport/metareport.go 86.96% <86.96%> (ø)
pkg/cli/flags.go 66.07% <100.00%> (ø)
pkg/agent/upstream/remote/remote.go 65.22% <0.00%> (+2.90%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8993204...527774b. Read the comment docs.

@pyroscopebot
Copy link
Collaborator

pyroscopebot commented Sep 15, 2021

Parameters

Details
Name Value
BENCH_RUN_FOR 10m
PYROBENCH_RAND_SEED 2306912
PYROBENCH_PROFILE_WIDTH 20
PYROBENCH_PROFILE_DEPTH 20
PYROBENCH_PROFILE_SYMBOL_LENGTH 30
PYROBENCH_APPS 20
PYROBENCH_CLIENTS 20
PYROBENCH_REQUESTS 10000

Result

main pr diff threshold
throughput 161.57 161.18 -0.38 (-0.24%) 5%
total items processed 128378.00 128886.00 508.00 (0.39%) 5%
Details
Name Description Query for main Query for pr
throughput rate(pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope:4040"}[5m]) rate(pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope_main:4040"}[5m])
total items processed pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope:4040"} pyroscope_http_request_duration_seconds_count{handler="/ingest", instance="pyroscope_main:4040"}

Screenshots

Throughput Throughput
Disk Usage Disk Usage
Memory Memory
Upload Errors (Total) Upload Errors (Total)
Successful Uploads (Total) Successful Uploads (Total)
CPU Utilization CPU Utilization

Generated by 🚫 dangerJS against 85251e0

@eh-am eh-am marked this pull request as draft September 15, 2021 21:15
@eh-am eh-am marked this pull request as ready for review September 15, 2021 21:42
Copy link
Member

@petethepig petethepig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eh-am Looks good, feel free to merge if it's ready.

@eh-am eh-am merged commit 239b28b into main Sep 16, 2021
@eh-am eh-am deleted the report-meta-table branch September 16, 2021 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants