Skip to content

Commit

Permalink
test: add a global dimension to test failure metric
Browse files Browse the repository at this point in the history
The current test failure metrics have high cardinality because they
include the test name. This makes it hard to visualize them in
CloudWatch since we run into aggregation limits.

It would be nice to have one global failure rate, and some views
per-kernel and per-CPU type.

This adds 10 new metrics = 6 per-CPU + 3 per-host kernel + 1 global
metric so it is not too costly.

Signed-off-by: Pablo Barbáchano <pablob@amazon.com>
  • Loading branch information
pb8o authored and andr3wy committed May 7, 2024
1 parent 0cdc9bb commit 4d1e052
Showing 1 changed file with 14 additions and 7 deletions.
21 changes: 14 additions & 7 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,17 +117,24 @@ def record_props(request, record_property):
def pytest_runtest_logreport(report):
"""Send general test metrics to CloudWatch"""
if report.when == "call":
dimensions = {
"test": report.nodeid,
"instance": global_props.instance,
"cpu_model": global_props.cpu_model,
"host_kernel": "linux-" + global_props.host_linux_version,
}
METRICS.set_dimensions(
{
"test": report.nodeid,
"instance": global_props.instance,
"cpu_model": global_props.cpu_model,
"host_kernel": "linux-" + global_props.host_linux_version,
},
# per host kernel
{"host_kernel": "linux-" + global_props.host_linux_version},
# per CPU
{"cpu_model": global_props.cpu_model},
# and global
{},
)
METRICS.set_property("result", report.outcome)
METRICS.set_property("location", report.location)
for prop_name, prop_val in report.user_properties:
METRICS.set_property(prop_name, prop_val)
METRICS.set_dimensions(dimensions)
METRICS.put_metric(
"duration",
report.duration,
Expand Down

0 comments on commit 4d1e052

Please sign in to comment.