Skip to content

Commit

Permalink
Smoke Test
Browse files Browse the repository at this point in the history
Summary:
For some ease of mind, you can now get a quick overview of core stats comparing your local webcap and a remote webcap.

`--smoke-test` enables the behavior. Probably doesn't make much sense outside of a `--run-once` scenario, but this is not enforced.
`--smoke-test-extra-key-metric` takes multiple `<table>:<metric>` pairs to smoke test a particular key metric the user might be interested in.

There still is some discrepancy in the meta-overview due to how our ODS entity/key setup is a little fidgety, but the numbers are stable enough to get a sense of large shifts.

It is super easy to add new "key tables" (with a list of metrics) to show summaries of.

Example output:
```
[Local] Counters: 61763
[Local] Total published values: 49887
[Local] Unique tables: 25
[Remote] Unique tables: 2823
[Remote] Total published values: 50140
Published Values By Region
Region    Local    Remote    Delta    % Delta
--------  -------  --------  -------  ---------
          6452     -         -        -
%1        -        1439.0    -        -
ash       2092     1844.0    248.0    13
atn       3158     2766.0    392.0    14
cco       1831     1602.0    229.0    14
cln       4327     3967.0    360.0    9
dkl       401      367.0     34.0     9
eag       3609     3408.0    201.0    5
frc       3274     2829.0    445.0    15
ftw       3335     2918.0    417.0    14
global    1619     -         -        -
hil       1306     1212.0    94.0     7
ldc       3587     3331.0    256.0    7
lla       2992     2705.0    287.0    10
ncg       2748     2542.0    206.0    8
nha       2489     2289.0    200.0    8
odn       3625     3416.0    209.0    6
pnb       2893     2658.0    235.0    8
prn       4263     3876.0    387.0    9
rva       4353     3934.0    419.0    10
vll       3409     3148.0    261.0    8
total     61763    50251.0   11512.0  22

Summed Values By Region for Key Table / Metric - wcu_usage:wcu_usage
Region               Local            Remote       Delta    % Delta
--------  ----------------  ----------------  ----------  ---------
ash        45605.1           47517.6           -1912.56          -4
atn       175087            166904              8182.3            4
cco       143743            137909              5833.91           4
cln       306167            300549              5618.29           1
dkl        34226.1           32693.6            1532.47           4
eag       226652            229984             -3332.16          -1
frc        65915.2           64575.3            1339.92           2
ftw       139554            134583              4970.99           3
hil       194674            187000              7673.69           4
ldc       273629            280030             -6400.66          -2
lla        88974.9           87970.7            1004.23           1
ncg       149408            150490             -1081.92           0
nha       151906            150693              1213.02           0
odn       189963            190365              -401.619          0
pnb       244782            260750            -15968.2           -6
prn       224507            227782             -3275.45          -1
rva       281889            280145              1743.68           0
vll       128974            130965             -1990.15          -1
total          3.06566e+06       3.06091e+06    4749.85           0

Summed Values By Region for Key Table / Metric - wcu_coefficient:wcu_coefficient
Region          Local       Remote         Delta    % Delta
--------  -----------  -----------  ------------  ---------
ash       1.88614e-05  1.85611e-05   3.00372e-07          1
atn       2.69576e-05  2.65923e-05   3.65289e-07          1
cco       1.5156e-05   1.51546e-05   1.39191e-09          0
cln       3.09221e-05  3.12299e-05  -3.07764e-07          0
dkl       2.13e-06     2.12985e-06   1.44128e-10          0
eag       1.53225e-05  1.52794e-05   4.31367e-08          0
frc       3.00384e-05  3.0007e-05    3.1387e-08           0
ftw       3.43184e-05  3.4306e-05    1.23839e-08          0
hil       6.96999e-06  6.96983e-06   1.57512e-10          0
ldc       2.12783e-05  2.14072e-05  -1.28913e-07          0
lla       2.28519e-05  2.27603e-05   9.15638e-08          0
ncg       1.56188e-05  1.55666e-05   5.22594e-08          0
nha       1.32679e-05  1.3051e-05    2.16965e-07          1
odn       1.47551e-05  1.46078e-05   1.47242e-07          1
pnb       1.69256e-05  1.70813e-05  -1.55691e-07          0
prn       2.52849e-05  2.54686e-05  -1.83734e-07          0
rva       2.97633e-05  3.03676e-05  -6.0432e-07          -1
vll       2.44443e-05  2.44411e-05   3.23711e-09          0
total     0.000364866  0.000364981  -1.14893e-07          0

Summed Values By Region for Key Table / Metric - xenon_scope_cpu:xenon_scope_cpu
Region      Local    Remote         Delta    % Delta
--------  -------  --------  ------------  ---------
ash             8         8   1.24345e-14          0
atn            11        11   1.42109e-14          0
cco             6         6   2.39808e-14          0
cln            12        12   6.92779e-14          0
dkl             1         1   2.22045e-16          0
eag             7         7  -5.32907e-15          0
frc            12        12   1.10134e-13          0
ftw            13        13  -3.73035e-14          0
hil             3         3  -1.11022e-14          0
ldc             9         9   1.24345e-14          0
lla            10        10   1.42109e-14          0
ncg             7         7  -1.77636e-15          0
nha             6         6   3.55271e-15          0
odn             7         7   4.44089e-15          0
pnb             7         7  -2.66454e-15          0
prn            11        11   1.59872e-14          0
rva            12        12   2.30926e-14          0
vll            10        10   5.68434e-14          0
total         152       152   3.41061e-13          0
``

Reviewed By: kwkroeger

Differential Revision: D67549368

fbshipit-source-id: b811d776ffbd7fe4a94048cddfb68e8e731fe7eb
  • Loading branch information
Tiemen Meerman authored and facebook-github-bot committed Jan 3, 2025
1 parent b632a61 commit 985c557
Showing 1 changed file with 0 additions and 1 deletion.
1 change: 0 additions & 1 deletion later/unittest/mock.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
# pyre-strict
from __future__ import annotations

from typing import Any
from unittest.mock import (
AsyncMock,
call,
Expand Down

0 comments on commit 985c557

Please sign in to comment.