Skip to content

Commit

Permalink
chore: Increase benchmark warning threshold for trial decrypt (AztecP…
Browse files Browse the repository at this point in the history
…rotocol#3602)

Trial decrypt measurements have a lot of variance, and often fire as
false positives. This increases the warning threshold for note decrypt
specifically to 75% to reduce noise.
  • Loading branch information
spalladino authored Dec 13, 2023
1 parent 59e8635 commit dbd6182
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion yarn-project/scripts/src/benchmarks/markdown.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ const WARNING_DIFF_THRESHOLD = 15;
const SMALL_MS_THRESHOLD = 200;
// What % diff should be considered as a warning for "small" ms measurements
const WARNING_DIFF_THRESHOLD_SMALL_MS = 30;
// What % diff should be considered as a warning for trial_decryption in particular
const WARNING_DIFF_THRESHOLD_TRIAL_DECRYPTION = 75;

const log = createConsoleLogger();

Expand All @@ -30,7 +32,9 @@ function isWarning(row: string, col: string, value: number, base: number | undef
return false;
}
const absPercentDiff = Math.abs(Math.round(((value - base) / base) * 100));
if ((row.endsWith('_ms') || col.endsWith('_ms')) && value < SMALL_MS_THRESHOLD) {
if (row.includes('trial_decrypt') || col.includes('trial_decrypt')) {
return absPercentDiff > WARNING_DIFF_THRESHOLD_TRIAL_DECRYPTION;
} else if ((row.endsWith('_ms') || col.endsWith('_ms')) && value < SMALL_MS_THRESHOLD) {
return absPercentDiff >= WARNING_DIFF_THRESHOLD_SMALL_MS;
} else {
return absPercentDiff > WARNING_DIFF_THRESHOLD;
Expand Down

0 comments on commit dbd6182

Please sign in to comment.