Skip to content

Commit

Permalink
Add a chart to show the total the number of models (#6169)
Browse files Browse the repository at this point in the history
Per the char with @zou3519 and @shunting314, I'm adding another chart to
show the total number of models covered in the benchmark run. This is
the denominator used to compute the pass rate. I'm showing this in a
separate chart because there is no easy way to include this information
in the current pass rate chart given the current implementation.

I also increase the legend padding a bit to make the chart easier to
view.

### Testing


https://torchci-git-fork-huydhn-show-inductor-passr-b83539-fbopensource.vercel.app/benchmark/compilers?dashboard=torchinductor&startTime=Thu%2C%2018%20Jul%202024%2022%3A52%3A52%20GMT&stopTime=Tue%2C%2014%20Jan%202025%2023%3A52%3A52%20GMT&granularity=week&mode=inference&dtype=bfloat16&deviceName=cuda%20(a100)&lBranch=main&lCommit=1dab79470dbecef79ba4c7d4308d8a181091e58e&rBranch=main&rCommit=b732b52f1e4378f8486ceb5e7026be3321c2651c
  • Loading branch information
huydhn authored Jan 15, 2025
1 parent 275141a commit a797a36
Showing 1 changed file with 41 additions and 0 deletions.
41 changes: 41 additions & 0 deletions torchci/components/benchmark/compilers/SummaryGraphPanel.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,16 @@ function SuiteGraphPanel({
"passrate",
false
);
const totalModelCountSeries = seriesWithInterpolatedTimes(
passrate,
startTime,
stopTime,
granularity,
groupByFieldName,
TIME_FIELD_NAME,
"total_count",
false
);

// Geomean speedup
const geomean = computeGeomean(data, models).filter((r: any) => {
Expand Down Expand Up @@ -265,6 +275,32 @@ function SuiteGraphPanel({
},
},
}}
legendPadding={310}
/>
</Grid2>

<Grid2 size={{ xs: 12, lg: 6 }} height={GRAPH_ROW_HEIGHT}>
<TimeSeriesPanelWithData
data={passrate}
series={totalModelCountSeries}
title={`Number of Models / ${SUITES[suite]}`}
groupByFieldName={groupByFieldName}
yAxisRenderer={(unit) => {
return `${unit}`;
}}
additionalOptions={{
yAxis: {
scale: true,
},
label: {
show: true,
align: "left",
formatter: (r: any) => {
return r.value[1];
},
},
}}
legendPadding={310}
/>
</Grid2>

Expand All @@ -290,6 +326,7 @@ function SuiteGraphPanel({
},
},
}}
legendPadding={310}
/>
</Grid2>

Expand All @@ -315,6 +352,7 @@ function SuiteGraphPanel({
},
},
}}
legendPadding={310}
/>
</Grid2>

Expand All @@ -339,6 +377,7 @@ function SuiteGraphPanel({
},
},
}}
legendPadding={310}
/>
</Grid2>

Expand All @@ -364,6 +403,7 @@ function SuiteGraphPanel({
},
},
}}
legendPadding={310}
/>
</Grid2>

Expand All @@ -389,6 +429,7 @@ function SuiteGraphPanel({
},
},
}}
legendPadding={310}
/>
</Grid2>
</Grid2>
Expand Down

0 comments on commit a797a36

Please sign in to comment.