Skip to content

Add Example Application: Llama 3.2 (1 billion parameters) #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 29 commits into
base: devel
Choose a base branch
from
Open

Add Example Application: Llama 3.2 (1 billion parameters) #8

wants to merge 29 commits into from

Conversation

@andrej
Copy link
Collaborator

@andrej andrej commented Nov 13, 2025 •
edited
Loading

To be merged after #7! This builds on that.

Added

  • NPU-offloaded implementation of Llama 3.2 (1 billion parameters)
  • New CI test for llama

Changed

  • Moved example -> operators
  • Created examples directory to hold llama
  • Removed reliance on environment variables for MLIR AIE and Peano directories, since we no longer use an env setup script

PR Merge Checklist

  1. The PR is rebased on the latest devel commit and pointing to devel.
  2. Your PR reviewed and approved.
  3. All checks are passing.

andrej added 27 commits November 13, 2025 01:44
@andrej
fix PR template
a0eb6a1
@andrej
add CI
c144b5b
@andrej
only run extensive test suite on pushes to main
164d3e6
@andrej
add license to test docker script
d658884
@andrej
add missing build action
ae39df9
@andrej
fix env setup
86a04a3
@andrej
add missing scripts
7745734
@andrej
update run_tests.py
b74749b
@andrej
acquire github runner registration token outside of runner
628f47e
@andrej
cleanup
0d26e9a
@andrej
it's Docker, not chroot
7f0d2ae
@andrej
use absolute path for results directory
6e3931f
@andrej
fix artifact upload path
cad513e
@andrej
rename action
a7043cd
@andrej
only commit benchmark results on pushes to devel or main
970e381
@andrej
fix the commit message
f08fb0a
@andrej
Rename examples to operators
cb03bfb
@andrej
remove unused file
e2a6819
@andrej
rename examples to operators
f5ad011
@andrej
add llama example
01e7736
@andrej
add llama to CI
6029581
@andrej
use environment-variable-less config for llama
8820283
@andrej
update llama test
fd59ea9
@andrej
use absolute path for examples test
b0bb1a7
@andrej
reformat
c1ffc02
@andrej
add reuse spec
02a98a6
@andrej
add CC0-1.0 license for shakespeare
56beda3
@andrej andrej force-pushed the bring-in-the-llama branch from 70b60e5 to 56beda3 Compare November 13, 2025 02:47
@andrej andrej force-pushed the bring-in-the-llama branch from 9dca7de to acdb8d6 Compare November 13, 2025 03:10
@github-actions
Copy link

github-actions bot commented Nov 13, 2025

📊 Test Results for Small Benchmark/Test Suite

2898354 (2025_11_13_03_56_46)

IRONCLAD

Tested on 2025_11_13_03_56_46 at commit 2898354.

Test Checks Latency (mean)Bandwidth (mean)GFLOP/s (mean)
axpy_1_cols_2_channels_2048_tile_2048_3.0 ✅ 5/5 159.40 0.07 n/a
axpy_2_cols_2_channels_2048_tile_1024_3.0 ✅ 5/5 157.80 0.07 n/a
axpy_4_cols_2_channels_2048_tile_512_3.0 ✅ 5/5 185.20 0.07 n/a
axpy_8_cols_2_channels_2048_tile_256_3.0 ✅ 5/5 174.00 0.07 n/a
dequant_1_cols_1_channels_2048_tile_2048 ✅ 5/5 162.60 0.03 n/a
dequant_1_cols_2_channels_2048_tile_1024 ✅ 5/5 189.60 0.03 n/a
dequant_2_cols_1_channels_2048_tile_1024 ✅ 5/5 189.60 0.03 n/a
dequant_2_cols_2_channels_2048_tile_512 ✅ 5/5 156.80 0.03 n/a
dequant_4_cols_1_channels_2048_tile_512 ✅ 5/5 141.40 0.04 n/a
dequant_4_cols_2_channels_2048_tile_256 ✅ 5/5 161.20 0.03 n/a
dequant_8_cols_1_channels_2048_tile_256 ✅ 5/5 171.60 0.03 n/a
dequant_8_cols_2_channels_2048_tile_128 ✅ 5/5 180.60 0.03 n/a
eltwise_add_1_cols_2_channels_2048_tile_2048 ✅ 5/5 160.20 0.07 n/a
eltwise_add_2_cols_2_channels_2048_tile_1024 ✅ 5/5 215.00 0.06 n/a
eltwise_add_4_cols_2_channels_2048_tile_512 ✅ 5/5 174.60 0.07 n/a
eltwise_add_8_cols_2_channels_2048_tile_256 ✅ 5/5 171.80 0.07 n/a
eltwise_mul_1_cols_2_channels_2048_tile_2048 ✅ 5/5 158.20 0.07 n/a
eltwise_mul_2_cols_2_channels_2048_tile_1024 ✅ 5/5 145.00 0.08 n/a
eltwise_mul_4_cols_2_channels_2048_tile_512 ✅ 5/5 125.20 0.09 n/a
eltwise_mul_8_cols_2_channels_2048_tile_256 ✅ 5/5 141.40 0.08 n/a
gelu_1_cols_1_channels_2048_tile_2048 ✅ 5/5 188.40 0.04 n/a
gelu_1_cols_2_channels_2048_tile_1024 ✅ 5/5 144.40 0.06 n/a
gelu_2_cols_1_channels_2048_tile_1024 ✅ 5/5 141.60 0.06 n/a
gelu_2_cols_2_channels_2048_tile_512 ✅ 5/5 138.00 0.06 n/a
gelu_4_cols_1_channels_2048_tile_512 ✅ 5/5 146.00 0.06 n/a
gelu_4_cols_2_channels_2048_tile_256 ✅ 5/5 164.20 0.05 n/a
gelu_8_cols_1_channels_2048_tile_256 ✅ 5/5 184.60 0.05 n/a
gelu_8_cols_2_channels_2048_tile_128 ✅ 5/5 175.00 0.05 n/a
gemm_2048x2048x2048_64x64x64_8_cols_0_bcolmaj_0_ccolmaj_0 ✅ 5/5 6432.60 3.65 n/a
layer_norm_1_cols_1_channels_2048_tile_2048 ✅ 5/5 166.40 0.08 n/a
layer_norm_1_cols_2_channels_2048_tile_1024 ✅ 5/5 152.40 0.08 n/a
layer_norm_2_cols_1_channels_2048_tile_1024 ✅ 5/5 128.00 0.10 n/a
layer_norm_2_cols_2_channels_2048_tile_512 ✅ 5/5 148.00 0.08 n/a
layer_norm_4_cols_1_channels_2048_tile_512 ✅ 5/5 192.20 0.07 n/a
layer_norm_4_cols_2_channels_2048_tile_256 ✅ 5/5 201.00 0.07 n/a
layer_norm_8_cols_1_channels_2048_tile_256 ✅ 5/5 167.00 0.07 n/a
layer_norm_8_cols_2_channels_2048_tile_128 ✅ 5/5 172.00 0.07 n/a
llama_3.2_1b ❌ 0/5 n/a n/a n/a
matrix_vector_mul_128x128_32_1col ✅ 5/5 704.60 0.05 0.05
matrix_vector_mul_2048x8192_1_1col ✅ 5/5 3179.40 10.61 10.60
matrix_vector_mul_2048x8192_1_2col ✅ 5/5 2041.40 16.53 16.52
matrix_vector_mul_2048x8192_1_4col ✅ 5/5 1714.80 19.60 19.59
matrix_vector_mul_2048x8192_1_8col ✅ 5/5 2102.40 15.98 15.97
matrix_vector_mul_8192x2048_4_1col ✅ 5/5 3140.80 10.71 10.71
matrix_vector_mul_8192x2048_4_2col ✅ 5/5 2112.80 15.95 15.94
matrix_vector_mul_8192x2048_4_4col ✅ 5/5 1793.40 19.21 19.20
matrix_vector_mul_8192x2048_4_8col ✅ 5/5 2027.40 16.65 16.64
mem_copy_1_cols_1_channels_2048_tile_2048 ✅ 5/5 140.20 0.06 n/a
mem_copy_1_cols_2_channels_2048_tile_1024 ✅ 5/5 152.40 0.05 n/a
mem_copy_2_cols_1_channels_2048_tile_1024 ✅ 5/5 168.20 0.05 n/a
mem_copy_2_cols_2_channels_2048_tile_512 ✅ 5/5 162.40 0.05 n/a
mem_copy_4_cols_1_channels_2048_tile_512 ✅ 5/5 164.20 0.05 n/a
mem_copy_4_cols_2_channels_2048_tile_256 ✅ 5/5 162.80 0.05 n/a
mem_copy_8_cols_1_channels_2048_tile_256 ✅ 5/5 158.00 0.05 n/a
mem_copy_8_cols_2_channels_2048_tile_128 ✅ 5/5 187.00 0.04 n/a
mha ✅ 5/5 n/a 0.09 63.40
relu_1_cols_1_channels_2048_tile_2048 ✅ 5/5 131.40 0.06 n/a
relu_2_cols_1_channels_2048_tile_1024 ✅ 5/5 148.00 0.05 n/a
relu_4_cols_1_channels_2048_tile_512 ✅ 5/5 132.80 0.06 n/a
relu_8_cols_1_channels_2048_tile_256 ✅ 5/5 157.60 0.05 n/a
rms_norm_1_cols_1_channels_2048_tile_2048 ✅ 5/5 130.80 0.09 n/a
rms_norm_1_cols_2_channels_2048_tile_1024 ✅ 5/5 145.20 0.08 n/a
rms_norm_2_cols_1_channels_2048_tile_1024 ✅ 5/5 134.80 0.09 n/a
rms_norm_2_cols_2_channels_2048_tile_512 ✅ 5/5 124.80 0.09 n/a
rms_norm_4_cols_1_channels_2048_tile_512 ✅ 5/5 174.40 0.07 n/a
rms_norm_4_cols_2_channels_2048_tile_256 ✅ 5/5 155.20 0.07 n/a
rms_norm_8_cols_1_channels_2048_tile_256 ✅ 5/5 138.60 0.08 n/a
rms_norm_8_cols_2_channels_2048_tile_128 ✅ 5/5 195.00 0.07 n/a
rope_1_cols_2_channels_4096_tile_4096_0 ✅ 5/5 173.20 0.14 n/a
rope_2_cols_2_channels_4096_tile_2048_0 ✅ 5/5 146.60 0.16 n/a
rope_4_cols_2_channels_4096_tile_1024_0 ✅ 5/5 172.20 0.14 n/a
rope_8_cols_2_channels_4096_tile_512_0 ✅ 5/5 219.60 0.11 n/a
silu_1_cols_1_channels_2048_tile_2048 ✅ 5/5 159.80 0.05 n/a
silu_2_cols_1_channels_2048_tile_1024 ✅ 5/5 139.20 0.06 n/a
silu_4_cols_1_channels_2048_tile_512 ✅ 5/5 183.20 0.04 n/a
silu_8_cols_1_channels_2048_tile_256 ✅ 5/5 160.40 0.05 n/a
softmax_1_cols_2_channels_4096_tile_2048 ✅ 5/5 157.20 0.15 n/a
softmax_2_cols_2_channels_4096_tile_1024 ✅ 5/5 145.00 0.16 n/a
softmax_2_cols_2_channels_4096_tile_512 ✅ 5/5 184.20 0.13 n/a
swiglu ✅ 5/5 n/a n/a n/a
transpose_2048_M_64_N_1_cols_1_channels_64_m_64_n_8_s ✅ 5/5 183.60 2.70 n/a
transpose_2048_M_64_N_1_cols_2_channels_64_m_64_n_8_s ✅ 5/5 173.60 2.97 n/a
weighted_rms_norm_1_cols_2_channels_2048_weights_2048 ✅ 5/5 145.20 0.08 n/a
weighted_rms_norm_2_cols_2_channels_2048_weights_1024 ✅ 5/5 181.40 0.07 n/a
weighted_rms_norm_4_cols_2_channels_2048_weights_512 ✅ 5/5 127.60 0.09 n/a
weighted_rms_norm_8_cols_2_channels_2048_weights_256 ✅ 5/5 202.80 0.06 n/a
📈 Trends (vs main branch) for Small Benchmark/Test Suite

2898354 (2025_11_13_03_56_46)

IRONCLAD Trends

axpy_1_cols_2_channels_2048_tile_2048_3.0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (+0.00%)0.08 (-6.62%)0.05 (-32.11%)135.00 (+0.00%)0.07 (-8.84%)0.01 (+387.88%)151.00 (+7.09%)30.65 (+622.43%)218.00 (+47.30%)159.40 (+13.05%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.08 (n/a)0.08 (n/a)135.00 (n/a)0.08 (n/a)0.00 (n/a)141.00 (n/a)4.24 (n/a)148.00 (n/a)141.00 (n/a)

axpy_2_cols_2_channels_2048_tile_1024_3.0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (-17.29%)0.08 (+7.89%)0.06 (+50.54%)133.00 (+20.91%)0.07 (+6.77%)0.01 (-49.53%)152.00 (-7.32%)22.38 (-59.77%)186.00 (-33.57%)157.80 (-12.62%)
fbf1591 — 2025-11-13 03:39:220.10 (n/a)0.07 (n/a)0.04 (n/a)110.00 (n/a)0.07 (n/a)0.02 (n/a)164.00 (n/a)55.64 (n/a)280.00 (n/a)180.60 (n/a)

axpy_4_cols_2_channels_2048_tile_512_3.0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (-8.09%)0.07 (-11.04%)0.04 (-22.34%)136.00 (+8.80%)0.07 (-14.39%)0.01 (+13.84%)163.00 (+12.41%)47.34 (+58.50%)273.00 (+28.77%)185.20 (+19.64%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.08 (n/a)0.05 (n/a)125.00 (n/a)0.08 (n/a)0.01 (n/a)145.00 (n/a)29.87 (n/a)212.00 (n/a)154.80 (n/a)

axpy_8_cols_2_channels_2048_tile_256_3.0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (-0.72%)0.07 (+6.86%)0.06 (+6.90%)139.00 (+0.72%)0.07 (+3.22%)0.01 (-21.19%)175.00 (-6.42%)20.59 (-27.08%)203.00 (-6.45%)174.00 (-4.19%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.06 (n/a)0.05 (n/a)138.00 (n/a)0.06 (n/a)0.01 (n/a)187.00 (n/a)28.24 (n/a)217.00 (n/a)181.60 (n/a)

dequant_1_cols_1_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.04 (+33.59%)0.03 (+21.52%)0.02 (+44.39%)128.00 (-25.15%)0.03 (+27.13%)0.01 (+37.34%)158.00 (-17.71%)31.05 (-31.88%)205.00 (-30.74%)162.60 (-21.53%)
fbf1591 — 2025-11-13 03:39:220.03 (n/a)0.03 (n/a)0.02 (n/a)171.00 (n/a)0.02 (n/a)0.00 (n/a)192.00 (n/a)45.59 (n/a)296.00 (n/a)207.20 (n/a)

dequant_1_cols_2_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.03 (-35.10%)0.03 (-18.89%)0.02 (-26.52%)151.00 (+54.08%)0.03 (-27.19%)0.00 (-37.16%)180.00 (+23.29%)33.13 (+39.24%)230.00 (+36.09%)189.60 (+36.80%)
fbf1591 — 2025-11-13 03:39:220.05 (n/a)0.03 (n/a)0.03 (n/a)98.00 (n/a)0.04 (n/a)0.01 (n/a)146.00 (n/a)23.80 (n/a)169.00 (n/a)138.60 (n/a)

dequant_2_cols_1_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.04 (-5.30%)0.03 (-20.86%)0.02 (-34.71%)132.00 (+5.60%)0.03 (-19.68%)0.01 (+116.20%)187.00 (+26.35%)41.19 (+259.35%)242.00 (+53.16%)189.60 (+30.04%)
fbf1591 — 2025-11-13 03:39:220.04 (n/a)0.03 (n/a)0.03 (n/a)125.00 (n/a)0.03 (n/a)0.00 (n/a)148.00 (n/a)11.46 (n/a)158.00 (n/a)145.80 (n/a)

dequant_2_cols_2_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.04 (+3.85%)0.03 (+14.29%)0.03 (+18.75%)130.00 (-3.70%)0.03 (+12.66%)0.00 (-26.30%)161.00 (-12.50%)16.07 (-41.02%)176.00 (-15.79%)156.80 (-12.50%)
fbf1591 — 2025-11-13 03:39:220.04 (n/a)0.03 (n/a)0.02 (n/a)135.00 (n/a)0.03 (n/a)0.00 (n/a)184.00 (n/a)27.24 (n/a)209.00 (n/a)179.20 (n/a)

dequant_4_cols_1_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.05 (+6.86%)0.03 (-8.33%)0.03 (+27.65%)102.00 (-6.42%)0.04 (+1.75%)0.01 (-13.97%)144.00 (+9.09%)22.23 (-41.13%)170.00 (-21.66%)141.40 (-4.33%)
fbf1591 — 2025-11-13 03:39:220.04 (n/a)0.04 (n/a)0.02 (n/a)109.00 (n/a)0.03 (n/a)0.01 (n/a)132.00 (n/a)37.76 (n/a)217.00 (n/a)147.80 (n/a)

dequant_4_cols_2_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.04 (-7.75%)0.03 (+17.86%)0.02 (-13.78%)129.00 (+8.40%)0.03 (+0.69%)0.01 (+5.01%)140.00 (-15.15%)36.04 (+31.56%)225.00 (+15.98%)161.20 (+0.50%)
fbf1591 — 2025-11-13 03:39:220.04 (n/a)0.03 (n/a)0.03 (n/a)119.00 (n/a)0.03 (n/a)0.01 (n/a)165.00 (n/a)27.39 (n/a)194.00 (n/a)160.40 (n/a)

dequant_8_cols_1_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.04 (+26.72%)0.03 (+2.55%)0.02 (-31.11%)116.00 (-21.09%)0.03 (+1.50%)0.01 (+191.96%)157.00 (-2.48%)52.25 (+255.96%)270.00 (+45.16%)171.60 (+5.54%)
fbf1591 — 2025-11-13 03:39:220.03 (n/a)0.03 (n/a)0.03 (n/a)147.00 (n/a)0.03 (n/a)0.00 (n/a)161.00 (n/a)14.68 (n/a)186.00 (n/a)162.60 (n/a)

dequant_8_cols_2_channels_2048_tile_128

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.03 (+3.16%)0.03 (+24.29%)0.02 (+61.43%)158.00 (-3.07%)0.03 (+27.24%)0.00 (-49.16%)177.00 (-19.55%)18.08 (-69.61%)210.00 (-38.05%)180.60 (-25.25%)
fbf1591 — 2025-11-13 03:39:220.03 (n/a)0.02 (n/a)0.01 (n/a)163.00 (n/a)0.02 (n/a)0.01 (n/a)220.00 (n/a)59.50 (n/a)339.00 (n/a)241.60 (n/a)

eltwise_add_1_cols_2_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (+5.04%)0.07 (-6.83%)0.06 (-13.41%)139.00 (-4.79%)0.07 (-5.77%)0.01 (+292.01%)161.00 (+7.33%)12.94 (+326.51%)179.00 (+15.48%)160.20 (+6.80%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.08 (n/a)0.07 (n/a)146.00 (n/a)0.08 (n/a)0.00 (n/a)150.00 (n/a)3.03 (n/a)155.00 (n/a)150.00 (n/a)

eltwise_add_2_cols_2_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (-33.93%)0.06 (-41.97%)0.04 (-46.79%)168.00 (+51.35%)0.06 (-42.66%)0.01 (+9.77%)193.00 (+72.32%)39.38 (+229.27%)265.00 (+87.94%)215.00 (+78.57%)
fbf1591 — 2025-11-13 03:39:220.10 (n/a)0.10 (n/a)0.08 (n/a)111.00 (n/a)0.10 (n/a)0.01 (n/a)112.00 (n/a)11.96 (n/a)141.00 (n/a)120.40 (n/a)

eltwise_add_4_cols_2_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (-2.44%)0.07 (-17.58%)0.05 (-29.75%)123.00 (+2.50%)0.07 (-17.35%)0.01 (+63.25%)165.00 (+21.32%)38.83 (+134.80%)242.00 (+42.35%)174.60 (+25.25%)
fbf1591 — 2025-11-13 03:39:220.10 (n/a)0.08 (n/a)0.07 (n/a)120.00 (n/a)0.08 (n/a)0.01 (n/a)136.00 (n/a)16.54 (n/a)170.00 (n/a)139.40 (n/a)

eltwise_add_8_cols_2_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (-28.03%)0.07 (-19.51%)0.06 (-19.31%)157.00 (+38.94%)0.07 (-20.54%)0.01 (-50.02%)164.00 (+24.24%)16.24 (-14.78%)202.00 (+23.93%)171.80 (+24.49%)
fbf1591 — 2025-11-13 03:39:220.10 (n/a)0.09 (n/a)0.07 (n/a)113.00 (n/a)0.08 (n/a)0.01 (n/a)132.00 (n/a)19.06 (n/a)163.00 (n/a)138.00 (n/a)

eltwise_mul_1_cols_2_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (+0.00%)0.07 (-8.33%)0.06 (-20.90%)124.00 (+0.00%)0.07 (-8.48%)0.01 (+75.86%)156.00 (+9.09%)27.67 (+121.90%)201.00 (+26.42%)158.20 (+11.72%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.08 (n/a)0.07 (n/a)124.00 (n/a)0.08 (n/a)0.01 (n/a)143.00 (n/a)12.47 (n/a)159.00 (n/a)141.60 (n/a)

eltwise_mul_2_cols_2_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (+0.83%)0.08 (-3.47%)0.07 (-3.49%)121.00 (-0.82%)0.08 (-1.56%)0.01 (+22.17%)144.00 (+3.60%)17.87 (+24.72%)172.00 (+3.61%)145.00 (+2.11%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.08 (n/a)0.07 (n/a)122.00 (n/a)0.08 (n/a)0.01 (n/a)139.00 (n/a)14.32 (n/a)166.00 (n/a)142.00 (n/a)

eltwise_mul_4_cols_2_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.10 (+0.00%)0.09 (+16.13%)0.08 (+39.86%)113.00 (+0.00%)0.09 (+14.66%)0.01 (-52.65%)124.00 (-13.89%)9.95 (-66.24%)143.00 (-28.50%)125.20 (-15.41%)
fbf1591 — 2025-11-13 03:39:220.10 (n/a)0.08 (n/a)0.06 (n/a)113.00 (n/a)0.08 (n/a)0.01 (n/a)144.00 (n/a)29.47 (n/a)200.00 (n/a)148.00 (n/a)

eltwise_mul_8_cols_2_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.10 (+23.01%)0.09 (+32.56%)0.06 (+9.18%)113.00 (-18.71%)0.08 (+20.69%)0.01 (+29.26%)129.00 (-24.56%)28.92 (+3.03%)196.00 (-8.41%)141.40 (-16.43%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.07 (n/a)0.05 (n/a)139.00 (n/a)0.07 (n/a)0.01 (n/a)171.00 (n/a)28.07 (n/a)214.00 (n/a)169.20 (n/a)

gelu_1_cols_1_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.05 (-25.29%)0.04 (-8.12%)0.04 (+14.08%)170.00 (+33.86%)0.04 (-9.81%)0.00 (-67.55%)197.00 (+8.84%)15.32 (-62.05%)206.00 (-12.34%)188.40 (+5.84%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.03 (n/a)127.00 (n/a)0.05 (n/a)0.01 (n/a)181.00 (n/a)40.36 (n/a)235.00 (n/a)178.00 (n/a)

gelu_1_cols_2_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (+4.50%)0.06 (-3.52%)0.04 (+7.61%)111.00 (-4.31%)0.06 (-0.16%)0.01 (-4.69%)142.00 (+3.65%)23.33 (-17.59%)184.00 (-7.07%)144.40 (-0.55%)
fbf1591 — 2025-11-13 03:39:220.07 (n/a)0.06 (n/a)0.04 (n/a)116.00 (n/a)0.06 (n/a)0.01 (n/a)137.00 (n/a)28.31 (n/a)198.00 (n/a)145.20 (n/a)

gelu_2_cols_1_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (-6.84%)0.06 (-2.14%)0.05 (+31.85%)117.00 (+7.34%)0.06 (-3.57%)0.01 (-47.19%)140.00 (+2.19%)14.36 (-58.30%)157.00 (-24.15%)141.60 (-0.14%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.06 (n/a)0.04 (n/a)109.00 (n/a)0.06 (n/a)0.01 (n/a)137.00 (n/a)34.44 (n/a)207.00 (n/a)141.80 (n/a)

gelu_2_cols_2_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (+3.20%)0.06 (-2.16%)0.05 (+5.66%)125.00 (-3.10%)0.06 (+1.78%)0.00 (-9.83%)139.00 (+2.21%)11.93 (-17.37%)159.00 (-5.36%)138.00 (-1.99%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.06 (n/a)0.05 (n/a)129.00 (n/a)0.06 (n/a)0.01 (n/a)136.00 (n/a)14.44 (n/a)168.00 (n/a)140.80 (n/a)

gelu_4_cols_1_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (+18.60%)0.06 (+12.59%)0.05 (+40.00%)129.00 (-15.69%)0.06 (+21.14%)0.00 (-29.79%)143.00 (-11.18%)11.93 (-56.86%)160.00 (-28.57%)146.00 (-18.71%)
fbf1591 — 2025-11-13 03:39:220.05 (n/a)0.05 (n/a)0.04 (n/a)153.00 (n/a)0.05 (n/a)0.01 (n/a)161.00 (n/a)27.66 (n/a)224.00 (n/a)179.60 (n/a)

gelu_4_cols_2_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (-17.65%)0.05 (-5.81%)0.05 (+0.55%)136.00 (+21.43%)0.05 (-7.83%)0.01 (-44.55%)172.00 (+6.17%)16.40 (-31.57%)181.00 (-0.55%)164.20 (+6.62%)
fbf1591 — 2025-11-13 03:39:220.07 (n/a)0.05 (n/a)0.05 (n/a)112.00 (n/a)0.05 (n/a)0.01 (n/a)162.00 (n/a)23.97 (n/a)182.00 (n/a)154.00 (n/a)

gelu_8_cols_1_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (+7.75%)0.04 (-13.79%)0.04 (+3.18%)129.00 (-7.19%)0.05 (-3.25%)0.01 (+31.51%)203.00 (+16.00%)33.76 (+16.40%)220.00 (-3.08%)184.60 (+4.77%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.04 (n/a)139.00 (n/a)0.05 (n/a)0.01 (n/a)175.00 (n/a)29.00 (n/a)227.00 (n/a)176.20 (n/a)

gelu_8_cols_2_channels_2048_tile_128

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.05 (-3.85%)0.05 (-2.44%)0.04 (-6.54%)156.00 (+4.00%)0.05 (-3.79%)0.01 (+3.03%)164.00 (+2.50%)21.95 (+14.96%)214.00 (+7.00%)175.00 (+4.17%)
fbf1591 — 2025-11-13 03:39:220.05 (n/a)0.05 (n/a)0.04 (n/a)150.00 (n/a)0.05 (n/a)0.01 (n/a)160.00 (n/a)19.09 (n/a)200.00 (n/a)168.00 (n/a)

gemm_2048x2048x2048_64x64x64_8_cols_0_bcolmaj_0_ccolmaj_0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:543.83 (+3.69%)3.60 (-1.77%)3.40 (-2.78%)6125.00 (-3.56%)3.65 (+0.31%)0.16 (+120.73%)6505.00 (+1.80%)287.21 (+116.64%)6895.00 (+2.86%)6432.60 (-0.15%)
fbf1591 — 2025-11-13 03:39:223.69 (n/a)3.67 (n/a)3.50 (n/a)6351.00 (n/a)3.64 (n/a)0.07 (n/a)6390.00 (n/a)132.58 (n/a)6703.00 (n/a)6442.40 (n/a)

layer_norm_1_cols_1_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.10 (+1.64%)0.08 (+6.96%)0.05 (-15.74%)122.00 (-1.61%)0.08 (+4.95%)0.02 (+39.64%)158.00 (-6.51%)42.09 (+60.37%)235.00 (+18.69%)166.40 (-1.54%)
fbf1591 — 2025-11-13 03:39:220.10 (n/a)0.07 (n/a)0.06 (n/a)124.00 (n/a)0.07 (n/a)0.01 (n/a)169.00 (n/a)26.24 (n/a)198.00 (n/a)169.00 (n/a)

layer_norm_1_cols_2_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (-4.38%)0.08 (+11.33%)0.07 (+26.67%)137.00 (+4.58%)0.08 (+6.00%)0.01 (-53.41%)150.00 (-10.18%)10.13 (-61.40%)165.00 (-21.05%)152.40 (-7.52%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.07 (n/a)0.06 (n/a)131.00 (n/a)0.08 (n/a)0.01 (n/a)167.00 (n/a)26.25 (n/a)209.00 (n/a)164.80 (n/a)

layer_norm_2_cols_1_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.11 (+11.61%)0.10 (+33.60%)0.08 (+48.98%)112.00 (-10.40%)0.10 (+30.36%)0.01 (-38.21%)125.00 (-25.15%)11.63 (-62.29%)147.00 (-32.88%)128.00 (-25.23%)
fbf1591 — 2025-11-13 03:39:220.10 (n/a)0.07 (n/a)0.06 (n/a)125.00 (n/a)0.07 (n/a)0.01 (n/a)167.00 (n/a)30.83 (n/a)219.00 (n/a)171.20 (n/a)

layer_norm_2_cols_2_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (+4.23%)0.08 (+32.19%)0.08 (+54.78%)142.00 (-4.05%)0.08 (+28.84%)0.00 (-70.49%)146.00 (-24.35%)5.69 (-81.52%)157.00 (-35.39%)148.00 (-24.26%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.06 (n/a)0.05 (n/a)148.00 (n/a)0.06 (n/a)0.01 (n/a)193.00 (n/a)30.79 (n/a)243.00 (n/a)195.40 (n/a)

layer_norm_4_cols_1_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (+5.44%)0.07 (+22.78%)0.04 (-8.31%)147.00 (-5.16%)0.07 (+15.88%)0.02 (+30.48%)180.00 (-18.55%)56.38 (+39.99%)301.00 (+9.06%)192.20 (-11.10%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.06 (n/a)0.04 (n/a)155.00 (n/a)0.06 (n/a)0.01 (n/a)221.00 (n/a)40.28 (n/a)276.00 (n/a)216.20 (n/a)

layer_norm_4_cols_2_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (+2.70%)0.06 (+19.90%)0.04 (-8.18%)148.00 (-2.63%)0.07 (+10.59%)0.02 (+7.49%)191.00 (-16.59%)61.41 (+18.59%)318.00 (+8.90%)201.00 (-8.22%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.05 (n/a)0.04 (n/a)152.00 (n/a)0.06 (n/a)0.01 (n/a)229.00 (n/a)51.78 (n/a)292.00 (n/a)219.00 (n/a)

layer_norm_8_cols_1_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (-18.31%)0.07 (-6.71%)0.07 (+39.68%)142.00 (+22.41%)0.07 (-4.48%)0.01 (-63.79%)164.00 (+7.19%)15.71 (-69.36%)189.00 (-28.41%)167.00 (-2.22%)
fbf1591 — 2025-11-13 03:39:220.11 (n/a)0.08 (n/a)0.05 (n/a)116.00 (n/a)0.08 (n/a)0.02 (n/a)153.00 (n/a)51.27 (n/a)264.00 (n/a)170.80 (n/a)

layer_norm_8_cols_2_channels_2048_tile_128

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (+18.06%)0.07 (+12.28%)0.06 (+20.19%)144.00 (-15.29%)0.07 (+15.16%)0.01 (-2.80%)171.00 (-10.94%)21.06 (-29.79%)208.00 (-16.80%)172.00 (-13.74%)
fbf1591 — 2025-11-13 03:39:220.07 (n/a)0.06 (n/a)0.05 (n/a)170.00 (n/a)0.06 (n/a)0.01 (n/a)192.00 (n/a)30.00 (n/a)250.00 (n/a)199.40 (n/a)

llama_3.2_1b

No metrics available.

matrix_vector_mul_128x128_32_1col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:540.05 (+6.83%)0.05 (+13.53%)0.04 (+9.33%)0.05 (+13.53%)0.00 (-1.82%)674.00 (-6.39%)0.05 (+11.03%)0.00 (-1.82%)702.00 (-11.92%)31.75 (-15.61%)0.04 (+9.33%)761.00 (-8.53%)704.60 (-9.97%)0.05 (+11.03%)0.05 (+6.82%)
fbf1591 — 2025-11-13 03:39:220.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)720.00 (n/a)0.04 (n/a)0.00 (n/a)797.00 (n/a)37.63 (n/a)0.04 (n/a)832.00 (n/a)782.60 (n/a)0.04 (n/a)0.05 (n/a)

matrix_vector_mul_2048x8192_1_1col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:5411.22 (-1.50%)10.88 (+0.78%)9.33 (-9.15%)10.87 (+0.78%)0.68 (+86.76%)2993.00 (+1.53%)10.61 (-2.30%)0.68 (+86.76%)3086.00 (-0.77%)219.23 (+110.02%)9.33 (-9.15%)3597.00 (+10.07%)3179.40 (+2.69%)10.60 (-2.30%)11.21 (-1.50%)
fbf1591 — 2025-11-13 03:39:2211.39 (n/a)10.80 (n/a)10.27 (n/a)10.79 (n/a)0.36 (n/a)2948.00 (n/a)10.86 (n/a)0.36 (n/a)3110.00 (n/a)104.38 (n/a)10.27 (n/a)3268.00 (n/a)3096.20 (n/a)10.85 (n/a)11.38 (n/a)

matrix_vector_mul_2048x8192_1_2col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:5417.89 (+7.78%)16.83 (+4.76%)14.84 (-5.04%)16.82 (+4.76%)1.14 (+253.99%)1877.00 (-7.22%)16.53 (+2.60%)1.14 (+254.02%)1995.00 (-4.55%)144.30 (+246.12%)14.83 (-5.04%)2262.00 (+5.31%)2041.40 (-2.10%)16.52 (+2.60%)17.88 (+7.78%)
fbf1591 — 2025-11-13 03:39:2216.60 (n/a)16.06 (n/a)15.63 (n/a)16.05 (n/a)0.32 (n/a)2023.00 (n/a)16.11 (n/a)0.32 (n/a)2090.00 (n/a)41.69 (n/a)15.62 (n/a)2148.00 (n/a)2085.20 (n/a)16.10 (n/a)16.59 (n/a)

matrix_vector_mul_2048x8192_1_4col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:5420.37 (-1.27%)19.53 (+4.19%)18.93 (+2.93%)19.52 (+4.19%)0.59 (-29.35%)1648.00 (+1.29%)19.60 (+2.90%)0.59 (-29.35%)1719.00 (-4.02%)51.17 (-30.11%)18.91 (+2.93%)1774.00 (-2.85%)1714.80 (-2.91%)19.59 (+2.90%)20.36 (-1.27%)
fbf1591 — 2025-11-13 03:39:2220.64 (n/a)18.75 (n/a)18.39 (n/a)18.73 (n/a)0.83 (n/a)1627.00 (n/a)19.04 (n/a)0.83 (n/a)1791.00 (n/a)73.21 (n/a)18.38 (n/a)1826.00 (n/a)1766.20 (n/a)19.03 (n/a)20.62 (n/a)

matrix_vector_mul_2048x8192_1_8col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:5416.38 (-15.76%)15.76 (-1.50%)15.70 (+16.51%)15.75 (-1.50%)0.31 (-86.50%)2050.00 (+18.70%)15.98 (-4.77%)0.31 (-86.50%)2131.00 (+1.52%)40.45 (-85.83%)15.69 (+16.51%)2138.00 (-14.17%)2102.40 (+3.05%)15.97 (-4.77%)16.37 (-15.76%)
fbf1591 — 2025-11-13 03:39:2219.44 (n/a)16.00 (n/a)13.48 (n/a)15.99 (n/a)2.29 (n/a)1727.00 (n/a)16.78 (n/a)2.29 (n/a)2099.00 (n/a)285.41 (n/a)13.47 (n/a)2491.00 (n/a)2040.20 (n/a)16.77 (n/a)19.43 (n/a)

matrix_vector_mul_8192x2048_4_1col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:5411.22 (-0.77%)10.97 (+1.67%)9.98 (+2.76%)10.97 (+1.67%)0.49 (-7.38%)2993.00 (+0.77%)10.71 (+0.20%)0.49 (-7.38%)3060.00 (-1.64%)148.13 (-10.34%)9.97 (+2.76%)3364.00 (-2.69%)3140.80 (-0.25%)10.71 (+0.20%)11.21 (-0.77%)
fbf1591 — 2025-11-13 03:39:2211.30 (n/a)10.79 (n/a)9.71 (n/a)10.79 (n/a)0.53 (n/a)2970.00 (n/a)10.69 (n/a)0.53 (n/a)3111.00 (n/a)165.20 (n/a)9.71 (n/a)3457.00 (n/a)3148.60 (n/a)10.68 (n/a)11.30 (n/a)

matrix_vector_mul_8192x2048_4_2col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:5416.86 (-2.06%)16.23 (+2.47%)14.12 (-2.31%)16.22 (+2.46%)0.96 (+1.72%)1991.00 (+2.10%)15.95 (-0.26%)0.96 (+1.72%)2069.00 (-2.41%)136.68 (+7.52%)14.12 (-2.31%)2377.00 (+2.37%)2112.80 (+0.29%)15.94 (-0.26%)16.85 (-2.06%)
fbf1591 — 2025-11-13 03:39:2217.22 (n/a)15.84 (n/a)14.46 (n/a)15.83 (n/a)0.94 (n/a)1950.00 (n/a)15.99 (n/a)0.94 (n/a)2120.00 (n/a)127.11 (n/a)14.45 (n/a)2322.00 (n/a)2106.60 (n/a)15.98 (n/a)17.21 (n/a)

matrix_vector_mul_8192x2048_4_4col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:5421.48 (+6.97%)20.54 (+7.52%)13.98 (-23.74%)20.52 (+7.52%)2.81 (+382.32%)1563.00 (-6.52%)19.21 (+0.35%)2.81 (+382.31%)1635.00 (-7.00%)315.95 (+495.17%)13.98 (-23.74%)2401.00 (+31.13%)1793.40 (+2.18%)19.20 (+0.35%)21.47 (+6.97%)
fbf1591 — 2025-11-13 03:39:2220.08 (n/a)19.10 (n/a)18.34 (n/a)19.09 (n/a)0.58 (n/a)1672.00 (n/a)19.15 (n/a)0.58 (n/a)1758.00 (n/a)53.09 (n/a)18.33 (n/a)1831.00 (n/a)1755.20 (n/a)19.13 (n/a)20.07 (n/a)

matrix_vector_mul_8192x2048_4_8col

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)GFLOP/s (min)Latency (max)Latency (mean)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:5418.77 (+0.28%)16.75 (-7.18%)15.35 (-5.58%)16.74 (-7.18%)1.25 (+44.26%)1789.00 (-0.28%)16.65 (-6.98%)1.25 (+44.26%)2005.00 (+7.74%)146.67 (+52.11%)15.34 (-5.58%)2188.00 (+5.91%)2027.40 (+7.82%)16.64 (-6.98%)18.76 (+0.28%)
fbf1591 — 2025-11-13 03:39:2218.72 (n/a)18.04 (n/a)16.25 (n/a)18.03 (n/a)0.86 (n/a)1794.00 (n/a)17.90 (n/a)0.86 (n/a)1861.00 (n/a)96.42 (n/a)16.24 (n/a)2066.00 (n/a)1880.40 (n/a)17.89 (n/a)18.70 (n/a)

mem_copy_1_cols_1_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (+12.61%)0.05 (-1.39%)0.05 (+23.95%)119.00 (-11.19%)0.06 (+12.81%)0.01 (-10.15%)144.00 (+1.41%)17.54 (-36.58%)167.00 (-19.32%)140.20 (-12.38%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.04 (n/a)134.00 (n/a)0.05 (n/a)0.01 (n/a)142.00 (n/a)27.66 (n/a)207.00 (n/a)160.00 (n/a)

mem_copy_1_cols_2_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (-6.92%)0.06 (+8.15%)0.04 (-30.19%)130.00 (+7.44%)0.05 (-4.71%)0.01 (+94.52%)135.00 (-7.53%)31.04 (+202.39%)212.00 (+43.24%)152.40 (+7.93%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.05 (n/a)121.00 (n/a)0.05 (n/a)0.00 (n/a)146.00 (n/a)10.26 (n/a)148.00 (n/a)141.20 (n/a)

mem_copy_2_cols_1_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (+5.98%)0.06 (-2.17%)0.03 (-47.39%)117.00 (-5.65%)0.05 (-10.42%)0.01 (+210.23%)138.00 (+2.22%)61.01 (+488.59%)287.00 (+90.07%)168.20 (+22.24%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.06 (n/a)0.05 (n/a)124.00 (n/a)0.06 (n/a)0.00 (n/a)135.00 (n/a)10.37 (n/a)151.00 (n/a)137.60 (n/a)

mem_copy_2_cols_2_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (-6.11%)0.04 (+5.75%)0.04 (+31.32%)131.00 (+6.50%)0.05 (+2.09%)0.01 (-45.61%)174.00 (-5.43%)19.06 (-54.49%)182.00 (-23.85%)162.40 (-6.24%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.04 (n/a)0.03 (n/a)123.00 (n/a)0.05 (n/a)0.01 (n/a)184.00 (n/a)41.89 (n/a)239.00 (n/a)173.20 (n/a)

mem_copy_4_cols_1_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.05 (-20.28%)0.05 (-16.56%)0.04 (-17.59%)143.00 (+25.44%)0.05 (-18.02%)0.01 (-18.20%)157.00 (+19.85%)20.61 (+22.62%)199.00 (+21.34%)164.20 (+21.99%)
fbf1591 — 2025-11-13 03:39:220.07 (n/a)0.06 (n/a)0.05 (n/a)114.00 (n/a)0.06 (n/a)0.01 (n/a)131.00 (n/a)16.81 (n/a)164.00 (n/a)134.60 (n/a)

mem_copy_4_cols_2_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.05 (+21.92%)0.05 (+17.90%)0.04 (+44.32%)146.00 (-17.98%)0.05 (+21.97%)0.00 (-25.82%)162.00 (-15.18%)11.55 (-58.13%)176.00 (-30.71%)162.80 (-18.92%)
fbf1591 — 2025-11-13 03:39:220.04 (n/a)0.04 (n/a)0.03 (n/a)178.00 (n/a)0.04 (n/a)0.00 (n/a)191.00 (n/a)27.58 (n/a)254.00 (n/a)200.80 (n/a)

mem_copy_8_cols_1_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (+36.21%)0.05 (+13.38%)0.03 (-9.01%)116.00 (-26.58%)0.05 (+13.33%)0.01 (+170.30%)142.00 (-11.80%)37.93 (+122.93%)222.00 (+9.90%)158.00 (-7.82%)
fbf1591 — 2025-11-13 03:39:220.05 (n/a)0.05 (n/a)0.04 (n/a)158.00 (n/a)0.04 (n/a)0.00 (n/a)161.00 (n/a)17.01 (n/a)202.00 (n/a)171.40 (n/a)

mem_copy_8_cols_2_channels_2048_tile_128

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (+19.70%)0.04 (+17.42%)0.03 (-22.10%)132.00 (-16.46%)0.04 (+11.81%)0.01 (+108.17%)178.00 (-14.83%)49.04 (+132.66%)276.00 (+28.37%)187.00 (-6.22%)
fbf1591 — 2025-11-13 03:39:220.05 (n/a)0.04 (n/a)0.04 (n/a)158.00 (n/a)0.04 (n/a)0.00 (n/a)209.00 (n/a)21.08 (n/a)215.00 (n/a)199.40 (n/a)

mha

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)GFLOP/s (median)GFLOP/s (stddev)Bandwidth (mean)Bandwidth (stddev)GFLOP/s (min)GFLOP/s (mean)GFLOP/s (max)
2898354 — 2025-11-13 03:55:540.09 (-2.50%)0.09 (-0.73%)0.08 (-2.69%)63.28 (-0.73%)1.38 (+1.81%)0.09 (-1.44%)0.00 (+1.82%)61.18 (-2.69%)63.40 (-1.44%)65.09 (-2.50%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.09 (n/a)0.09 (n/a)63.75 (n/a)1.35 (n/a)0.09 (n/a)0.00 (n/a)62.87 (n/a)64.33 (n/a)66.76 (n/a)

relu_1_cols_1_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (+12.71%)0.06 (+20.77%)0.05 (+33.99%)118.00 (-11.28%)0.06 (+19.30%)0.01 (-33.66%)130.00 (-17.20%)12.01 (-54.73%)153.00 (-25.37%)131.40 (-17.67%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.04 (n/a)133.00 (n/a)0.05 (n/a)0.01 (n/a)157.00 (n/a)26.53 (n/a)205.00 (n/a)159.60 (n/a)

relu_2_cols_1_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (+1.59%)0.05 (-10.76%)0.05 (+0.62%)126.00 (-1.56%)0.05 (-3.42%)0.01 (+34.10%)158.00 (+12.06%)15.70 (+31.65%)162.00 (-0.61%)148.00 (+4.08%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.05 (n/a)128.00 (n/a)0.05 (n/a)0.00 (n/a)141.00 (n/a)11.92 (n/a)163.00 (n/a)142.20 (n/a)

relu_4_cols_1_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (+31.07%)0.06 (+22.39%)0.05 (+90.68%)103.00 (-23.70%)0.06 (+44.36%)0.01 (-19.33%)134.00 (-18.29%)22.66 (-68.82%)161.00 (-47.56%)132.80 (-36.34%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.02 (n/a)135.00 (n/a)0.04 (n/a)0.01 (n/a)164.00 (n/a)72.66 (n/a)307.00 (n/a)208.60 (n/a)

relu_8_cols_1_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.06 (+8.73%)0.05 (+1.91%)0.04 (-3.41%)126.00 (-8.03%)0.05 (+1.76%)0.01 (+59.38%)157.00 (-1.88%)18.18 (+53.02%)176.00 (+3.53%)157.60 (-0.88%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.04 (n/a)137.00 (n/a)0.05 (n/a)0.00 (n/a)160.00 (n/a)11.88 (n/a)170.00 (n/a)159.00 (n/a)

rms_norm_1_cols_1_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.10 (+10.34%)0.09 (+11.38%)0.07 (+12.26%)116.00 (-9.38%)0.09 (+10.65%)0.01 (+9.18%)123.00 (-10.22%)14.61 (-12.29%)155.00 (-10.92%)130.80 (-9.67%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.08 (n/a)0.07 (n/a)128.00 (n/a)0.08 (n/a)0.01 (n/a)137.00 (n/a)16.65 (n/a)174.00 (n/a)144.80 (n/a)

rms_norm_1_cols_2_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (+4.07%)0.08 (+1.32%)0.07 (+55.28%)123.00 (-3.91%)0.08 (+10.64%)0.01 (-45.76%)152.00 (-1.30%)13.79 (-67.94%)161.00 (-35.60%)145.20 (-13.47%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.07 (n/a)0.05 (n/a)128.00 (n/a)0.07 (n/a)0.01 (n/a)154.00 (n/a)43.02 (n/a)250.00 (n/a)167.80 (n/a)

rms_norm_2_cols_1_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (+13.01%)0.09 (+32.28%)0.07 (+11.11%)123.00 (-11.51%)0.09 (+18.47%)0.01 (+28.57%)127.00 (-24.40%)12.27 (+1.47%)153.00 (-10.00%)134.80 (-15.43%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.07 (n/a)0.07 (n/a)139.00 (n/a)0.07 (n/a)0.01 (n/a)168.00 (n/a)12.09 (n/a)170.00 (n/a)159.40 (n/a)

rms_norm_2_cols_2_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.10 (-9.48%)0.09 (+29.03%)0.08 (+78.52%)116.00 (+10.48%)0.09 (+21.59%)0.00 (-77.91%)124.00 (-22.50%)6.05 (-86.41%)135.00 (-43.98%)124.80 (-23.25%)
fbf1591 — 2025-11-13 03:39:220.11 (n/a)0.07 (n/a)0.05 (n/a)105.00 (n/a)0.08 (n/a)0.02 (n/a)160.00 (n/a)44.50 (n/a)241.00 (n/a)162.60 (n/a)

rms_norm_4_cols_1_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (-3.94%)0.07 (-12.73%)0.05 (-19.91%)127.00 (+4.10%)0.07 (-15.55%)0.01 (+27.45%)165.00 (+14.58%)31.28 (+70.37%)211.00 (+24.85%)174.40 (+20.61%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.08 (n/a)0.07 (n/a)122.00 (n/a)0.08 (n/a)0.01 (n/a)144.00 (n/a)18.36 (n/a)169.00 (n/a)144.60 (n/a)

rms_norm_4_cols_2_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (+2.96%)0.08 (+4.11%)0.06 (+29.74%)135.00 (-2.88%)0.07 (+5.31%)0.01 (-33.28%)146.00 (-3.95%)21.11 (-50.50%)195.00 (-22.92%)155.20 (-8.06%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.08 (n/a)0.05 (n/a)139.00 (n/a)0.07 (n/a)0.01 (n/a)152.00 (n/a)42.65 (n/a)253.00 (n/a)168.80 (n/a)

rms_norm_8_cols_1_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.09 (+14.88%)0.08 (+7.80%)0.08 (+12.50%)121.00 (-12.95%)0.08 (+12.79%)0.01 (+10.82%)141.00 (-7.24%)10.01 (-16.75%)152.00 (-11.11%)138.60 (-11.38%)
fbf1591 — 2025-11-13 03:39:220.08 (n/a)0.08 (n/a)0.07 (n/a)139.00 (n/a)0.07 (n/a)0.01 (n/a)152.00 (n/a)12.03 (n/a)171.00 (n/a)156.40 (n/a)

rms_norm_8_cols_2_channels_2048_tile_128

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (+7.75%)0.07 (+32.90%)0.03 (-10.27%)142.00 (-7.19%)0.07 (+25.37%)0.02 (+26.47%)155.00 (-24.76%)87.63 (+35.30%)370.00 (+11.45%)195.00 (-15.88%)
fbf1591 — 2025-11-13 03:39:220.07 (n/a)0.06 (n/a)0.03 (n/a)153.00 (n/a)0.05 (n/a)0.01 (n/a)206.00 (n/a)64.77 (n/a)332.00 (n/a)231.80 (n/a)

rope_1_cols_2_channels_4096_tile_4096_0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.16 (+0.00%)0.15 (+15.89%)0.08 (-4.07%)139.00 (+0.00%)0.14 (+12.60%)0.03 (+2.04%)151.00 (-13.71%)48.75 (+9.25%)270.00 (+4.25%)173.20 (-10.63%)
fbf1591 — 2025-11-13 03:39:220.16 (n/a)0.13 (n/a)0.09 (n/a)139.00 (n/a)0.12 (n/a)0.03 (n/a)175.00 (n/a)44.62 (n/a)259.00 (n/a)193.80 (n/a)

rope_2_cols_2_channels_4096_tile_2048_0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.17 (-14.82%)0.16 (+6.43%)0.13 (-9.04%)135.00 (+17.39%)0.16 (-3.68%)0.01 (-33.22%)140.00 (-6.04%)15.50 (-13.10%)177.00 (+9.94%)146.60 (+3.09%)
fbf1591 — 2025-11-13 03:39:220.20 (n/a)0.15 (n/a)0.14 (n/a)115.00 (n/a)0.16 (n/a)0.02 (n/a)149.00 (n/a)17.84 (n/a)161.00 (n/a)142.20 (n/a)

rope_4_cols_2_channels_4096_tile_1024_0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.17 (-10.22%)0.15 (-9.87%)0.09 (-9.50%)137.00 (+11.38%)0.14 (-11.23%)0.03 (-2.52%)152.00 (+10.95%)38.96 (+14.12%)242.00 (+10.50%)172.20 (+13.14%)
fbf1591 — 2025-11-13 03:39:220.19 (n/a)0.17 (n/a)0.10 (n/a)123.00 (n/a)0.16 (n/a)0.03 (n/a)137.00 (n/a)34.14 (n/a)219.00 (n/a)152.20 (n/a)

rope_8_cols_2_channels_4096_tile_512_0

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.13 (+0.00%)0.12 (+3.54%)0.07 (-25.24%)180.00 (+0.00%)0.11 (-2.97%)0.02 (+103.43%)198.00 (-3.41%)51.02 (+172.49%)317.00 (+33.76%)219.60 (+6.81%)
fbf1591 — 2025-11-13 03:39:220.13 (n/a)0.11 (n/a)0.10 (n/a)180.00 (n/a)0.11 (n/a)0.01 (n/a)205.00 (n/a)18.73 (n/a)237.00 (n/a)205.60 (n/a)

silu_1_cols_1_channels_2048_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.05 (-1.42%)0.05 (+4.46%)0.04 (+6.09%)141.00 (+1.44%)0.05 (+3.70%)0.01 (-13.51%)157.00 (-4.27%)19.82 (-17.13%)197.00 (-5.74%)159.80 (-4.08%)
fbf1591 — 2025-11-13 03:39:220.05 (n/a)0.05 (n/a)0.04 (n/a)139.00 (n/a)0.05 (n/a)0.01 (n/a)164.00 (n/a)23.92 (n/a)209.00 (n/a)166.60 (n/a)

silu_2_cols_1_channels_2048_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (+22.52%)0.05 (+5.71%)0.04 (+34.71%)111.00 (-18.38%)0.06 (+15.31%)0.01 (+17.93%)140.00 (-5.41%)23.51 (-31.53%)170.00 (-25.76%)139.20 (-13.86%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.03 (n/a)136.00 (n/a)0.05 (n/a)0.01 (n/a)148.00 (n/a)34.33 (n/a)229.00 (n/a)161.60 (n/a)

silu_4_cols_1_channels_2048_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.05 (-2.16%)0.04 (-17.82%)0.03 (-29.80%)139.00 (+2.21%)0.04 (-14.24%)0.01 (+124.44%)174.00 (+21.68%)41.91 (+217.11%)245.00 (+42.44%)183.20 (+21.81%)
fbf1591 — 2025-11-13 03:39:220.06 (n/a)0.05 (n/a)0.04 (n/a)136.00 (n/a)0.05 (n/a)0.00 (n/a)143.00 (n/a)13.22 (n/a)172.00 (n/a)150.40 (n/a)

silu_8_cols_1_channels_2048_tile_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.05 (+1.41%)0.05 (+9.09%)0.04 (+49.45%)142.00 (-1.39%)0.05 (+9.68%)0.00 (-50.31%)154.00 (-8.33%)14.24 (-68.70%)182.00 (-33.09%)160.40 (-12.45%)
fbf1591 — 2025-11-13 03:39:220.05 (n/a)0.05 (n/a)0.03 (n/a)144.00 (n/a)0.04 (n/a)0.01 (n/a)168.00 (n/a)45.48 (n/a)272.00 (n/a)183.20 (n/a)

softmax_1_cols_2_channels_4096_tile_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.17 (-11.85%)0.15 (-4.52%)0.12 (-11.52%)135.00 (+13.45%)0.15 (-5.22%)0.02 (-8.05%)155.00 (+4.73%)20.24 (+18.92%)191.00 (+13.02%)157.20 (+5.65%)
fbf1591 — 2025-11-13 03:39:220.19 (n/a)0.15 (n/a)0.14 (n/a)119.00 (n/a)0.16 (n/a)0.02 (n/a)148.00 (n/a)17.02 (n/a)169.00 (n/a)148.80 (n/a)

softmax_2_cols_2_channels_4096_tile_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.18 (-0.80%)0.17 (-1.45%)0.12 (-8.11%)125.00 (+0.81%)0.16 (+1.16%)0.02 (+20.05%)138.00 (+1.47%)22.35 (+28.40%)185.00 (+8.82%)145.00 (-0.41%)
fbf1591 — 2025-11-13 03:39:220.18 (n/a)0.17 (n/a)0.13 (n/a)124.00 (n/a)0.16 (n/a)0.02 (n/a)136.00 (n/a)17.41 (n/a)170.00 (n/a)145.60 (n/a)

softmax_2_cols_2_channels_4096_tile_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.15 (-0.66%)0.14 (+4.29%)0.09 (+9.56%)151.00 (+0.67%)0.13 (-1.24%)0.02 (-8.82%)163.00 (-4.12%)37.81 (-18.65%)251.00 (-8.73%)184.20 (+0.00%)
fbf1591 — 2025-11-13 03:39:220.15 (n/a)0.13 (n/a)0.08 (n/a)150.00 (n/a)0.13 (n/a)0.03 (n/a)170.00 (n/a)46.48 (n/a)275.00 (n/a)184.20 (n/a)

swiglu

No metrics available.

transpose_2048_M_64_N_1_cols_1_channels_64_m_64_n_8_s

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:543.05 (+2.50%)2.89 (+30.18%)2.21 (+25.79%)160.00 (-2.44%)2.70 (+16.75%)0.32 (-27.74%)169.00 (-23.18%)22.89 (-43.51%)221.00 (-20.50%)183.60 (-16.09%)
fbf1591 — 2025-11-13 03:39:222.98 (n/a)2.22 (n/a)1.76 (n/a)164.00 (n/a)2.31 (n/a)0.44 (n/a)220.00 (n/a)40.51 (n/a)278.00 (n/a)218.80 (n/a)

transpose_2048_M_64_N_1_cols_2_channels_64_m_64_n_8_s

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:543.51 (-14.39%)3.37 (+12.41%)1.91 (-23.44%)139.00 (+16.81%)2.97 (-7.35%)0.63 (+13.27%)145.00 (-11.04%)45.04 (+71.17%)256.00 (+30.61%)173.60 (+10.86%)
fbf1591 — 2025-11-13 03:39:224.10 (n/a)3.00 (n/a)2.49 (n/a)119.00 (n/a)3.21 (n/a)0.56 (n/a)163.00 (n/a)26.31 (n/a)196.00 (n/a)156.60 (n/a)

weighted_rms_norm_1_cols_2_channels_2048_weights_2048

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.10 (+0.89%)0.08 (-6.76%)0.06 (-16.29%)112.00 (-0.88%)0.08 (-6.42%)0.01 (+62.04%)148.00 (+7.25%)24.45 (+92.23%)178.00 (+19.46%)145.20 (+9.01%)
fbf1591 — 2025-11-13 03:39:220.10 (n/a)0.08 (n/a)0.08 (n/a)113.00 (n/a)0.09 (n/a)0.01 (n/a)138.00 (n/a)12.72 (n/a)149.00 (n/a)133.20 (n/a)

weighted_rms_norm_2_cols_2_channels_2048_weights_1024

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.08 (-14.58%)0.07 (-22.54%)0.05 (-34.51%)144.00 (+17.07%)0.07 (-22.43%)0.01 (+100.59%)173.00 (+29.10%)33.97 (+253.74%)226.00 (+52.70%)181.40 (+32.80%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.09 (n/a)0.08 (n/a)123.00 (n/a)0.08 (n/a)0.01 (n/a)134.00 (n/a)9.60 (n/a)148.00 (n/a)136.60 (n/a)

weighted_rms_norm_4_cols_2_channels_2048_weights_512

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.11 (+14.02%)0.09 (+5.43%)0.08 (-0.71%)107.00 (-12.30%)0.09 (+4.66%)0.01 (+107.42%)129.00 (-5.15%)12.13 (+83.14%)141.00 (+0.71%)127.60 (-3.77%)
fbf1591 — 2025-11-13 03:39:220.09 (n/a)0.08 (n/a)0.08 (n/a)122.00 (n/a)0.09 (n/a)0.00 (n/a)136.00 (n/a)6.62 (n/a)140.00 (n/a)132.60 (n/a)

weighted_rms_norm_8_cols_2_channels_2048_weights_256

Commit/Date Bandwidth (max)Bandwidth (median)Bandwidth (min)Latency (min)Bandwidth (mean)Bandwidth (stddev)Latency (median)Latency (stddev)Latency (max)Latency (mean)
2898354 — 2025-11-13 03:55:540.07 (+3.23%)0.06 (+0.53%)0.04 (-15.41%)155.00 (-3.12%)0.06 (-2.74%)0.01 (+37.63%)187.00 (-0.53%)47.60 (+59.94%)292.00 (+18.22%)202.80 (+5.41%)
fbf1591 — 2025-11-13 03:39:220.07 (n/a)0.06 (n/a)0.05 (n/a)160.00 (n/a)0.06 (n/a)0.01 (n/a)188.00 (n/a)29.76 (n/a)247.00 (n/a)192.40 (n/a)

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

📊 Test Results for Test Example Applications

2898354 (2025_11_13_04_03_33)

IRONCLAD

Tested on 2025_11_13_04_03_33 at commit 2898354.

Test Checks TTFT (mean)TPS (mean)
llama_3.2_1b ✅ 5/5 4.26 13.01
📈 Trends (vs main branch) for Test Example Applications

2898354 (2025_11_13_04_03_33)

IRONCLAD Trends

llama_3.2_1b

Commit/Date Num Tokens (max)Num Tokens (mean)Total (stddev)Total (min)TTFT (mean)Total (mean)TTFT (stddev)Total (max)TPS (max)TTFT (median)Num Tokens (stddev)TPS (min)TPS (mean)Total (median)TTFT (max)Num Tokens (min)Num Tokens (median)TPS (median)TTFT (min)TPS (stddev)
2898354 — 2025-11-13 04:02:0440.00 (n/a)40.00 (n/a)0.10 (n/a)7.11 (n/a)4.26 (n/a)7.26 (n/a)0.06 (n/a)7.37 (n/a)13.23 (n/a)4.28 (n/a)0.00 (n/a)12.69 (n/a)13.01 (n/a)7.30 (n/a)4.34 (n/a)40.00 (n/a)40.00 (n/a)13.20 (n/a)4.17 (n/a)0.25 (n/a)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@andrej