Skip to content

Commit

Permalink
Merge pull request #12 from hrzndhrn/benchee-1-3
Browse files Browse the repository at this point in the history
Add updates for benchee 1.3
  • Loading branch information
NickNeck authored Dec 24, 2023
2 parents de9a434 + 8b8b882 commit a1776c3
Show file tree
Hide file tree
Showing 7 changed files with 263 additions and 106 deletions.
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
# Changelog

## 0.3.1 - 2022-11-18
## 0.3.3 - 2023-12-24

Changes for `benchee` 1.3.

## 0.3.2 - 2022-11-18

Create dirs for output files.

Expand Down
120 changes: 19 additions & 101 deletions examples/bench/basic/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Benchmark Example

Benchmark run from 2022-11-18 15:22:20.915093Z UTC
Benchmark run from 2023-12-17 06:58:37.321435Z UTC

## System

Expand All @@ -21,10 +21,10 @@ Benchmark suite executing on the following system:
<td style="white-space: nowrap">16 GB</td>
</tr><tr>
<th style="white-space: nowrap">Elixir Version</th>
<td style="white-space: nowrap">1.14.1</td>
<td style="white-space: nowrap">1.15.7</td>
</tr><tr>
<th style="white-space: nowrap">Erlang Version</th>
<td style="white-space: nowrap">25.1.2</td>
<td style="white-space: nowrap">26.2</td>
</tr>
</table>

Expand All @@ -49,88 +49,6 @@ Benchmark suite executing with the following configuration:



__Input: long_list__

Run Time

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">IPS</th>
<th style="text-align: right">Average</th>
<th style="text-align: right">Devitation</th>
<th style="text-align: right">Median</th>
<th style="text-align: right">99th&nbsp;%</th>
</tr>

<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap; text-align: right">2.59 K</td>
<td style="white-space: nowrap; text-align: right">386.13 &micro;s</td>
<td style="white-space: nowrap; text-align: right">&plusmn;81.99%</td>
<td style="white-space: nowrap; text-align: right">249.25 &micro;s</td>
<td style="white-space: nowrap; text-align: right">1294.67 &micro;s</td>
</tr>

<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap; text-align: right">2.24 K</td>
<td style="white-space: nowrap; text-align: right">446.12 &micro;s</td>
<td style="white-space: nowrap; text-align: right">&plusmn;34.33%</td>
<td style="white-space: nowrap; text-align: right">389.54 &micro;s</td>
<td style="white-space: nowrap; text-align: right">900.29 &micro;s</td>
</tr>

</table>


Run Time Comparison

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">IPS</th>
<th style="text-align: right">Slower</th>
<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap;text-align: right">2.59 K</td>
<td>&nbsp;</td>
</tr>

<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap; text-align: right">2.24 K</td>
<td style="white-space: nowrap; text-align: right">1.16x</td>
</tr>

</table>



Memory Usage

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">Average</th>
<th style="text-align: right">Factor</th>
</tr>
<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap">625 KB</td>
<td>&nbsp;</td>
</tr>
<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap">781.25 KB</td>
<td>1.25x</td>
</tr>
</table>



__Input: short_list__

Run Time

<table style="width: 1%">
Expand All @@ -145,20 +63,20 @@ Run Time

<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap; text-align: right">7.94 M</td>
<td style="white-space: nowrap; text-align: right">126.01 ns</td>
<td style="white-space: nowrap; text-align: right">&plusmn;47.04%</td>
<td style="white-space: nowrap; text-align: right">120.90 ns</td>
<td style="white-space: nowrap; text-align: right">145.80 ns</td>
<td style="white-space: nowrap; text-align: right">3.23 K</td>
<td style="white-space: nowrap; text-align: right">309.46 &micro;s</td>
<td style="white-space: nowrap; text-align: right">&plusmn;34.59%</td>
<td style="white-space: nowrap; text-align: right">259.75 &micro;s</td>
<td style="white-space: nowrap; text-align: right">560.50 &micro;s</td>
</tr>

<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap; text-align: right">7.44 M</td>
<td style="white-space: nowrap; text-align: right">134.46 ns</td>
<td style="white-space: nowrap; text-align: right">&plusmn;43.20%</td>
<td style="white-space: nowrap; text-align: right">125 ns</td>
<td style="white-space: nowrap; text-align: right">187.50 ns</td>
<td style="white-space: nowrap; text-align: right">2.39 K</td>
<td style="white-space: nowrap; text-align: right">418.13 &micro;s</td>
<td style="white-space: nowrap; text-align: right">&plusmn;18.55%</td>
<td style="white-space: nowrap; text-align: right">398.54 &micro;s</td>
<td style="white-space: nowrap; text-align: right">602.67 &micro;s</td>
</tr>

</table>
Expand All @@ -173,14 +91,14 @@ Run Time Comparison
<th style="text-align: right">Slower</th>
<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap;text-align: right">7.94 M</td>
<td style="white-space: nowrap;text-align: right">3.23 K</td>
<td>&nbsp;</td>
</tr>

<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap; text-align: right">7.44 M</td>
<td style="white-space: nowrap; text-align: right">1.07x</td>
<td style="white-space: nowrap; text-align: right">2.39 K</td>
<td style="white-space: nowrap; text-align: right">1.35x</td>
</tr>

</table>
Expand All @@ -197,12 +115,12 @@ Memory Usage
</tr>
<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap">640 B</td>
<td style="white-space: nowrap">781.22 KB</td>
<td>&nbsp;</td>
</tr>
<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap">800 B</td>
<td>1.25x</td>
<td style="white-space: nowrap">937.50 KB</td>
<td>1.2x</td>
</tr>
</table>
208 changes: 208 additions & 0 deletions examples/bench/no_input/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
Benchmark Example

Benchmark run from 2023-12-17 06:53:53.740071Z UTC

## System

Benchmark suite executing on the following system:

<table style="width: 1%">
<tr>
<th style="width: 1%; white-space: nowrap">Operating System</th>
<td>macOS</td>
</tr><tr>
<th style="white-space: nowrap">CPU Information</th>
<td style="white-space: nowrap">Apple M1</td>
</tr><tr>
<th style="white-space: nowrap">Number of Available Cores</th>
<td style="white-space: nowrap">8</td>
</tr><tr>
<th style="white-space: nowrap">Available Memory</th>
<td style="white-space: nowrap">16 GB</td>
</tr><tr>
<th style="white-space: nowrap">Elixir Version</th>
<td style="white-space: nowrap">1.15.7</td>
</tr><tr>
<th style="white-space: nowrap">Erlang Version</th>
<td style="white-space: nowrap">26.2</td>
</tr>
</table>

## Configuration

Benchmark suite executing with the following configuration:

<table style="width: 1%">
<tr>
<th style="width: 1%">:time</th>
<td style="white-space: nowrap">10 ms</td>
</tr><tr>
<th>:parallel</th>
<td style="white-space: nowrap">1</td>
</tr><tr>
<th>:warmup</th>
<td style="white-space: nowrap">20 ms</td>
</tr>
</table>

## Statistics



__Input: long_list__

Run Time

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">IPS</th>
<th style="text-align: right">Average</th>
<th style="text-align: right">Devitation</th>
<th style="text-align: right">Median</th>
<th style="text-align: right">99th&nbsp;%</th>
</tr>

<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap; text-align: right">3.85 K</td>
<td style="white-space: nowrap; text-align: right">259.90 &micro;s</td>
<td style="white-space: nowrap; text-align: right">&plusmn;29.67%</td>
<td style="white-space: nowrap; text-align: right">226.92 &micro;s</td>
<td style="white-space: nowrap; text-align: right">509 &micro;s</td>
</tr>

<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap; text-align: right">2.65 K</td>
<td style="white-space: nowrap; text-align: right">377.94 &micro;s</td>
<td style="white-space: nowrap; text-align: right">&plusmn;21.63%</td>
<td style="white-space: nowrap; text-align: right">372.46 &micro;s</td>
<td style="white-space: nowrap; text-align: right">567.92 &micro;s</td>
</tr>

</table>


Run Time Comparison

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">IPS</th>
<th style="text-align: right">Slower</th>
<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap;text-align: right">3.85 K</td>
<td>&nbsp;</td>
</tr>

<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap; text-align: right">2.65 K</td>
<td style="white-space: nowrap; text-align: right">1.45x</td>
</tr>

</table>



Memory Usage

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">Average</th>
<th style="text-align: right">Factor</th>
</tr>
<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap">625 KB</td>
<td>&nbsp;</td>
</tr>
<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap">781.25 KB</td>
<td>1.25x</td>
</tr>
</table>



__Input: short_list__

Run Time

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">IPS</th>
<th style="text-align: right">Average</th>
<th style="text-align: right">Devitation</th>
<th style="text-align: right">Median</th>
<th style="text-align: right">99th&nbsp;%</th>
</tr>

<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap; text-align: right">7.90 M</td>
<td style="white-space: nowrap; text-align: right">126.51 ns</td>
<td style="white-space: nowrap; text-align: right">&plusmn;45.72%</td>
<td style="white-space: nowrap; text-align: right">120.90 ns</td>
<td style="white-space: nowrap; text-align: right">146.35 ns</td>
</tr>

<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap; text-align: right">7.47 M</td>
<td style="white-space: nowrap; text-align: right">133.86 ns</td>
<td style="white-space: nowrap; text-align: right">&plusmn;42.20%</td>
<td style="white-space: nowrap; text-align: right">125 ns</td>
<td style="white-space: nowrap; text-align: right">187.50 ns</td>
</tr>

</table>


Run Time Comparison

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">IPS</th>
<th style="text-align: right">Slower</th>
<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap;text-align: right">7.90 M</td>
<td>&nbsp;</td>
</tr>

<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap; text-align: right">7.47 M</td>
<td style="white-space: nowrap; text-align: right">1.06x</td>
</tr>

</table>



Memory Usage

<table style="width: 1%">
<tr>
<th>Name</th>
<th style="text-align: right">Average</th>
<th style="text-align: right">Factor</th>
</tr>
<tr>
<td style="white-space: nowrap">flat_map</td>
<td style="white-space: nowrap">640 B</td>
<td>&nbsp;</td>
</tr>
<tr>
<td style="white-space: nowrap">map.flatten</td>
<td style="white-space: nowrap">800 B</td>
<td>1.25x</td>
</tr>
</table>
Loading

0 comments on commit a1776c3

Please sign in to comment.