Skip to content

Commit

Permalink
benchmark results
Browse files Browse the repository at this point in the history
  • Loading branch information
@djwatson
djwatson committed Oct 18, 2023
1 parent 4108868 commit 537d1f3
Show file tree
Hide file tree
Showing 3 changed files with 36 additions and 38 deletions.
13 changes: 6 additions & 7 deletions doc/TODO.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,13 @@
* input/output buffering: wc, cat, dynamic
* read: dynamic read1 sum1
* call/cc: ctak, fibc
* string ops: string
* matrix: ??
* graphs: LOOP_opt, sccp, GC
* peval - sccp
* conform - needs sccp,

* lattice: mostly catching the inner nested loop (lex-fixed) as a loop, and not allocating a closure for it.
Need to inline called-once functions??
* dynamic: read is super slow, read by char
* graphs: LOOP_opt, sccp, GC
* ?? peval - sccp

* conform - needs sccp, nboyer, sboyer, graphs - needs opt_loop

# TODO
* fix buffering
Expand Down Expand Up @@ -151,4 +150,4 @@
* Nice to have:
* custom expander, so we can have:
* better error reporting on frontend
* debugger?
* debugger?
52 changes: 26 additions & 26 deletions doc/benchmark_results2.org
View file
Original file line number Diff line number Diff line change
@@ -1,47 +1,47 @@
| benchmark | hawk | chez | diff | win | reason | loop trace |
|-----------+--------+---------+-------------+-------+-------------------------------------------------+------------|
| ack | 1.52 | 1.9 | -25. | *WIN* | | x |
| ack | 1.83 | 1.9 | -3.8251366 | *WIN* | | x |
| tak | 1.26 | 1.56 | -23.809524 | *WIN* | | x |
| fib | 2.34 | 3.0 | -28.205128 | *WIN* | | |
| sum | .89 | 1.546 | -73.707865 | *WIN* | | x |
| diviter | 1.034 | 1.041 | -0.67698259 | *WIN* | alloc inline, fusion | x |
| divrec | 1.593 | 1.636 | -2.6993095 | *WIN* | same | x |
| nqueens | 2.83 | 3.8 | -34.275618 | *WIN* | closure analysis | x |
| mperm | 5.05 | 7.2 | -42.574257 | *WIN* | | |
| ntakl | 1.62 | 2.6 | -60.493827 | *WIN* | FRONTEND | x |
| takl | 1.85 | 2.7 | -45.945946 | *WIN* | same | x |
| wc | 1.15 | .902 | 21.565217 | | getc, print_obj, fputc, plt, call constant pool | x |
| cat | 1.77 | 1.278 | 27.796610 | | same as wc | x |
| cpstak | 1.91 | 2.1 | -9.9476440 | *WIN* | typecheck | |
| primes | .47 | .6999 | -48.914894 | *WIN* | | x |
| nqueens | 2.94 | 3.8 | -29.251701 | *WIN* | closure analysis | x |
| mperm | 5.168 | 7.2 | -39.318885 | *WIN* | | |
| ntakl | 1.76 | 2.6 | -47.727273 | *WIN* | FRONTEND | x |
| takl | 1.97 | 2.7 | -37.055838 | *WIN* | same | x |
| wc | 1.29 | .902 | 30.077519 | | getc, print_obj, fputc, plt, call constant pool | x |
| cat | 1.70 | 1.278 | 24.823529 | | same as wc | x |
| cpstak | 1.77 | 2.1 | -18.644068 | *WIN* | typecheck | |
| primes | .61 | .6999 | -14.737705 | *WIN* | | x |
| deriv | .53 | .908 | -71.320755 | *WIN* | merge GC, lea fusion, typechecks, snapshot min | |
| triangl | .8 | 1.184 | -48. | *WIN* | | x |
| triangl | .66 | 1.184 | -79.393939 | *WIN* | | x |
| destruc | 1.08 | 1.28 | -18.518519 | *WIN* | | x |
| gcbench | .59 | .556 | 5.7627119 | | bigger nursury | |
| puzzle | 1.32 | 1.55 | -17.424242 | *WIN* | loop opt | x |
| paraffins | 3.24 | 4.270 | -31.790123 | *WIN* | | x |
| maze | .437 | .845 | -93.363844 | *WIN* | | x |
| mazefun | 1.79 | 1.731 | 3.2960894 | | | x |
| gcbench | .49 | .556 | -13.469388 | *WIN* | bigger nursury | |
| puzzle | 1.56 | 1.55 | 0.64102564 | | loop opt | x |
| paraffins | 4.0 | 4.270 | -6.75 | *WIN* | | x |
| maze | .40 | .845 | -111.25 | *WIN* | | x |
| mazefun | 1.672 | 1.731 | -3.5287081 | *WIN* | | x |
| lattice | 2.7 | 2.7 | 0. | *WIN* | closure sinking, sccp | x |
| browse | .92 | .987 | -7.2826087 | *WIN* | | |
| graphs | 1.75 | 1.3 | 25.714286 | | LOOP, closure analysis , closure sinking | nested |
| conform | 1.89 | 1.422 | 24.761905 | | | |
| sboyer | .87 | .731 | 15.977011 | | ?? typecheck | |
| conform | 1.80 | 1.422 | 21. | | | |
| sboyer | .756 | .731 | 3.3068783 | | ?? typecheck | |
| nboyer | 1.42 | 1.483 | -4.4366197 | *WIN* | ?? | |
| array1 | 2.0 | 4.5 | -125. | *WIN* | | x |
| earley | 1.99 | 3.1 | -55.778894 | *WIN* | | x |
| scheme | 1.47 | 2.4 | -63.265306 | *WIN* | | x |
| ctak | 3.19 | .4 | 87.460815 | | cc representation | x |
| scheme | 1.53 | 2.4 | -56.862745 | *WIN* | | x |
| ctak | 3.45 | .4 | 88.405797 | | cc representation | x |
| fibc | 1.54 | .328 | 78.701299 | | cc representation | x |
| string | 5.5 | 2.5 | 54.545455 | | string ops | |
| string | .5 | 2.5 | -400. | *WIN* | string ops | |
| read1 | 1.00 | 0.574 | 42.6 | | read | |
| parsing | 2.1 | 2.1 | 5.4054054 | | sccp | |
| peval | 1.55 | 1.46 | 5.8064516 | | loop catching. sccp | |
| slatex | 1.91 | 1.958 | -2.5130890 | *WIN* | | |
| parsing | 1.85 | 2.1 | -13.513514 | *WIN* | sccp | |
| peval | 1.62 | 1.46 | 9.8765432 | | loop catching. sccp | |
| slatex | 1.98 | 1.958 | 1.1111111 | | | |
| equal | .200 | .398 | -99. | *WIN* | | |
| compiler | 2.19 | 2.004 | 8.4931507 | | loop catching | |
| matrix | 1.2 | .899 | 25.083333 | | loop catching?? opt_loop | |
| compiler | 2.11 | 2.004 | 5.0236967 | | loop catching | |
| matrix | 1.11 | .899 | 19.009009 | | loop catching?? opt_loop | |
| dynamic | 1.88 | 1.447 | 23.031915 | | read | |
| TOTAL | 72.464 | 76.9779 | -6.2291621 | *WIN* | | |
| TOTAL | 68.653 | 76.9779 | -12.126054 | *WIN* | | |
|-----------+--------+---------+-------------+-------+-------------------------------------------------+------------|
#+TBLFM: $4=($2-$3) * 100.0 /$2::$5='(if (not (= $2 0)) (if (>= $3 $2) '*WIN* "") "BAD");N::@>$2=vsum(@2..@-1)::@>$3=vsum(@2..@-1)
9 changes: 4 additions & 5 deletions docs/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,6 @@ <h2>Benchmarks</h2>
</div>
<div id="charts1"></div>
Hawk has an unoptimized read/write path.
Hawk does not optimize continuations or string operations either.
<div id="charts2"></div>
Hawk JIT does not currently support flonums, so all flonums benchmarks mostly use the VM.
<div id="charts3"></div>
Expand Down Expand Up @@ -199,7 +198,7 @@ <h2>Benchmarks</h2>
makechart('browse', [0.9500000000002728, 1.1690000295639038, 1.476638, .92]);
makechart('sum', [1.6199999999998909, 1.6970000267028809, 1.363083, 0.86]);
makechart('nboyer', [1.3900000000003274, 1.2589999437332153, 2.450626, 1.42]);
makechart('gcbench', [0.5399999999999636, 0.5889999866485596, 1.04997, 0.59]);
makechart('gcbench', [0.5399999999999636, 0.5889999866485596, 1.04997, 0.42]);
makechart('fib', [2.980000000000018, 2.680999994277954, 2.147171, 2.34]);
makechart('cpstak', [2.130000000000109, 3.247999906539917, 2.585882, 1.81]);
makechart('maze', [.437, 1.062000036239624, 0.448774, 0.42]);
Expand All @@ -208,7 +207,7 @@ <h2>Benchmarks</h2>
makechart('ntakl', [2.6599999999998545, 1.5379999876022339, 1.935883, 1.6]);
makechart('slatex', [1.849999999999909, 1.840999960899353, 3.885483, 1.91]);
makechart('compiler', [2.2, 1.059000015258789, 0.0, 2.19]);
makechart('sboyer', [0.7400000000002365, 0.6370000243186951, 1.000899, .88]);
makechart('sboyer', [0.7400000000002365, 0.6370000243186951, 1.000899, .76]);
makechart('scheme', [1.9800000000000182, 1.878999948501587, 4.262164, 1.47]);
makechart('earley', [2.9200000000000728, 0.0, 3.036918, 1.99]);
makechart('peval', [1.400000000000091, 1.8600000143051147, 1.82575, 1.55]);
Expand All @@ -219,14 +218,14 @@ <h2>Benchmarks</h2>
makechart('divrec', [1.6200000000, 1.9700000286102295, 1.559836, 1.59]);
makechart('graphs', [1.3099999999999454, 1.7000000476837158, 5.411687, 1.75]);
makechart('mperm', [6.8400000000001455, 4.39300012588501, 4.34869, 5.05]);
makechart('mazefun', [1.5799999999999272, 1.6059999465942383, 2.093023, 1.76]);
makechart('mazefun', [1.5799999999999272, 1.6059999465942383, 2.093023, 1.672]);
makechart('array1', [4.5900000000001455, 1.4989999532699585, 2.981551, 2.00]);
makechart('wc', [0.9400000000000546, 1.5779999494552612, 5.82148, 1.15]);
makechart('string', [2.9100000000003092, 2.611999988555908, 1.703866, .5]);
makechart('Totals', sums);
cur_section = "charts2";
makechart('ctak', [0.41000000000030923, 16.945999145507812, 1.854662, 3.29]);
makechart('fibc', [0.30999999999994543, 9.291000366210938, 1.176995, 1.58]);
makechart('string', [2.9100000000003092, 2.611999988555908, 1.703866, 5.41]);
makechart('dynamic', [1.3899999999998727, 2.3949999809265137, 3.172476, 1.88]);
makechart('cat', [1.3100000000004002, 2.421999931335449, 10.767127, 1.65]);
makechart('tail', [0.0, 3.299999952316284, 2.195105, 0.24]);
Expand Down

0 comments on commit 537d1f3

Please sign in to comment.