Performance Gap Between GraalJS and V8 on Octane

[XiaowenHu96]

Hi there,

I've recently come across some older talks about Truffle and GraalVM, and I find the concept fascinating. As part of my exploration, I wanted to see if it's really possible to achieve performant programming language implementations with GraalVM and the Truffle framework and decided to benchmark GraalJS against V8.

However, the performance results I observed seem to diverge significantly from the claims I’ve seen (on par with v8). I followed some of the recommendations discussed in older issues, such as:

• Issue #404
• Issue #149

What I did:

1. Enforced warmup for every benchmark.
2. Increased warmup time from 1 second to 60 seconds.
3. Increased warmup iterations from 32 to 512.

My Environment:

• V8 version:

~/playground/v8/out/x64.release/d8 
> V8 version 13.3.0 (candidate)

• GraalJS version:

~/playground/graaljs-24.1.1-linux-amd64/bin/js --version
> GraalVM JavaScript (Oracle GraalVM JVM 24.1.1)

Benchmark Results:

Below are the results I obtained. As you can see, the performance gap between GraalJS and V8 remains significant across most of the benchmarks. I'm wondering whether the performance differences between V8 and GraalJS have changed recently, or if I might have missed something in my setup.

Disclaimer: I'm not particularly familiar with either JavaScript or Java, but I'm genuinely interested in the performance that Partial Evaluation can achieve with a simple interpreter. So, please forgive me if I made any silly mistakes in the benchmark.

Benchmark	GraalJS Score	V8 Score	% Difference
Richards	18891.0	19490.0	-3.1%
DeltaBlue	37182.0	60438.0	-38.5%
Crypto	14864.0	25040.0	-40.6%
RayTrace	84150.0	55877.0	+50.6%
EarleyBoyer	27718.0	44941.0	-38.3%
RegExp	4542.0	5596.0	-18.8%
Splay	15735.0	20900.0	-24.7%
SplayLatency	7104.0	65056.0	-89.1%
NavierStokes	23120.0	24647.0	-6.2%
PdfJS	22671.0	42284.0	-46.4%
Mandreel	17634.0	29427.0	-40.1%
MandreelLatency	39254.0	127826.0	-69.3%
Gameboy	57540.0	65042.0	-11.5%
CodeLoad	6737.0	20687.0	-67.4%
Box2D	61922.0	66436.0	-6.8%
zlib	22520.0	62815.0	-64.1%
Typescript	35717.0	95938.0	-62.8%
Octane	22350.0	39145.0	-42.9%