perf(lexer): skip single space in read_next_token

[overlookmotel]

It's very common for tokens to be separated by a single space. e.g. const x = 1, x === y.

Previously a single space resulted in calling the SPS byte handler, which consumes the space, and then going round the loop again in Lexer::read_next_token.

Instead, branchlessly consume a single space (if there is one) before calling the byte handler.

Gives between 2% and 7% perf improvement on parser benchmarks.

---

This also enables a further optimization (not yet implemented).

Now the handler for whitespace (SPS) no longer has a hot path for single spaces - it's now only called for a tab, or a 2nd space in a row. In both those cases, it's quite likely there'll be more whitespace following it, so it can now be optimized for that case, and continue consuming bytes until it finds one that isn't whitespace.

If handlers for whitespace, line breaks, and comments all continue consuming bytes until they find a "real" token, then we can get rid of Kind::Skip, and remove the loop from read_next_token. This would remove another unpredictable branch.

[overlookmotel]

• perf(lexer): skip single space in read_next_token #15513 👈 (View in Graphite)
• perf(lexer): inline handle_byte into read_next_token #15520
• ci(benchmarks/lexer): ensure next_token is inlined into lexer benchmark #15519
• ci(benchmarks/lexer): fix lexer benchmark #15517
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• 0-merge - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

CodSpeed Performance Report

Merging #15513 will improve performances by 6.57%

_{Comparing 11-09-perf_lexer_skip_single_space_in_read_next_token_ (6dba827) with 11-09-perf_lexer_inline_handle_byte_into_read_next_token_ (ff4461f)}

Summary

⚡ 2 improvements
✅ 35 untouched

Benchmarks breakdown

	Mode	Benchmark	BASE	HEAD	Change
⚡	Simulation	parser[RadixUIAdoptionSection.jsx]	87 µs	81.6 µs	+6.57%
⚡	Simulation	parser[react.development.js]	1.3 ms	1.3 ms	+3.83%

[overlookmotel]

First version of this PR showed very weird benchmark results - massive regression on the lexer benchmarks, but improvement on all parser benchmarks (between 2% - 7%).

Turns out that this was an oddity of the benchmark code. The increase in size of next_token lead to it no longer getting inlined into lexer benchmark, which was a massive perf hit.

#15519 and #15520 fix that, so now we can see the effects of this change in isolation. And they're quite good!

[Boshen]

Claude study assembly code and discovered the jump table has a huge branch mis-prediction problem, but I couldn't figure out where the branch is.

Question: can we use simd to find the next position that is not a whitespace?

Thought: We should study v8 at some point.

[Boshen]

Merge activity

• Nov 9, 4:45 PM UTC: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
• Nov 9, 4:45 PM UTC: Boshen added this pull request to the Graphite merge queue.
• Nov 9, 4:55 PM UTC: Merged by the Graphite merge queue.

[overlookmotel]

Claude study assembly code and discovered the jump table has a huge branch mis-prediction problem, but I couldn't figure out where the branch is.

Interesting! Can you post what he said?

I don't think it's surprising though. The branch is the jump table. CPU cannot predict which way it'll jump.

I've opened an issue in backlog about this: oxc-project/backlog#192

Question: can we use simd to find the next position that is not a whitespace?

Yes, and Rust stabilized avx512 support recently. But I don't think the problem of poor branch prediction is specific to whitespace. It affects all tokens.

Thought: We should study v8 at some point.

Yes probably!