Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up id generator #74

Merged
merged 4 commits into from
Apr 15, 2023
Merged

Conversation

plantfansam
Copy link
Contributor

Background

Older versions of Lua, such as 5.1 (which LuaJIT, and hence OpenResty, is compatible with) has trouble with really big numbers. This is why the repo's ID generator involves concatenating strings. I don't know the full details here, so please correct me if I'm missing something.

Benchmarks

Before

5m new ids: 10.136033 seconds.

After

5m new ids: 0.118243 seconds.

Code Changes

I investigated speeding ID generation up (it's a substantial portion of all CPU time), and I think that this makes it 92 times faster (woo). The existing code generates a random number between 0 and 255, which will generate a hexadecimal character between 00 and ff. For a span ID, it does this 8 times (resulting in a 16 character hexadecimal span ID); for a trace id, it does it 16 times (resulting in a 32 character hexadecimal trace id). See bit.tohex here: http://bitop.luajit.org/api.html.

This code change moves from generating a random number between 0 and 4294967295, which is FFFFFFFF rendered as a decimal:

image

What to look for

Are we still generating the full range of IDs? A cursory look suggest yes:

20393bcd168228a23cfe370dce47e29e
efc6a2911cc64d91
30fbc93b2d20a53dfdca0b5dc0783394
44b7ac08f82986d9
534cd7dd6a3583de52c5a4e294fa60ed
ae137d1e9ee516de
23f1e6467f793b43286a502f59d9e221
6f05e3965b2532ea
39141e3c83070fa1266e129fc098743e
5c56d44e74978487
ebc6b6dd1a0bafdde0a0fc6117c00b8b
23a22eb783b92fb5
d5b70f5c829b52e4c73e1a88d7804755
4ca8a96f4bc5ce9f
b03c2c0dea2828670e58106c0857bffb
b272d4dbe63da077

@plantfansam plantfansam requested a review from yangxikun April 14, 2023 14:32
@yangxikun yangxikun merged commit bb56b3c into yangxikun:main Apr 15, 2023
@plantfansam plantfansam deleted the speed-up-id-generator branch April 17, 2023 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants