Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use string concatenation to speed up dedupe #362

Merged
merged 1 commit into from
Jan 8, 2024
Merged

Conversation

jonkoops
Copy link
Collaborator

@jonkoops jonkoops commented Jan 8, 2024

Changes the dedupe variant so that the resulting class is built up using string concatenation, rather than joining an array. This is similar to the performance enhancements implemented under #336.

Benchmarks (compared to main)

This yields significant performance improvements, or in the worst case equivalent performance. The following benchmarks were ran on my 2021 MacBook Pro 14-inch with an M1 Pro chip running macOS Sonoma (14.2.1):

Node.js (v21.5.0)

Benchmarking 'strings'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 4,977,383 200.90875648794707 ±0.55% 2488692
dedupe/main 4,092,089 244.37392188339444 ±0.53% 2046045

Benchmarking 'object'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 9,563,547 104.56371003714477 ±0.46% 4781774
dedupe/main 7,027,889 142.29022110478644 ±0.56% 3513945

Benchmarking 'strings, object'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 5,042,013 198.33345643226724 ±0.55% 2521007
dedupe/main 4,292,539 232.96234816653524 ±0.50% 2146270

Benchmarking 'mix'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 2,167,627 461.3338635593327 ±0.71% 1083814
dedupe/main 1,963,135 509.3892965132103 ±0.54% 981568

Benchmarking 'arrays'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 2,126,593 470.23572153389307 ±0.92% 1063297
dedupe/main 1,877,663 532.5767602717776 ±0.92% 938832
Chrome (v120.0.6099.129)

Benchmarking 'strings'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 3,353,243 298.2187378562112 ±2.83% 1676957
dedupe/main 3,045,720 328.32949810615634 ±2.77% 1523165

Benchmarking 'object'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 5,188,200 192.74506198515607 ±2.77% 2594619
dedupe/main 4,344,629 230.16924641594636 ±2.79% 2172749

Benchmarking 'strings, object'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 3,534,859 282.896712971826 ±2.77% 1767783
dedupe/main 3,127,515 319.742568880824 ±2.77% 1563758

Benchmarking 'mix'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 2,075,360 481.8441137923059 ±2.77% 1037680
dedupe/main 1,873,217 533.8408788903893 ±2.79% 936796

Benchmarking 'arrays'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 1,911,809 523.0645304847641 ±2.77% 955905
dedupe/main 1,735,074 576.3439670405542 ±2.77% 867711
Safari (v17.2.1)

Benchmarking 'strings'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 2,429,255 411.6486693868418 ±8.76% 1214628
dedupe/main 2,074,807 481.9723077990835 ±8.76% 1037404

Benchmarking 'object'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 4,625,790 216.1792708674465 ±8.76% 2317521
dedupe/main 3,398,125 294.2798689661608 ±8.76% 1702461

Benchmarking 'strings, object'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 2,395,912 417.3775998450694 ±8.76% 1197956
dedupe/main 1,990,235 502.453094348856 ±8.75% 997108

Benchmarking 'mix'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 1,369,487 730.1998995245018 ±8.76% 684744
dedupe/main 1,196,281 835.9233023651644 ±8.76% 598141

Benchmarking 'arrays'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 963,977 1037.3680727153676 ±8.76% 481989
dedupe/main 867,898 1152.2088602791353 ±8.75% 434817
Firefox (v121.0)

Benchmarking 'strings'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 3.953.654 252.93058016710611 ±8.76% 1976827
dedupe/main 3.755.970 266.24280811614574 ±8.76% 1877985

Benchmarking 'object'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 7.178.242 139.3098755934949 ±8.76% 3589121
dedupe/main 5.771.996 173.25029331274658 ±8.76% 2885998

Benchmarking 'strings, object'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 4.278.290 233.73824588795992 ±8.76% 2139145
dedupe/main 3.907.572 255.9133907193521 ±8.76% 1953786

Benchmarking 'mix'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 1.960.945 509.9579488675364 ±9.57% 980473
dedupe/main 2.081.000 480.53820278712163 ±8.76% 1040500

Benchmarking 'arrays'.

Task Name ops/sec Average Time (ns) Margin Samples
dedupe/local 1.882.678 531.1582756052813 ±8.76% 941339
dedupe/main 1.786.142 559.8659009194118 ±8.76% 893071

@jonkoops jonkoops self-assigned this Jan 8, 2024
@jonkoops jonkoops merged commit 08e91d2 into main Jan 8, 2024
5 checks passed
@jonkoops jonkoops deleted the bind-string-concat branch January 8, 2024 21:45
@jonkoops
Copy link
Collaborator Author

jonkoops commented Jan 8, 2024

Since this is quite a nice performance improvement I think I will back-port this to 2.x as well.

@krzysiek1507
Copy link

Hey! It looks like 2.5.2 hasn't been pushed to npmjs.com?

@jonkoops
Copy link
Collaborator Author

Correct, see #364. I am waiting for @JedWatson to take action.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants