Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace LINQ's custom Set with HashSet #49543

Closed
wants to merge 1 commit into from
Closed

Replace LINQ's custom Set with HashSet #49543

wants to merge 1 commit into from

Conversation

stephentoub
Copy link
Member

This was discussed in #42760 (comment), but I'm seeing very different numbers. There were significant improvements to HashSet<T> in .NET 5, so maybe that's contributing, and there have also been a few tweaks since, or maybe my changes in LINQ are subtly different from what was tested there. In my tests, throughput is generally always better. Allocation was also called out as an issue there, but the main allocation difference between the internal Set<T> and the public HashSet<T> is in their resizing algorithm. The internal set starts at a capacity of 7, and when it needs to grow it does (size*2)+1. In contrast, HashSet<T> does RoundUpToNearestPrime(size*2). This means that depending on the capacity selected, one or the other may end up allocating more memory. For example, if you add 1000 items, the custom Set will end up with a capacity of 1023 and HashSet<T> will end up with a capacity of 1597. But if you add 1100 items, the custom Set will end up with a capacity of 2047 while HashSet<T> will still end up with a capacity of 1597. We can't predict what sizes will be used, and since which allocates more depends on the exact sizes used, I don't think it's something we need to make a decision here based on... better to eliminate the proliferation of custom implementations and just use the public one we encourage everyone to use.

Using @adamsitnik's benchmarks from the above comment, but also adding 16 and 1100 sizes:

Type Method Toolchain Size Unique Mean Ratio Allocated
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 10 False 225.8 ns 1.00 312 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 10 False 179.0 ns 0.79 328 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 10 False 351.3 ns 1.00 336 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 10 False 262.4 ns 0.75 352 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 10 False 184.1 ns 1.00 312 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 10 False 175.2 ns 0.96 392 B
BenchmarksRefactor<String> Count \main\corerun.exe 10 False 341.7 ns 1.00 336 B
BenchmarksRefactor<String> Count \pr\corerun.exe 10 False 279.2 ns 0.82 432 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 10 False 192.9 ns 1.00 360 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 10 False 188.0 ns 0.97 440 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 10 False 312.9 ns 1.00 400 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 10 False 298.0 ns 0.95 496 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 10 False 200.9 ns 1.00 392 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 10 False 200.8 ns 1.00 472 B
BenchmarksRefactor<String> ToList \main\corerun.exe 10 False 373.1 ns 1.00 432 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 10 False 336.1 ns 0.90 528 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 10 True 287.2 ns 1.00 608 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 10 True 274.2 ns 0.96 656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 10 True 464.0 ns 1.00 688 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 10 True 366.2 ns 0.79 744 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 10 True 256.4 ns 1.00 608 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 10 True 183.6 ns 0.72 392 B
BenchmarksRefactor<String> Count \main\corerun.exe 10 True 409.5 ns 1.00 688 B
BenchmarksRefactor<String> Count \pr\corerun.exe 10 True 283.2 ns 0.69 432 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 10 True 293.6 ns 1.00 672 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 10 True 205.1 ns 0.70 456 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 10 True 406.3 ns 1.00 792 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 10 True 313.4 ns 0.77 536 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 10 True 286.1 ns 1.00 704 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 10 True 223.2 ns 0.79 488 B
BenchmarksRefactor<String> ToList \main\corerun.exe 10 True 413.2 ns 1.00 824 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 10 True 370.6 ns 0.90 568 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 16 False 362.1 ns 1.00 608 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 16 False 353.8 ns 0.99 656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 16 False 589.6 ns 1.00 688 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 16 False 487.3 ns 0.83 744 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 16 False 340.8 ns 1.00 608 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 16 False 253.4 ns 0.75 488 B
BenchmarksRefactor<String> Count \main\corerun.exe 16 False 572.5 ns 1.00 688 B
BenchmarksRefactor<String> Count \pr\corerun.exe 16 False 411.3 ns 0.72 552 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 16 False 345.0 ns 1.00 664 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 16 False 270.3 ns 0.78 544 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 16 False 556.6 ns 1.00 776 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 16 False 444.2 ns 0.80 640 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 16 False 383.3 ns 1.00 696 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 16 False 295.0 ns 0.75 576 B
BenchmarksRefactor<String> ToList \main\corerun.exe 16 False 586.6 ns 1.00 808 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 16 False 477.0 ns 0.81 672 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 16 True 521.0 ns 1.00 1,160 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 16 True 372.9 ns 0.71 656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 16 True 753.3 ns 1.00 1,360 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 16 True 574.4 ns 0.76 744 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 16 True 465.8 ns 1.00 1,160 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 16 True 248.4 ns 0.53 488 B
BenchmarksRefactor<String> Count \main\corerun.exe 16 True 595.9 ns 1.00 1,360 B
BenchmarksRefactor<String> Count \pr\corerun.exe 16 True 449.0 ns 0.75 552 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 16 True 476.9 ns 1.00 1,248 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 16 True 272.0 ns 0.57 576 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 16 True 732.0 ns 1.00 1,512 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 16 True 495.4 ns 0.68 704 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 16 True 486.2 ns 1.00 1,280 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 16 True 305.0 ns 0.63 608 B
BenchmarksRefactor<String> ToList \main\corerun.exe 16 True 710.4 ns 1.00 1,544 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 16 True 572.4 ns 0.81 736 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 1000 False 17,657.4 ns 1.00 16,680 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 1000 False 14,913.8 ns 0.84 27,704 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 1000 False 38,157.2 ns 1.00 20,688 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 1000 False 28,738.9 ns 0.75 34,464 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 1000 False 15,508.3 ns 1.00 16,680 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 1000 False 10,572.1 ns 0.68 17,864 B
BenchmarksRefactor<String> Count \main\corerun.exe 1000 False 35,428.3 ns 1.00 20,688 B
BenchmarksRefactor<String> Count \pr\corerun.exe 1000 False 19,822.8 ns 0.56 22,272 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 1000 False 16,335.0 ns 1.00 18,704 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 1000 False 11,501.6 ns 0.70 19,888 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 1000 False 37,916.3 ns 1.00 24,712 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 1000 False 21,331.2 ns 0.56 26,296 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 1000 False 16,358.9 ns 1.00 18,736 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 1000 False 12,361.5 ns 0.76 19,920 B
BenchmarksRefactor<String> ToList \main\corerun.exe 1000 False 37,611.2 ns 1.00 24,744 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 1000 False 22,884.3 ns 0.61 26,328 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 1000 True 22,293.5 ns 1.00 33,104 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 1000 True 21,617.0 ns 0.97 58,656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 1000 True 44,756.4 ns 1.00 41,200 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 1000 True 37,429.3 ns 0.84 73,136 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 1000 True 19,202.4 ns 1.00 33,104 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 1000 True 10,727.9 ns 0.56 17,864 B
BenchmarksRefactor<String> Count \main\corerun.exe 1000 True 39,359.2 ns 1.00 41,200 B
BenchmarksRefactor<String> Count \pr\corerun.exe 1000 True 20,789.9 ns 0.53 22,272 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 1000 True 21,073.9 ns 1.00 37,128 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 1000 True 11,987.6 ns 0.57 21,888 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 1000 True 42,851.5 ns 1.00 49,224 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 1000 True 24,023.6 ns 0.56 30,296 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 1000 True 20,791.0 ns 1.00 37,160 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 1000 True 13,798.4 ns 0.66 21,920 B
BenchmarksRefactor<String> ToList \main\corerun.exe 1000 True 42,520.5 ns 1.00 49,256 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 1000 True 27,274.7 ns 0.64 30,328 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 1100 False 21,539.6 ns 1.00 33,104 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 1100 False 16,387.9 ns 0.76 27,704 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 1100 False 42,856.4 ns 1.00 41,200 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 1100 False 31,200.8 ns 0.73 34,464 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 1100 False 19,095.1 ns 1.00 33,104 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 1100 False 11,476.6 ns 0.60 17,864 B
BenchmarksRefactor<String> Count \main\corerun.exe 1100 False 41,511.4 ns 1.00 41,200 B
BenchmarksRefactor<String> Count \pr\corerun.exe 1100 False 22,463.6 ns 0.54 22,272 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 1100 False 19,978.1 ns 1.00 35,328 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 1100 False 12,558.6 ns 0.63 20,088 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 1100 False 41,811.6 ns 1.00 45,624 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 1100 False 24,369.0 ns 0.58 26,696 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 1100 False 20,350.9 ns 1.00 35,360 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 1100 False 13,875.3 ns 0.71 20,120 B
BenchmarksRefactor<String> ToList \main\corerun.exe 1100 False 43,764.9 ns 1.00 45,656 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 1100 False 25,885.8 ns 0.59 26,728 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 1100 True 29,284.6 ns 1.00 65,912 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 1100 True 22,889.5 ns 0.78 58,656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 1100 True 54,052.7 ns 1.00 82,192 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 1100 True 40,416.3 ns 0.75 73,136 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 1100 True 24,855.2 ns 1.00 65,912 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 1100 True 11,441.4 ns 0.46 17,864 B
BenchmarksRefactor<String> Count \main\corerun.exe 1100 True 47,837.4 ns 1.00 82,192 B
BenchmarksRefactor<String> Count \pr\corerun.exe 1100 True 23,635.0 ns 0.49 22,272 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 1100 True 27,212.2 ns 1.00 70,336 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 1100 True 12,910.3 ns 0.47 22,288 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 1100 True 51,809.0 ns 1.00 91,016 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 1100 True 27,709.1 ns 0.53 31,096 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 1100 True 26,964.9 ns 1.00 70,368 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 1100 True 14,814.4 ns 0.55 22,320 B
BenchmarksRefactor<String> ToList \main\corerun.exe 1100 True 52,484.9 ns 1.00 91,048 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 1100 True 31,256.5 ns 0.60 31,128 B

As can be seen, almost every test gets measurably faster, in some cases 2x, and memory usage can either be better or worse depending on the use case. And we get to get rid of a custom hash set implementations. Yay.

Closes #47173

@stephentoub stephentoub added this to the 6.0.0 milestone Mar 12, 2021
@ghost
Copy link

ghost commented Mar 12, 2021

Tagging subscribers to this area: @eiriktsarpalis
See info in area-owners.md if you want to be subscribed.

Issue Details

This was discussed in #42760 (comment), but I'm seeing very different numbers. There were significant improvements to HashSet<T> in .NET 5, so maybe that's contributing, and there have also been a few tweaks since, or maybe my changes in LINQ are subtly different from what was tested there. In my tests, throughput is generally always better. Allocation was also called out as an issue there, but the main allocation difference between the internal Set<T> and the public HashSet<T> is in their resizing algorithm. The internal set starts at a capacity of 7, and when it needs to grow it does (size*2)+1. In contrast, HashSet<T> does RoundUpToNearestPrime(size*2). This means that depending on the capacity selected, one or the other may end up allocating more memory. For example, if you add 1000 items, the custom Set will end up with a capacity of 1023 and HashSet<T> will end up with a capacity of 1597. But if you add 1100 items, the custom Set will end up with a capacity of 2047 while HashSet<T> will still end up with a capacity of 1597. We can't predict what sizes will be used, and since which allocates more depends on the exact sizes used, I don't think it's something we need to make a decision here based on... better to eliminate the proliferation of custom implementations and just use the public one we encourage everyone to use.

Using @adamsitnik's benchmarks from the above comment, but also adding 16 and 1100 sizes:

Type Method Toolchain Size Unique Mean Ratio Allocated
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 10 False 225.8 ns 1.00 312 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 10 False 179.0 ns 0.79 328 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 10 False 351.3 ns 1.00 336 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 10 False 262.4 ns 0.75 352 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 10 False 184.1 ns 1.00 312 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 10 False 175.2 ns 0.96 392 B
BenchmarksRefactor<String> Count \main\corerun.exe 10 False 341.7 ns 1.00 336 B
BenchmarksRefactor<String> Count \pr\corerun.exe 10 False 279.2 ns 0.82 432 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 10 False 192.9 ns 1.00 360 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 10 False 188.0 ns 0.97 440 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 10 False 312.9 ns 1.00 400 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 10 False 298.0 ns 0.95 496 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 10 False 200.9 ns 1.00 392 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 10 False 200.8 ns 1.00 472 B
BenchmarksRefactor<String> ToList \main\corerun.exe 10 False 373.1 ns 1.00 432 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 10 False 336.1 ns 0.90 528 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 10 True 287.2 ns 1.00 608 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 10 True 274.2 ns 0.96 656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 10 True 464.0 ns 1.00 688 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 10 True 366.2 ns 0.79 744 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 10 True 256.4 ns 1.00 608 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 10 True 183.6 ns 0.72 392 B
BenchmarksRefactor<String> Count \main\corerun.exe 10 True 409.5 ns 1.00 688 B
BenchmarksRefactor<String> Count \pr\corerun.exe 10 True 283.2 ns 0.69 432 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 10 True 293.6 ns 1.00 672 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 10 True 205.1 ns 0.70 456 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 10 True 406.3 ns 1.00 792 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 10 True 313.4 ns 0.77 536 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 10 True 286.1 ns 1.00 704 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 10 True 223.2 ns 0.79 488 B
BenchmarksRefactor<String> ToList \main\corerun.exe 10 True 413.2 ns 1.00 824 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 10 True 370.6 ns 0.90 568 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 16 False 362.1 ns 1.00 608 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 16 False 353.8 ns 0.99 656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 16 False 589.6 ns 1.00 688 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 16 False 487.3 ns 0.83 744 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 16 False 340.8 ns 1.00 608 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 16 False 253.4 ns 0.75 488 B
BenchmarksRefactor<String> Count \main\corerun.exe 16 False 572.5 ns 1.00 688 B
BenchmarksRefactor<String> Count \pr\corerun.exe 16 False 411.3 ns 0.72 552 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 16 False 345.0 ns 1.00 664 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 16 False 270.3 ns 0.78 544 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 16 False 556.6 ns 1.00 776 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 16 False 444.2 ns 0.80 640 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 16 False 383.3 ns 1.00 696 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 16 False 295.0 ns 0.75 576 B
BenchmarksRefactor<String> ToList \main\corerun.exe 16 False 586.6 ns 1.00 808 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 16 False 477.0 ns 0.81 672 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 16 True 521.0 ns 1.00 1,160 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 16 True 372.9 ns 0.71 656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 16 True 753.3 ns 1.00 1,360 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 16 True 574.4 ns 0.76 744 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 16 True 465.8 ns 1.00 1,160 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 16 True 248.4 ns 0.53 488 B
BenchmarksRefactor<String> Count \main\corerun.exe 16 True 595.9 ns 1.00 1,360 B
BenchmarksRefactor<String> Count \pr\corerun.exe 16 True 449.0 ns 0.75 552 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 16 True 476.9 ns 1.00 1,248 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 16 True 272.0 ns 0.57 576 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 16 True 732.0 ns 1.00 1,512 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 16 True 495.4 ns 0.68 704 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 16 True 486.2 ns 1.00 1,280 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 16 True 305.0 ns 0.63 608 B
BenchmarksRefactor<String> ToList \main\corerun.exe 16 True 710.4 ns 1.00 1,544 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 16 True 572.4 ns 0.81 736 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 1000 False 17,657.4 ns 1.00 16,680 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 1000 False 14,913.8 ns 0.84 27,704 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 1000 False 38,157.2 ns 1.00 20,688 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 1000 False 28,738.9 ns 0.75 34,464 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 1000 False 15,508.3 ns 1.00 16,680 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 1000 False 10,572.1 ns 0.68 17,864 B
BenchmarksRefactor<String> Count \main\corerun.exe 1000 False 35,428.3 ns 1.00 20,688 B
BenchmarksRefactor<String> Count \pr\corerun.exe 1000 False 19,822.8 ns 0.56 22,272 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 1000 False 16,335.0 ns 1.00 18,704 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 1000 False 11,501.6 ns 0.70 19,888 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 1000 False 37,916.3 ns 1.00 24,712 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 1000 False 21,331.2 ns 0.56 26,296 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 1000 False 16,358.9 ns 1.00 18,736 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 1000 False 12,361.5 ns 0.76 19,920 B
BenchmarksRefactor<String> ToList \main\corerun.exe 1000 False 37,611.2 ns 1.00 24,744 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 1000 False 22,884.3 ns 0.61 26,328 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 1000 True 22,293.5 ns 1.00 33,104 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 1000 True 21,617.0 ns 0.97 58,656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 1000 True 44,756.4 ns 1.00 41,200 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 1000 True 37,429.3 ns 0.84 73,136 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 1000 True 19,202.4 ns 1.00 33,104 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 1000 True 10,727.9 ns 0.56 17,864 B
BenchmarksRefactor<String> Count \main\corerun.exe 1000 True 39,359.2 ns 1.00 41,200 B
BenchmarksRefactor<String> Count \pr\corerun.exe 1000 True 20,789.9 ns 0.53 22,272 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 1000 True 21,073.9 ns 1.00 37,128 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 1000 True 11,987.6 ns 0.57 21,888 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 1000 True 42,851.5 ns 1.00 49,224 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 1000 True 24,023.6 ns 0.56 30,296 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 1000 True 20,791.0 ns 1.00 37,160 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 1000 True 13,798.4 ns 0.66 21,920 B
BenchmarksRefactor<String> ToList \main\corerun.exe 1000 True 42,520.5 ns 1.00 49,256 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 1000 True 27,274.7 ns 0.64 30,328 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 1100 False 21,539.6 ns 1.00 33,104 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 1100 False 16,387.9 ns 0.76 27,704 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 1100 False 42,856.4 ns 1.00 41,200 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 1100 False 31,200.8 ns 0.73 34,464 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 1100 False 19,095.1 ns 1.00 33,104 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 1100 False 11,476.6 ns 0.60 17,864 B
BenchmarksRefactor<String> Count \main\corerun.exe 1100 False 41,511.4 ns 1.00 41,200 B
BenchmarksRefactor<String> Count \pr\corerun.exe 1100 False 22,463.6 ns 0.54 22,272 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 1100 False 19,978.1 ns 1.00 35,328 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 1100 False 12,558.6 ns 0.63 20,088 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 1100 False 41,811.6 ns 1.00 45,624 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 1100 False 24,369.0 ns 0.58 26,696 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 1100 False 20,350.9 ns 1.00 35,360 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 1100 False 13,875.3 ns 0.71 20,120 B
BenchmarksRefactor<String> ToList \main\corerun.exe 1100 False 43,764.9 ns 1.00 45,656 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 1100 False 25,885.8 ns 0.59 26,728 B
BenchmarksRefactor<Int32> Enumerate \main\corerun.exe 1100 True 29,284.6 ns 1.00 65,912 B
BenchmarksRefactor<Int32> Enumerate \pr\corerun.exe 1100 True 22,889.5 ns 0.78 58,656 B
BenchmarksRefactor<String> Enumerate \main\corerun.exe 1100 True 54,052.7 ns 1.00 82,192 B
BenchmarksRefactor<String> Enumerate \pr\corerun.exe 1100 True 40,416.3 ns 0.75 73,136 B
BenchmarksRefactor<Int32> Count \main\corerun.exe 1100 True 24,855.2 ns 1.00 65,912 B
BenchmarksRefactor<Int32> Count \pr\corerun.exe 1100 True 11,441.4 ns 0.46 17,864 B
BenchmarksRefactor<String> Count \main\corerun.exe 1100 True 47,837.4 ns 1.00 82,192 B
BenchmarksRefactor<String> Count \pr\corerun.exe 1100 True 23,635.0 ns 0.49 22,272 B
BenchmarksRefactor<Int32> ToArray \main\corerun.exe 1100 True 27,212.2 ns 1.00 70,336 B
BenchmarksRefactor<Int32> ToArray \pr\corerun.exe 1100 True 12,910.3 ns 0.47 22,288 B
BenchmarksRefactor<String> ToArray \main\corerun.exe 1100 True 51,809.0 ns 1.00 91,016 B
BenchmarksRefactor<String> ToArray \pr\corerun.exe 1100 True 27,709.1 ns 0.53 31,096 B
BenchmarksRefactor<Int32> ToList \main\corerun.exe 1100 True 26,964.9 ns 1.00 70,368 B
BenchmarksRefactor<Int32> ToList \pr\corerun.exe 1100 True 14,814.4 ns 0.55 22,320 B
BenchmarksRefactor<String> ToList \main\corerun.exe 1100 True 52,484.9 ns 1.00 91,048 B
BenchmarksRefactor<String> ToList \pr\corerun.exe 1100 True 31,256.5 ns 0.60 31,128 B

As can be seen, almost every test gets measurably faster, in some cases 2x, and memory usage can either be better or worse depending on the use case. And we get to get rid of a custom hash set implementations. Yay.

Closes #47173

Author: stephentoub
Assignees: -
Labels:

area-System.Linq, area-System.Linq.Parallel, tenet-performance
Milestone: 6.0.0

eiriktsarpalis
eiriktsarpalis approved these changes Mar 12, 2021
View reviewed changes
Copy link
Member

@eiriktsarpalis eiriktsarpalis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@stephentoub stephentoub force-pushed the linqhashset branch 2 times, most recently from ec29094 to ffc5696 Compare March 13, 2021 04:00
@stephentoub stephentoub mentioned this pull request Mar 14, 2021
Merged
@stephentoub
Copy link
Member Author

Closing in favor of #49591. It seems like something is wrong with this PR.

This was referenced Mar 15, 2021
Open
Closed
Closed
@stephentoub stephentoub deleted the linqhashset branch April 2, 2021 03:43
@ghost ghost locked as resolved and limited conversation to collaborators May 2, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@benaadams benaadams benaadams approved these changes

@eiriktsarpalis eiriktsarpalis eiriktsarpalis approved these changes

Assignees
No one assigned
Labels
area-System.Linq.Parallel area-System.Linq tenet-performance Performance related issue
Projects
None yet
Milestone
6.0.0
Development

Successfully merging this pull request may close these issues.

possible perf optimizations for System.Linq.Set & System.Linq.Parallel.Set
3 participants
@stephentoub @benaadams @eiriktsarpalis