Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use SpinLock for Histogram updates #3550

Closed

Conversation

utpilla
Copy link
Contributor

@utpilla utpilla commented Aug 5, 2022

Use SpinLock for Histogram updates

Stress Test Results (high contention) with Prometheus Exporter scraping interval = 10s

Multiple threads updating the same Histogram MetricPoint in parallel

C:\opentelemetry-dotnet\test\OpenTelemetry.Tests.Stress.Metrics>dotnet run --framework net6.0 --configuration Release
Running (concurrency = 8, prometheusEndpoint = http://localhost:9184/metrics/), press <Esc> to stop...
2022-08-05T01:16:52.4112443Z Loops: 608,880,817, Loops/Second: 5,245,688, CPU Cycles/Loop: 809, RunwayTime (Seconds): 120
2022-08-05T01:16:52.6155305Z Loops: 609,941,306, Loops/Second: 5,189,631, CPU Cycles/Loop: 771, RunwayTime (Seconds): 120
2022-08-05T01:16:52.8181754Z Loops: 610,999,272, Loops/Second: 5,234,378, CPU Cycles/Loop: 797, RunwayTime (Seconds): 120
2022-08-05T01:16:53.0246690Z Loops: 612,015,025, Loops/Second: 4,943,883, CPU Cycles/Loop: 757, RunwayTime (Seconds): 121
2022-08-05T01:16:53.2289088Z Loops: 613,059,613, Loops/Second: 5,122,985, CPU Cycles/Loop: 715, RunwayTime (Seconds): 121
2022-08-05T01:16:53.4429513Z Loops: 614,153,916, Loops/Second: 5,128,685, CPU Cycles/Loop: 785, RunwayTime (Seconds): 121
2022-08-05T01:16:53.6454101Z Loops: 615,189,648, Loops/Second: 5,128,836, CPU Cycles/Loop: 794, RunwayTime (Seconds): 121

Stress Test Results (low contention) with Prometheus Exporter scraping interval = 10s

Multiple threads updating random Histogram MetricPoints in parallel

C:\opentelemetry-dotnet\test\OpenTelemetry.Tests.Stress.Metrics>dotnet run --framework net6.0 --configuration Release
Running (concurrency = 8, prometheusEndpoint = http://localhost:9184/metrics/), press <Esc> to stop...
2022-08-05T01:11:04.2566381Z Loops: 2,304,159,048, Loops/Second: 18,347,248, CPU Cycles/Loop: 1,199, RunwayTime (Seconds): 123
2022-08-05T01:11:04.4657673Z Loops: 2,308,429,744, Loops/Second: 18,360,469, CPU Cycles/Loop: 1,169, RunwayTime (Seconds): 123
2022-08-05T01:11:04.6900137Z Loops: 2,312,599,088, Loops/Second: 18,632,700, CPU Cycles/Loop: 1,151, RunwayTime (Seconds): 123
2022-08-05T01:11:04.9063550Z Loops: 2,316,445,679, Loops/Second: 18,469,933, CPU Cycles/Loop: 1,149, RunwayTime (Seconds): 123
2022-08-05T01:11:05.1313989Z Loops: 2,320,606,516, Loops/Second: 18,258,368, CPU Cycles/Loop: 1,137, RunwayTime (Seconds): 124
2022-08-05T01:11:05.3684671Z Loops: 2,324,848,830, Loops/Second: 18,537,724, CPU Cycles/Loop: 1,152, RunwayTime (Seconds): 124
2022-08-05T01:11:05.6004662Z Loops: 2,329,282,597, Loops/Second: 19,176,912, CPU Cycles/Loop: 1,165, RunwayTime (Seconds): 124

Benchmark Results

// * Summary *

BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
Intel Core i7-9700 CPU 3.00GHz, 1 CPU, 8 logical and 8 physical cores
.NET SDK=6.0.302
[Host] : .NET 6.0.7 (6.0.722.32202), X64 RyuJIT
DefaultJob : .NET 6.0.7 (6.0.722.32202), X64 RyuJIT

Method BoundCount Mean Error StdDev Median
HistogramHotPath 10 99.71 ns 1.953 ns 2.325 ns 100.7 ns
HistogramWith1LabelHotPath 10 156.72 ns 3.127 ns 3.840 ns 157.2 ns
HistogramWith3LabelsHotPath 10 256.47 ns 5.065 ns 8.600 ns 257.4 ns
HistogramWith5LabelsHotPath 10 354.00 ns 7.002 ns 12.804 ns 356.7 ns
HistogramWith7LabelsHotPath 10 416.66 ns 8.325 ns 14.798 ns 419.3 ns
HistogramHotPath 20 102.69 ns 2.057 ns 2.112 ns 102.9 ns
HistogramWith1LabelHotPath 20 160.90 ns 3.242 ns 4.438 ns 160.5 ns
HistogramWith3LabelsHotPath 20 264.90 ns 5.252 ns 8.629 ns 266.4 ns
HistogramWith5LabelsHotPath 20 352.43 ns 7.052 ns 11.974 ns 356.5 ns
HistogramWith7LabelsHotPath 20 423.21 ns 8.465 ns 16.106 ns 430.2 ns
HistogramHotPath 50 116.43 ns 2.297 ns 2.905 ns 117.3 ns
HistogramWith1LabelHotPath 50 168.06 ns 3.324 ns 4.659 ns 168.7 ns
HistogramWith3LabelsHotPath 50 279.30 ns 5.545 ns 10.945 ns 282.4 ns
HistogramWith5LabelsHotPath 50 373.72 ns 7.441 ns 11.363 ns 374.3 ns
HistogramWith7LabelsHotPath 50 438.49 ns 8.742 ns 8.977 ns 441.1 ns
HistogramHotPath 100 132.19 ns 2.671 ns 3.076 ns 133.2 ns
HistogramWith1LabelHotPath 100 187.76 ns 3.621 ns 4.169 ns 188.3 ns
HistogramWith3LabelsHotPath 100 300.96 ns 6.044 ns 8.273 ns 304.9 ns
HistogramWith5LabelsHotPath 100 405.44 ns 7.943 ns 12.827 ns 408.4 ns
HistogramWith7LabelsHotPath 100 479.73 ns 8.904 ns 8.329 ns 481.0 ns

@codecov
Copy link

codecov bot commented Aug 5, 2022

Codecov Report

Merging #3550 (baee221) into main (666a0c1) will increase coverage by 0.24%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3550      +/-   ##
==========================================
+ Coverage   87.04%   87.28%   +0.24%     
==========================================
  Files         275      275              
  Lines        9959     9968       +9     
==========================================
+ Hits         8669     8701      +32     
+ Misses       1290     1267      -23     
Impacted Files Coverage Δ
src/OpenTelemetry/Metrics/MetricPoint.cs 87.24% <100.00%> (+2.24%) ⬆️
...tation/OpenTelemetryProtocolExporterEventSource.cs 75.00% <0.00%> (-10.00%) ⬇️
...ZPages/Implementation/ZPagesExporterEventSource.cs 56.25% <0.00%> (-6.25%) ⬇️
src/OpenTelemetry/Metrics/HistogramBuckets.cs 96.15% <0.00%> (-3.85%) ⬇️
...metryProtocol/Implementation/ActivityExtensions.cs 94.50% <0.00%> (+3.29%) ⬆️
...emetry.Api/Internal/OpenTelemetryApiEventSource.cs 82.35% <0.00%> (+8.82%) ⬆️
...xporter.OpenTelemetryProtocol/OtlpTraceExporter.cs 77.27% <0.00%> (+18.18%) ⬆️
src/OpenTelemetry/Logs/Pool/LogRecordSharedPool.cs 100.00% <0.00%> (+21.05%) ⬆️
...entation/ExportClient/OtlpGrpcTraceExportClient.cs 78.57% <0.00%> (+28.57%) ⬆️

@github-actions
Copy link
Contributor

This PR was marked stale due to lack of activity and will be closed in 7 days. Commenting or Pushing will instruct the bot to automatically remove the label. This bot runs once per day.

@github-actions github-actions bot added the Stale Issues and pull requests which have been flagged for closing due to inactivity label Aug 13, 2022
@github-actions
Copy link
Contributor

Closed as inactive. Feel free to reopen if this PR is still being worked on.

@github-actions github-actions bot closed this Aug 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Stale Issues and pull requests which have been flagged for closing due to inactivity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant