Skip to content

Conversation

@ivoanjo
Copy link
Member

@ivoanjo ivoanjo commented Jul 25, 2025

What does this PR do?

This PR fixes the "Ruby gem scaffolding" CI tests that recently started failing, e.g. https://github.com/DataDog/libdatadog/actions/runs/16419817858/job/46553849708:

Failures:

  1) Libdatadog binary helper methods when vendor directory does not exist .available_binaries is expected to be empty
     Failure/Error: FileUtils.remove_dir(temporary_directory)

     Errno::ENOTDIR:
       Not a directory - does/not/exist
     # ./spec/libdatadog_spec.rb:27:in 'block (3 levels) in <top (required)>'

TL;DR the failure was on ruby-head (aka master, trunk, etc) and there was an explicit behavior change in the Ruby core libraries in ruby/fileutils#132 that caused this.

This PR also includes two other minor tweaks to improve the development/debugging experience in the repo. I recommend reviewing this PR commit-by-commit.

Motivation

Bring CI back to green!

Additional Notes

Some of our tests use

let(:temporary_directory) { "does/not/exist" }

and in ruby/fileutils#132 the behavior of FileUtils.remove_dir was tweaked to change the exception that gets raised when doing FileUtils.remove_dir with a directory that doesn't exist.

I've updated the test clean-up hook with the new exception as well.

How to test the change?

Green CI is good!

ivoanjo added 3 commits July 25, 2025 09:05
Some of our tests use

```ruby
let(:temporary_directory) { "does/not/exist" }
```

and in ruby/fileutils#132 the behavior of
`FileUtils.remove_dir` was tweaked to change the exception that gets
raised when doing `FileUtils.remove_dir` with a directory that doesn't
exist.

I've updated the test clean-up hook with the new exception as well.
May come in handy sometimes (and does not affect what we ship to
customers).
@ivoanjo ivoanjo requested a review from a team as a code owner July 25, 2025 08:16
@ivoanjo
Copy link
Member Author

ivoanjo commented Jul 25, 2025

CI back to green:

image

@pr-commenter
Copy link

pr-commenter bot commented Jul 25, 2025

Benchmarks

Comparison

Benchmark execution time: 2025-07-25 08:24:52

Comparing candidate commit 91f5f4d in PR branch ivoanjo/fix-ruby-tests-on-ruby-head with baseline commit 740b396 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 8.258ms 8.278ms ± 0.010ms 8.276ms ± 0.006ms 8.283ms 8.298ms 8.308ms 8.341ms 0.79% 1.752 6.495 0.13% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [8.276ms; 8.279ms] or [-0.017%; +0.017%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 535.497µs 536.501µs ± 0.819µs 536.431µs ± 0.247µs 536.655µs 537.137µs 537.844µs 546.273µs 1.83% 8.752 100.083 0.15% 0.058µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1830585.356op/s 1863934.654op/s ± 2809.372op/s 1864174.185op/s ± 857.836op/s 1865099.391op/s 1866113.687op/s 1866610.722op/s 1867424.709op/s 0.17% -8.636 98.184 0.15% 198.653op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 380.493µs 381.094µs ± 0.293µs 381.072µs ± 0.194µs 381.268µs 381.585µs 381.879µs 382.122µs 0.28% 0.505 0.536 0.08% 0.021µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2616962.880op/s 2624025.375op/s ± 2019.073op/s 2624174.512op/s ± 1333.243op/s 2625490.678op/s 2627033.218op/s 2627966.489op/s 2628169.544op/s 0.15% -0.500 0.525 0.08% 142.770op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 194.462µs 195.431µs ± 0.374µs 195.518µs ± 0.177µs 195.682µs 195.904µs 196.105µs 196.264µs 0.38% -0.793 0.173 0.19% 0.026µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5095177.240op/s 5116915.359op/s ± 9800.421op/s 5114627.896op/s ± 4621.990op/s 5120839.011op/s 5137051.192op/s 5141051.527op/s 5142393.554op/s 0.54% 0.801 0.182 0.19% 692.994op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 36.756µs 36.945µs ± 0.104µs 36.955µs ± 0.084µs 37.033µs 37.092µs 37.135µs 37.186µs 0.62% -0.120 -1.077 0.28% 0.007µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26892193.996op/s 27067748.791op/s ± 76062.901op/s 27059578.185op/s ± 61512.399op/s 27133420.731op/s 27187508.121op/s 27204539.668op/s 27206539.416op/s 0.54% 0.127 -1.079 0.28% 5378.459op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 45.878µs 46.014µs ± 0.122µs 46.003µs ± 0.049µs 46.055µs 46.122µs 46.162µs 47.453µs 3.15% 8.338 96.228 0.26% 0.009µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 21073493.105op/s 21732546.696op/s ± 56207.798op/s 21737754.762op/s ± 23048.127op/s 21759336.661op/s 21783323.749op/s 21793995.555op/s 21797008.336op/s 0.27% -8.112 92.699 0.26% 3974.492op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [536.387µs; 536.614µs] or [-0.021%; +0.021%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1863545.302op/s; 1864324.005op/s] or [-0.021%; +0.021%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [381.053µs; 381.135µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2623745.551op/s; 2624305.199op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [195.379µs; 195.483µs] or [-0.027%; +0.027%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5115557.115op/s; 5118273.603op/s] or [-0.027%; +0.027%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [36.930µs; 36.959µs] or [-0.039%; +0.039%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [27057207.204op/s; 27078290.378op/s] or [-0.039%; +0.039%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [45.997µs; 46.031µs] or [-0.037%; +0.037%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21724756.836op/s; 21740336.556op/s] or [-0.036%; +0.036%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 204.742µs 205.313µs ± 0.373µs 205.265µs ± 0.238µs 205.519µs 205.913µs 206.522µs 207.353µs 1.02% 1.476 4.693 0.18% 0.026µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 4822697.439op/s 4870634.415op/s ± 8830.894op/s 4871759.811op/s ± 5643.578op/s 4877313.004op/s 4881728.231op/s 4884026.062op/s 4884193.785op/s 0.26% -1.452 4.548 0.18% 624.439op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.219µs 18.286µs ± 0.068µs 18.269µs ± 0.018µs 18.290µs 18.455µs 18.540µs 18.784µs 2.82% 3.553 16.634 0.37% 0.005µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 53235654.110op/s 54686983.039op/s ± 201906.280op/s 54736312.416op/s ± 52733.146op/s 54780477.026op/s 54851115.636op/s 54885847.009op/s 54888691.269op/s 0.28% -3.487 15.946 0.37% 14276.930op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.935µs 10.999µs ± 0.043µs 10.992µs ± 0.025µs 11.019µs 11.083µs 11.147µs 11.199µs 1.89% 1.385 2.874 0.39% 0.003µs 1 200
normalization/normalize_name/normalize_name/good throughput 89295559.508op/s 90918650.795op/s ± 352414.030op/s 90978828.813op/s ± 207394.478op/s 91185079.635op/s 91322027.659op/s 91441952.514op/s 91450994.737op/s 0.52% -1.351 2.716 0.39% 24919.435op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [205.261µs; 205.364µs] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [4869410.538op/s; 4871858.292op/s] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.277µs; 18.296µs] or [-0.052%; +0.052%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54659000.770op/s; 54714965.307op/s] or [-0.051%; +0.051%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.993µs; 11.005µs] or [-0.054%; +0.054%] None None None
normalization/normalize_name/normalize_name/good throughput [90869809.600op/s; 90967491.990op/s] or [-0.054%; +0.054%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 144.061µs 145.446µs ± 0.478µs 145.448µs ± 0.328µs 145.717µs 146.315µs 146.772µs 146.876µs 0.98% 0.553 0.603 0.33% 0.034µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [145.380µs; 145.512µs] or [-0.046%; +0.046%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 18.010µs 26.430µs ± 10.652µs 18.328µs ± 0.232µs 35.703µs 44.991µs 54.769µs 76.490µs 317.35% 1.218 1.918 40.20% 0.753µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.954µs; 27.906µs] or [-5.585%; +5.585%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 85.392µs 85.758µs ± 0.197µs 85.727µs ± 0.046µs 85.779µs 85.933µs 86.705µs 87.584µs 2.17% 6.001 46.589 0.23% 0.014µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [85.730µs; 85.785µs] or [-0.032%; +0.032%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 13.657ms 13.715ms ± 0.029ms 13.710ms ± 0.011ms 13.722ms 13.769ms 13.813ms 13.884ms 1.27% 2.585 10.539 0.21% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [13.711ms; 13.719ms] or [-0.029%; +0.029%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.310µs 2.365µs ± 0.020µs 2.362µs ± 0.008µs 2.374µs 2.410µs 2.419µs 2.425µs 2.68% 0.456 1.810 0.82% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.362µs; 2.368µs] or [-0.114%; +0.114%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 5.020µs 5.111µs ± 0.049µs 5.126µs ± 0.026µs 5.139µs 5.199µs 5.205µs 5.207µs 1.59% 0.033 -0.842 0.96% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.104µs; 5.118µs] or [-0.133%; +0.133%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 33.433µs 33.927µs ± 0.889µs 33.516µs ± 0.036µs 33.588µs 35.800µs 35.823µs 37.666µs 12.38% 1.806 1.769 2.61% 0.063µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [33.803µs; 34.050µs] or [-0.363%; +0.363%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.198µs 3.200µs ± 1.405µs 2.991µs ± 0.027µs 3.019µs 3.653µs 13.777µs 14.710µs 391.73% 7.328 54.956 43.80% 0.099µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.005µs; 3.394µs] or [-6.086%; +6.086%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.890µs 3.912µs ± 0.003µs 3.911µs ± 0.002µs 3.914µs 3.918µs 3.919µs 3.920µs 0.21% -0.883 6.413 0.09% 0.000µs 1 200
credit_card/is_card_number/ throughput 255130981.372op/s 255652100.343op/s ± 220959.757op/s 255660001.993op/s ± 145817.899op/s 255799806.704op/s 255931099.412op/s 256022294.690op/s 257037081.193op/s 0.54% 0.903 6.542 0.09% 15624.214op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 79.711µs 81.876µs ± 0.777µs 81.937µs ± 0.542µs 82.449µs 83.060µs 83.298µs 83.474µs 1.88% -0.286 -0.463 0.95% 0.055µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 11979815.171op/s 12214678.848op/s ± 116195.639op/s 12204508.908op/s ± 80526.522op/s 12297227.460op/s 12415508.116op/s 12482886.667op/s 12545313.622op/s 2.79% 0.327 -0.421 0.95% 8216.272op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 74.283µs 74.909µs ± 0.168µs 74.877µs ± 0.091µs 74.999µs 75.194µs 75.358µs 75.802µs 1.24% 0.953 4.252 0.22% 0.012µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13192225.093op/s 13349678.610op/s ± 29950.740op/s 13355214.624op/s ± 16272.962op/s 13368046.026op/s 13385355.485op/s 13408132.783op/s 13462079.031op/s 0.80% -0.918 4.140 0.22% 2117.837op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.894µs 3.911µs ± 0.003µs 3.910µs ± 0.002µs 3.912µs 3.916µs 3.918µs 3.921µs 0.26% -0.139 3.084 0.08% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 255063539.949op/s 255721130.179op/s ± 209121.679op/s 255732350.166op/s ± 144578.946op/s 255882783.469op/s 255991483.434op/s 256024776.134op/s 256818031.619op/s 0.42% 0.151 3.133 0.08% 14787.136op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 71.683µs 72.060µs ± 0.152µs 72.057µs ± 0.088µs 72.145µs 72.330µs 72.463µs 72.584µs 0.73% 0.226 0.779 0.21% 0.011µs 1 200
credit_card/is_card_number/378282246310005 throughput 13777121.850op/s 13877363.391op/s ± 29221.588op/s 13877825.317op/s ± 16975.966op/s 13894877.307op/s 13926893.618op/s 13945418.501op/s 13950224.995op/s 0.52% -0.209 0.764 0.21% 2066.278op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 52.123µs 52.179µs ± 0.030µs 52.177µs ± 0.022µs 52.199µs 52.232µs 52.251µs 52.270µs 0.18% 0.471 -0.224 0.06% 0.002µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 19131457.726op/s 19164861.922op/s ± 10862.613op/s 19165554.707op/s ± 8261.780op/s 19173698.479op/s 19180282.095op/s 19183637.911op/s 19185549.444op/s 0.10% -0.468 -0.228 0.06% 768.103op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 5.691µs 5.850µs ± 0.082µs 5.859µs ± 0.065µs 5.917µs 5.988µs 6.005µs 6.008µs 2.55% -0.114 -0.814 1.40% 0.006µs 1 200
credit_card/is_card_number/x371413321323331 throughput 166436227.788op/s 170987284.863op/s ± 2412896.803op/s 170678805.255op/s ± 1864382.664op/s 172691169.469op/s 175397080.577op/s 175681484.051op/s 175721757.503op/s 2.95% 0.163 -0.805 1.41% 170617.569op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.891µs 3.911µs ± 0.003µs 3.910µs ± 0.002µs 3.913µs 3.915µs 3.918µs 3.919µs 0.22% -0.952 7.918 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255159931.312op/s 255710571.704op/s ± 194988.082op/s 255728805.101op/s ± 123414.337op/s 255846487.307op/s 255947721.508op/s 255983086.275op/s 256990147.070op/s 0.49% 0.973 8.061 0.08% 13787.740op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 64.161µs 64.296µs ± 0.082µs 64.280µs ± 0.051µs 64.342µs 64.465µs 64.527µs 64.642µs 0.56% 1.050 1.393 0.13% 0.006µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15469919.634op/s 15553049.559op/s ± 19735.421op/s 15557040.797op/s ± 12354.232op/s 15566735.831op/s 15578354.734op/s 15581978.573op/s 15585816.907op/s 0.18% -1.041 1.362 0.13% 1395.505op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 57.368µs 57.592µs ± 0.123µs 57.574µs ± 0.075µs 57.651µs 57.841µs 57.962µs 58.111µs 0.93% 1.166 1.900 0.21% 0.009µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 17208377.830op/s 17363509.644op/s ± 37077.833op/s 17369046.622op/s ± 22590.263op/s 17389793.062op/s 17413890.538op/s 17421880.368op/s 17431266.463op/s 0.36% -1.150 1.846 0.21% 2621.799op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.893µs 3.911µs ± 0.003µs 3.911µs ± 0.002µs 3.913µs 3.916µs 3.917µs 3.918µs 0.16% -1.061 7.172 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 255252058.163op/s 255669922.218op/s ± 184707.648op/s 255662660.469op/s ± 127378.679op/s 255807643.858op/s 255914697.080op/s 255964700.624op/s 256858818.560op/s 0.47% 1.079 7.293 0.07% 13060.803op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 54.555µs 54.782µs ± 0.149µs 54.745µs ± 0.074µs 54.840µs 55.068µs 55.269µs 55.483µs 1.35% 1.636 3.578 0.27% 0.011µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 18023480.163op/s 18254425.776op/s ± 49383.078op/s 18266388.021op/s ± 24747.027op/s 18287544.814op/s 18309913.629op/s 18322936.950op/s 18330052.120op/s 0.35% -1.613 3.468 0.27% 3491.911op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 52.124µs 52.191µs ± 0.041µs 52.186µs ± 0.028µs 52.215µs 52.265µs 52.289µs 52.385µs 0.38% 0.992 1.678 0.08% 0.003µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 19089255.593op/s 19160267.547op/s ± 15006.917op/s 19162109.809op/s ± 10212.326op/s 19172104.110op/s 19179407.741op/s 19182451.167op/s 19184968.577op/s 0.12% -0.985 1.649 0.08% 1061.149op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 5.693µs 5.865µs ± 0.068µs 5.869µs ± 0.046µs 5.916µs 5.967µs 6.007µs 6.045µs 3.00% -0.188 -0.255 1.16% 0.005µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 165429000.111op/s 170527041.181op/s ± 1994186.888op/s 170391265.752op/s ± 1334781.329op/s 171701342.303op/s 173935022.681op/s 175415410.699op/s 175661104.888op/s 3.09% 0.248 -0.235 1.17% 141010.307op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.911µs; 3.912µs] or [-0.012%; +0.012%] None None None
credit_card/is_card_number/ throughput [255621477.446op/s; 255682723.240op/s] or [-0.012%; +0.012%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [81.768µs; 81.984µs] or [-0.131%; +0.131%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12198575.250op/s; 12230782.446op/s] or [-0.132%; +0.132%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [74.885µs; 74.932µs] or [-0.031%; +0.031%] None None None
credit_card/is_card_number/ 378282246310005 throughput [13345527.726op/s; 13353829.495op/s] or [-0.031%; +0.031%] None None None
credit_card/is_card_number/37828224631 execution_time [3.910µs; 3.911µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/37828224631 throughput [255692147.926op/s; 255750112.433op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number/378282246310005 execution_time [72.039µs; 72.081µs] or [-0.029%; +0.029%] None None None
credit_card/is_card_number/378282246310005 throughput [13873313.560op/s; 13881413.222op/s] or [-0.029%; +0.029%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [52.175µs; 52.183µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [19163356.468op/s; 19166367.376op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/x371413321323331 execution_time [5.838µs; 5.861µs] or [-0.195%; +0.195%] None None None
credit_card/is_card_number/x371413321323331 throughput [170652880.573op/s; 171321689.154op/s] or [-0.196%; +0.196%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.910µs; 3.911µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ throughput [255683548.232op/s; 255737595.177op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [64.285µs; 64.307µs] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15550314.419op/s; 15555784.699op/s] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [57.575µs; 57.609µs] or [-0.030%; +0.030%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17358371.013op/s; 17368648.275op/s] or [-0.030%; +0.030%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.911µs; 3.912µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255644323.514op/s; 255695520.921op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [54.761µs; 54.802µs] or [-0.038%; +0.038%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [18247581.756op/s; 18261269.795op/s] or [-0.037%; +0.037%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [52.186µs; 52.197µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [19158187.733op/s; 19162347.362op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [5.855µs; 5.874µs] or [-0.162%; +0.162%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [170250666.057op/s; 170803416.304op/s] or [-0.162%; +0.162%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 247.110ns 257.032ns ± 12.649ns 251.523ns ± 3.204ns 260.716ns 283.886ns 298.613ns 298.880ns 18.83% 1.684 1.863 4.91% 0.894ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [255.279ns; 258.785ns] or [-0.682%; +0.682%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 91f5f4d 1753431179 ivoanjo/fix-ruby-tests-on-ruby-head
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 60.564ms 61.001ms ± 2.260ms 60.690ms ± 0.040ms 60.737ms 61.044ms 78.423ms 79.825ms 31.53% 7.946 61.454 3.70% 0.160ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [60.688ms; 61.314ms] or [-0.513%; +0.513%] None None None

Baseline

Omitted due to size.

@datadog-datadog-prod-us1
Copy link

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 91f5f4d | Was this helpful? Give us feedback!

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.04%. Comparing base (740b396) to head (91f5f4d).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1153      +/-   ##
==========================================
- Coverage   71.05%   71.04%   -0.02%     
==========================================
  Files         348      348              
  Lines       53839    53839              
==========================================
- Hits        38254    38248       -6     
- Misses      15585    15591       +6     
Components Coverage Δ
datadog-crashtracker 43.94% <ø> (+0.02%) ⬆️
datadog-crashtracker-ffi 5.93% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 86.82% <ø> (ø)
data-pipeline-ffi 88.36% <ø> (ø)
ddcommon 82.99% <ø> (ø)
ddcommon-ffi 70.01% <ø> (ø)
ddtelemetry 60.08% <ø> (ø)
ddtelemetry-ffi 21.32% <ø> (ø)
dogstatsd-client 83.26% <ø> (ø)
datadog-ipc 82.58% <ø> (ø)
datadog-profiling 77.13% <ø> (ø)
datadog-profiling-ffi 62.12% <ø> (ø)
datadog-sidecar 39.10% <ø> (ø)
datdog-sidecar-ffi 11.55% <ø> (ø)
spawn-worker 55.35% <ø> (ø)
tinybytes 91.59% <ø> (ø)
datadog-trace-normalization 98.24% <ø> (ø)
datadog-trace-obfuscation 94.17% <ø> (ø)
datadog-trace-protobuf 77.10% <ø> (ø)
datadog-trace-utils 89.94% <ø> (ø)
datadog-tracer-flare 75.14% <ø> (ø)
datadog-log 76.31% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ivoanjo
Copy link
Member Author

ivoanjo commented Jul 25, 2025

/merge

@dd-devflow-routing-codex
Copy link

dd-devflow-routing-codex bot commented Jul 25, 2025

View all feedbacks in Devflow UI.

2025-07-25 12:31:04 UTC ℹ️ Start processing command /merge


2025-07-25 12:31:11 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 39m (p90).


2025-07-25 13:08:08 UTC ℹ️ MergeQueue: This merge request was merged

@dd-mergequeue dd-mergequeue bot merged commit 8014aa9 into main Jul 25, 2025
47 checks passed
@dd-mergequeue dd-mergequeue bot deleted the ivoanjo/fix-ruby-tests-on-ruby-head branch July 25, 2025 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants