Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support x-datadog-test-session-token for parallel tests with the test agent #578

Merged
merged 1 commit into from
Aug 12, 2024
Merged

Support x-datadog-test-session-token for parallel tests with the test agent #578

merged 1 commit into from
Aug 12, 2024

Conversation

bwoebi
Copy link
Contributor

@bwoebi bwoebi commented Aug 6, 2024

I've decided to put it into Endpoint, as it has to be used at basically any place Endpoint is used.

@bwoebi bwoebi requested review from a team as code owners August 6, 2024 11:34
@codecov-commenter
Copy link

codecov-commenter commented Aug 6, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 27.18447% with 75 lines in your changes missing coverage. Please review.

Project coverage is 71.13%. Comparing base (b9d4da7) to head (1d00fa9).
Report is 57 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #578      +/-   ##
==========================================
- Coverage   71.27%   71.13%   -0.15%     
==========================================
  Files         220      220              
  Lines       29929    30015      +86     
==========================================
+ Hits        21331    21350      +19     
- Misses       8598     8665      +67
Components Coverage Δ
crashtracker 21.17% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 50.00% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 82.78% <50.00%> (-0.30%) ⬇️
ddcommon-ffi 69.72% <44.44%> (-0.49%) ⬇️
ddtelemetry 59.02% <ø> (ø)
ipc 84.18% <ø> (ø)
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 34.03% <4.34%> (-0.52%) ⬇️
sidecar-ffi 0.00% <0.00%> (ø)
spawn-worker 54.87% <ø> (ø)
trace-mini-agent 70.88% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.16% <ø> (ø)
trace-utils 92.97% <94.44%> (+<0.01%) ⬆️

@pr-commenter
Copy link

pr-commenter bot commented Aug 6, 2024 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2024-08-12 16:50:52

Comparing candidate commit 1d00fa9 in PR branch bob/test-session-token with baseline commit b9d4da7 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 133.996µs 134.368µs ± 0.206µs 134.347µs ± 0.130µs 134.472µs 134.708µs 135.115µs 135.239µs 0.66% 1.275 2.769 0.15% 0.015µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [134.340µs; 134.397µs] or [-0.021%; +0.021%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 280.848µs 282.783µs ± 2.542µs 281.410µs ± 0.295µs 283.544µs 288.703µs 289.769µs 289.901µs 3.02% 1.544 1.067 0.90% 0.180µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3449447.690op/s 3536556.447op/s ± 31360.450op/s 3553532.466op/s ± 3730.358op/s 3555973.444op/s 3558642.942op/s 3559368.081op/s 3560644.030op/s 0.20% -1.526 1.001 0.88% 2217.519op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 27.191µs 27.493µs ± 0.159µs 27.495µs ± 0.119µs 27.600µs 27.744µs 27.832µs 27.978µs 1.75% 0.222 -0.454 0.58% 0.011µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 35742867.851op/s 36373809.876op/s ± 210716.270op/s 36370110.561op/s ± 158495.962op/s 36549195.585op/s 36683329.741op/s 36768157.028op/s 36776797.413op/s 1.12% -0.196 -0.486 0.58% 14899.890op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 16.569µs 16.619µs ± 0.175µs 16.598µs ± 0.020µs 16.622µs 16.684µs 16.730µs 19.028µs 14.64% 13.154 178.521 1.05% 0.012µs 1 200
normalization/normalize_name/normalize_name/good throughput 52554177.967op/s 60177117.699op/s ± 557211.351op/s 60248688.047op/s ± 71224.020op/s 60319064.652op/s 60336912.035op/s 60347354.111op/s 60352240.435op/s 0.17% -12.906 173.913 0.92% 39400.793op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [282.431µs; 283.136µs] or [-0.125%; +0.125%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3532210.191op/s; 3540902.704op/s] or [-0.123%; +0.123%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [27.471µs; 27.515µs] or [-0.080%; +0.080%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [36344606.627op/s; 36403013.124op/s] or [-0.080%; +0.080%] None None None
normalization/normalize_name/normalize_name/good execution_time [16.595µs; 16.643µs] or [-0.146%; +0.146%] None None None
normalization/normalize_name/normalize_name/good throughput [60099893.565op/s; 60254341.833op/s] or [-0.128%; +0.128%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.149µs 1.348µs ± 0.061µs 1.352µs ± 0.050µs 1.400µs 1.431µs 1.440µs 1.441µs 6.61% -0.687 0.644 4.50% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.339µs; 1.356µs] or [-0.626%; +0.626%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 73.946µs 74.102µs ± 0.139µs 74.078µs ± 0.039µs 74.124µs 74.249µs 74.582µs 75.501µs 1.92% 6.692 58.303 0.19% 0.010µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [74.082µs; 74.121µs] or [-0.026%; +0.026%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.641µs 2.696µs ± 0.019µs 2.697µs ± 0.005µs 2.702µs 2.735µs 2.737µs 2.740µs 1.59% -0.698 2.787 0.69% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.693µs; 2.699µs] or [-0.095%; +0.095%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 2.415µs 2.416µs ± 0.001µs 2.416µs ± 0.001µs 2.416µs 2.417µs 2.417µs 2.418µs 0.08% 0.495 -0.531 0.03% 0.000µs 1 200
credit_card/is_card_number/ throughput 413601958.429op/s 413939172.823op/s ± 107211.953op/s 413947728.175op/s ± 86506.971op/s 414033722.544op/s 414079604.959op/s 414093322.369op/s 414107716.431op/s 0.04% -0.494 -0.533 0.03% 7581.030op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 105.272µs 108.554µs ± 2.704µs 107.805µs ± 2.130µs 111.704µs 112.561µs 113.349µs 113.592µs 5.37% 0.348 -1.448 2.48% 0.191µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 8803418.329op/s 9217685.014op/s ± 227719.331op/s 9275989.923op/s ± 186233.359op/s 9436861.207op/s 9484291.839op/s 9497371.851op/s 9499185.393op/s 2.41% -0.316 -1.471 2.46% 16102.188op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 100.861µs 104.405µs ± 2.342µs 103.689µs ± 1.495µs 105.922µs 108.922µs 110.049µs 111.666µs 7.69% 0.799 -0.288 2.24% 0.166µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 8955246.436op/s 9582819.901op/s ± 211324.881op/s 9644208.884op/s ± 140764.861op/s 9746764.407op/s 9837114.547op/s 9874531.995op/s 9914621.681op/s 2.80% -0.729 -0.428 2.20% 14942.926op/s 1 200
credit_card/is_card_number/37828224631 execution_time 2.415µs 2.416µs ± 0.005µs 2.416µs ± 0.001µs 2.416µs 2.417µs 2.418µs 2.487µs 2.93% 13.612 187.121 0.21% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 402142975.232op/s 413867730.576op/s ± 842239.416op/s 413940702.999op/s ± 88113.407op/s 414023526.887op/s 414090963.362op/s 414124032.633op/s 414159285.661op/s 0.05% -13.588 186.671 0.20% 59555.320op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 97.789µs 101.279µs ± 2.890µs 100.035µs ± 1.745µs 104.330µs 106.014µs 106.498µs 107.228µs 7.19% 0.501 -1.306 2.85% 0.204µs 1 200
credit_card/is_card_number/378282246310005 throughput 9325920.594op/s 9881562.157op/s ± 278114.052op/s 9996501.991op/s ± 175710.858op/s 10118530.770op/s 10203648.718op/s 10217568.606op/s 10226065.587op/s 2.30% -0.463 -1.345 2.81% 19665.633op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 105.399µs 108.445µs ± 0.904µs 108.452µs ± 0.622µs 109.045µs 109.830µs 111.045µs 111.201µs 2.53% 0.048 0.616 0.83% 0.064µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 8992753.893op/s 9221899.161op/s ± 76832.796op/s 9220671.861op/s ± 52909.615op/s 9274109.876op/s 9338990.379op/s 9386725.007op/s 9487790.297op/s 2.90% 0.017 0.610 0.83% 5432.899op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 22.472µs 23.649µs ± 0.568µs 23.642µs ± 0.394µs 24.030µs 24.573µs 24.844µs 25.053µs 5.97% 0.076 -0.544 2.40% 0.040µs 1 200
credit_card/is_card_number/x371413321323331 throughput 39915153.939op/s 42310134.582op/s ± 1016406.222op/s 42298475.349op/s ± 696550.021op/s 43033697.184op/s 43960759.958op/s 44470016.814op/s 44500751.422op/s 5.21% 0.027 -0.565 2.40% 71870.773op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 2.414µs 2.416µs ± 0.001µs 2.416µs ± 0.001µs 2.416µs 2.417µs 2.417µs 2.418µs 0.10% 0.610 0.077 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 413560013.502op/s 413946334.515op/s ± 110106.934op/s 413952954.247op/s ± 85820.485op/s 414039768.521op/s 414088811.698op/s 414113823.753op/s 414165583.400op/s 0.05% -0.609 0.073 0.03% 7785.736op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 89.258µs 90.017µs ± 0.521µs 89.904µs ± 0.238µs 90.226µs 90.987µs 91.813µs 92.595µs 2.99% 1.590 3.916 0.58% 0.037µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 10799775.442op/s 11109356.899op/s ± 63755.796op/s 11122939.726op/s ± 29498.486op/s 11148315.300op/s 11193468.171op/s 11202115.542op/s 11203475.318op/s 0.72% -1.533 3.635 0.57% 4508.216op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 83.089µs 85.526µs ± 1.415µs 85.405µs ± 0.986µs 86.357µs 88.355µs 88.884µs 88.992µs 4.20% 0.529 -0.297 1.65% 0.100µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 11236976.992op/s 11695477.567op/s ± 191943.321op/s 11708857.754op/s ± 134529.718op/s 11848443.561op/s 11971543.499op/s 12007921.839op/s 12035299.843op/s 2.79% -0.459 -0.387 1.64% 13572.442op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 2.415µs 2.416µs ± 0.001µs 2.416µs ± 0.000µs 2.416µs 2.417µs 2.418µs 2.419µs 0.12% 0.775 1.124 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 413449959.719op/s 413920368.251op/s ± 112977.677op/s 413930879.249op/s ± 85314.396op/s 414013377.564op/s 414068871.792op/s 414101046.888op/s 414116541.540op/s 0.04% -0.773 1.116 0.03% 7988.728op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 81.960µs 84.736µs ± 0.889µs 84.973µs ± 0.554µs 85.374µs 85.878µs 86.071µs 86.136µs 1.37% -0.830 0.179 1.05% 0.063µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 11609522.563op/s 11802615.898op/s ± 124884.303op/s 11768442.342op/s ± 77214.373op/s 11875827.754op/s 12038246.078op/s 12164911.119op/s 12201098.926op/s 3.68% 0.877 0.288 1.06% 8830.654op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 104.645µs 108.154µs ± 1.030µs 108.144µs ± 0.598µs 108.797µs 109.922µs 110.392µs 111.059µs 2.70% -0.145 0.562 0.95% 0.073µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 9004216.006op/s 9246875.714op/s ± 88182.470op/s 9246900.622op/s ± 51391.429op/s 9296630.948op/s 9399935.254op/s 9468623.597op/s 9556154.636op/s 3.34% 0.218 0.626 0.95% 6235.442op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 22.293µs 23.710µs ± 0.551µs 23.706µs ± 0.375µs 24.052µs 24.611µs 25.009µs 25.104µs 5.90% 0.073 -0.254 2.32% 0.039µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 39834304.397op/s 42198558.556op/s ± 979861.289op/s 42183873.119op/s ± 670832.886op/s 42895162.818op/s 43885165.422op/s 44271917.003op/s 44857969.252op/s 6.34% 0.047 -0.276 2.32% 69286.656op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [2.416µs; 2.416µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number/ throughput [413924314.278op/s; 413954031.369op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [108.179µs; 108.928µs] or [-0.345%; +0.345%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [9186125.305op/s; 9249244.723op/s] or [-0.342%; +0.342%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [104.080µs; 104.729µs] or [-0.311%; +0.311%] None None None
credit_card/is_card_number/ 378282246310005 throughput [9553532.305op/s; 9612107.497op/s] or [-0.306%; +0.306%] None None None
credit_card/is_card_number/37828224631 execution_time [2.416µs; 2.417µs] or [-0.029%; +0.029%] None None None
credit_card/is_card_number/37828224631 throughput [413751004.294op/s; 413984456.859op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/378282246310005 execution_time [100.879µs; 101.680µs] or [-0.395%; +0.395%] None None None
credit_card/is_card_number/378282246310005 throughput [9843018.224op/s; 9920106.090op/s] or [-0.390%; +0.390%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [108.320µs; 108.570µs] or [-0.115%; +0.115%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [9211250.875op/s; 9232547.448op/s] or [-0.115%; +0.115%] None None None
credit_card/is_card_number/x371413321323331 execution_time [23.570µs; 23.727µs] or [-0.333%; +0.333%] None None None
credit_card/is_card_number/x371413321323331 throughput [42169270.455op/s; 42450998.709op/s] or [-0.333%; +0.333%] None None None
credit_card/is_card_number_no_luhn/ execution_time [2.416µs; 2.416µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/ throughput [413931074.753op/s; 413961594.277op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [89.945µs; 90.089µs] or [-0.080%; +0.080%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [11100520.959op/s; 11118192.839op/s] or [-0.080%; +0.080%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [85.330µs; 85.722µs] or [-0.229%; +0.229%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [11668876.068op/s; 11722079.065op/s] or [-0.227%; +0.227%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [2.416µs; 2.416µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [413904710.631op/s; 413936025.870op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [84.613µs; 84.859µs] or [-0.145%; +0.145%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [11785308.135op/s; 11819923.662op/s] or [-0.147%; +0.147%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [108.012µs; 108.297µs] or [-0.132%; +0.132%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [9234654.472op/s; 9259096.956op/s] or [-0.132%; +0.132%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [23.634µs; 23.787µs] or [-0.322%; +0.322%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [42062759.205op/s; 42334357.907op/s] or [-0.322%; +0.322%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.440µs 2.919µs ± 1.436µs 2.754µs ± 0.020µs 2.769µs 2.983µs 12.793µs 16.426µs 496.33% 8.006 64.410 49.09% 0.102µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.720µs; 3.118µs] or [-6.821%; +6.821%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 505.930µs 506.833µs ± 0.955µs 506.632µs ± 0.180µs 506.830µs 507.718µs 511.592µs 513.479µs 1.35% 4.742 24.696 0.19% 0.067µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1947497.975op/s 1973042.248op/s ± 3683.079op/s 1973818.574op/s ± 699.556op/s 1974405.778op/s 1975330.045op/s 1975889.603op/s 1976559.631op/s 0.14% -4.718 24.471 0.19% 260.433op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 389.471µs 391.624µs ± 1.170µs 391.417µs ± 0.742µs 392.414µs 393.611µs 394.110µs 396.271µs 1.24% 0.769 1.050 0.30% 0.083µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2523527.593op/s 2553490.488op/s ± 7610.893op/s 2554819.458op/s ± 4844.011op/s 2558752.373op/s 2564304.585op/s 2566913.247op/s 2567585.563op/s 0.50% -0.747 0.979 0.30% 538.171op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 181.120µs 181.508µs ± 0.127µs 181.530µs ± 0.078µs 181.606µs 181.666µs 181.738µs 181.766µs 0.13% -0.544 -0.545 0.07% 0.009µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5501581.223op/s 5509414.533op/s ± 3859.655op/s 5508737.408op/s ± 2363.712op/s 5511485.108op/s 5515754.294op/s 5516423.435op/s 5521191.506op/s 0.23% 0.547 -0.543 0.07% 272.919op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 45.460µs 45.656µs ± 0.075µs 45.652µs ± 0.050µs 45.707µs 45.774µs 45.821µs 45.843µs 0.42% -0.153 -0.165 0.16% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 21813446.662op/s 21903012.816op/s ± 35814.127op/s 21904913.154op/s ± 23818.957op/s 21924776.977op/s 21966757.150op/s 21985081.327op/s 21997465.509op/s 0.42% 0.162 -0.162 0.16% 2532.441op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 50.499µs 50.821µs ± 0.118µs 50.815µs ± 0.090µs 50.914µs 51.011µs 51.080µs 51.117µs 0.60% 0.039 -0.328 0.23% 0.008µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19562927.378op/s 19677157.956op/s ± 45580.470op/s 19679331.095op/s ± 34932.307op/s 19709534.794op/s 19743184.375op/s 19777041.973op/s 19802275.445op/s 0.62% -0.027 -0.325 0.23% 3223.026op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [506.701µs; 506.966µs] or [-0.026%; +0.026%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1972531.809op/s; 1973552.687op/s] or [-0.026%; +0.026%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [391.462µs; 391.786µs] or [-0.041%; +0.041%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2552435.692op/s; 2554545.285op/s] or [-0.041%; +0.041%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [181.490µs; 181.525µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5508879.622op/s; 5509949.444op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [45.646µs; 45.666µs] or [-0.023%; +0.023%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [21898049.322op/s; 21907976.310op/s] or [-0.023%; +0.023%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [50.804µs; 50.837µs] or [-0.032%; +0.032%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [19670840.941op/s; 19683474.970op/s] or [-0.032%; +0.032%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 36.842µs 37.524µs ± 1.293µs 36.943µs ± 0.053µs 36.987µs 40.312µs 40.358µs 40.910µs 10.74% 1.705 0.934 3.44% 0.091µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [37.345µs; 37.703µs] or [-0.477%; +0.477%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 297.740ns 307.455ns ± 12.627ns 301.077ns ± 2.603ns 309.036ns 336.953ns 338.950ns 359.098ns 19.27% 1.675 1.836 4.10% 0.893ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [305.705ns; 309.205ns] or [-0.569%; +0.569%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1d00fa9 1723480858 bob/test-session-token
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 16.839µs 21.279µs ± 13.103µs 16.970µs ± 0.046µs 17.040µs 51.656µs 53.134µs 138.741µs 717.55% 4.695 32.074 61.42% 0.927µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [19.463µs; 23.095µs] or [-8.534%; +8.534%] None None None

Baseline

Omitted due to size.

@pawelchcki
Copy link
Contributor

scenario:credit_card/is_card_number/37828224631000521389798
🟥 execution_time [+11.565ns; +12.044ns] or [+10.646%; +11.087%]
🟥 throughput [-918917.691op/s; -883968.291op/s] or [-9.982%; -9.602%]

This is benchmark is propably sensing larger binary, or just depends too much memory layout. We should make sure its not that sensitive.

@pawelchcki
Copy link
Contributor

This is benchmark is propably sensing larger binary, or just depends too much memory layout. We should make sure its not that sensitive.

I see @bantonsson is wokring on this already in #576.

Lets safely ignore the benchmark resutls in this PR- these changes shouldn't cause any detectable regression.

@bwoebi bwoebi force-pushed the bob/test-session-token branch 3 times, most recently from f337fcb to f803a89 Compare August 12, 2024 16:33
@bwoebi
Support x-datadog-test-session-token for parallel tests with the test…
1d00fa9 
… agent

Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi merged commit 16528ff into main Aug 12, 2024
34 checks passed
@bwoebi bwoebi deleted the bob/test-session-token branch August 12, 2024 17:10
hoolioh pushed a commit that referenced this pull request Aug 21, 2024
@bwoebi @hoolioh
Support x-datadog-test-session-token for parallel tests with the test…
f38aa17 
… agent (#578)

Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pawelchcki pawelchcki pawelchcki approved these changes

Assignees
No one assigned
Labels
common mini-agent sidecar
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bwoebi @codecov-commenter @pawelchcki