Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate benchmarks for comparison from common bechmarks #150

Merged
merged 6 commits into from
Jul 2, 2024

Conversation

OptimumCode
Copy link
Owner

No description provided.

@OptimumCode OptimumCode added the repository enhancement Changes to make the repository look better label Jul 2, 2024
Copy link

codecov bot commented Jul 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.68%. Comparing base (1ce535f) to head (096cc11).
Report is 6 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #150   +/-   ##
=======================================
  Coverage   90.68%   90.68%           
=======================================
  Files         130      130           
  Lines        3842     3842           
  Branches     1048     1048           
=======================================
  Hits         3484     3484           
  Misses         65       65           
  Partials      293      293           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'KMP JSON schema validator'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 0.50.

Benchmark suite Current: b57be65 Previous: 88e2fb1 Ratio
jvm.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 2605.658603613708 ops/s 2653.595932185305 ops/s 0.98
jvm.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 2565.4568858514504 ops/s 2596.966861257367 ops/s 0.99
jvm.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 2469.8908528423217 ops/s 2479.338639230217 ops/s 1.00
jvm.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 2358.8478936017023 ops/s 2284.539603300548 ops/s 1.03
jvm.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 896.2193200363384 ops/s 941.7114536137551 ops/s 0.95
jvm.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 917.3652112908449 ops/s 922.0441016382626 ops/s 0.99
jvm.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 3641.821813286976 ops/s 3579.9316553633844 ops/s 1.02
jvm.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 8044083.825494352 ops/s 7621300.44650594 ops/s 1.06
jvm.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 683.5218079038149 ops/s 674.0463841540486 ops/s 1.01
jvm.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 672.5670941160065 ops/s 666.3336069371293 ops/s 1.01
jvm.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 384.06815440893104 us/op 391.9498227495268 us/op 0.98
jvm.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 387.2138842149635 us/op 382.56600177969256 us/op 1.01
jvm.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 387.6856228290704 us/op 393.5684466125768 us/op 0.99
jvm.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 429.01818136369764 us/op 426.70684804938884 us/op 1.01
jvm.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1054.1006489117112 us/op 1076.276657959926 us/op 0.98
jvm.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1085.6996814756985 us/op 1096.1003341788087 us/op 0.99
jvm.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 276.59141658889035 us/op 284.91422327214264 us/op 0.97
jvm.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 0.12806074761112377 us/op 0.12477345211023014 us/op 1.03
jvm.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1483.1398087010261 us/op 1476.3657089369349 us/op 1.00
jvm.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1482.4918518206825 us/op 1481.3692200056958 us/op 1.00
linuxX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1753.5099510309278 us/op 1617.3969675409837 us/op 1.08
linuxX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1757.452131793961 us/op 1703.4540571428574 us/op 1.03
linuxX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 3858.502333463035 us/op 3776.909825 us/op 1.02
linuxX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 3920.84222578125 us/op 3839.7078536121676 us/op 1.02
linuxX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1125.9156673076925 us/op 1126.848831171786 us/op 1.00
linuxX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 0.8320370619436972 us/op 0.8387993731003904 us/op 0.99
linuxX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 6191.562040490798 us/op 5500.8891117021285 us/op 1.13
linuxX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 5857.01443 us/op 5502.535467032967 us/op 1.06
linuxX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1562.5010475590552 us/op 1548.702371121352 us/op 1.01
linuxX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1595.943885055644 us/op 1593.0107977198695 us/op 1.00
linuxX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 608.395496866347 ops/sec 622.1350901250372 ops/sec 0.98
linuxX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 566.4132159911586 ops/sec 588.1531661342109 ops/sec 0.96
linuxX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 258.1365425629106 ops/sec 259.9879196093481 ops/sec 0.99
linuxX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 256.26905654958244 ops/sec 255.45221639373344 ops/sec 1.00
linuxX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 891.9863656041161 ops/sec 892.2849675464407 ops/sec 1.00
linuxX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1186554.3960522749 ops/sec 1199954.1195795832 ops/sec 0.99
linuxX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 167.0979750759658 ops/sec 177.95366701748884 ops/sec 0.94
linuxX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 169.57470154964693 ops/sec 180.69576999638036 ops/sec 0.94
linuxX64.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 644.112655490751 ops/sec 651.3011940200743 ops/sec 0.99
linuxX64.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 623.2226478783192 ops/sec 623.3692880507754 ops/sec 1.00
macosArm64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1043.560785235602 us/op 1325.9605106249999 us/op 0.79
macosArm64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1136.9097669265034 us/op 1317.886141011984 us/op 0.86
macosArm64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 2544.8678464285717 us/op 3004.6197703812313 us/op 0.85
macosArm64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 2530.3239421465964 us/op 2970.9433105421685 us/op 0.85
macosArm64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 752.8182058130978 us/op 841.7279331805682 us/op 0.89
macosArm64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 0.5736408076258213 us/op 0.634891532725674 us/op 0.90
macosArm64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 3529.610580357142 us/op 4609.096784474887 us/op 0.77
macosArm64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 3669.9671195652177 us/op 4869.894245714286 us/op 0.75
macosArm64.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1001.5346264914053 us/op 1141.4959121893492 us/op 0.88
macosArm64.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1014.9402397979796 us/op 1250.8724039340102 us/op 0.81
macosArm64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 953.4353612172229 ops/sec 794.8628645980135 ops/sec 1.20
macosArm64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 882.1347019075071 ops/sec 732.0228387143751 ops/sec 1.21
macosArm64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 398.6530565353647 ops/sec 336.54259684903155 ops/sec 1.18
macosArm64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 396.3335416767539 ops/sec 335.0481939684594 ops/sec 1.18
macosArm64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1379.1583214000827 ops/sec 1209.0794387942765 ops/sec 1.14
macosArm64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1755772.763422075 ops/sec 1574950.1674223018 ops/sec 1.11
macosArm64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 260.1056756080779 ops/sec 214.4373867323598 ops/sec 1.21
macosArm64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 281.2879339712196 ops/sec 224.9835488830835 ops/sec 1.25
macosArm64.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 1009.0735939660741 ops/sec 837.0111590713557 ops/sec 1.21
macosArm64.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 982.3634025399876 ops/sec 806.6204898450799 ops/sec 1.22
macosX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 2797.157587811634 us/op 2973.168883775811 us/op 0.94
macosX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 3024.098404140127 us/op 3350.707884539473 us/op 0.90
macosX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 6108.410110967741 us/op 6495.621247468354 us/op 0.94
macosX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 6260.548285276074 us/op 6899.0915493589755 us/op 0.91
macosX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 2008.02796031746 us/op 2364.0163889624723 us/op 0.85
macosX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1.627560581158944 us/op 1.8179743787817926 us/op 0.90
macosX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 8944.784581249998 us/op 9010.80259099099 us/op 0.99
macosX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 8895.103347272725 us/op 9141.43195625 us/op 0.97
macosX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 2691.884683923706 us/op 2933.309914457831 us/op 0.92
macosX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 2707.4215760107822 us/op 3001.459158064516 us/op 0.90
macosX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 358.7136834475051 ops/sec 349.530087997283 ops/sec 1.03
macosX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 341.4295541681428 ops/sec 336.15833230707284 ops/sec 1.02
macosX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 165.00864938341365 ops/sec 162.56661904420747 ops/sec 1.02
macosX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 162.43808455208207 ops/sec 159.57151156408517 ops/sec 1.02
macosX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 503.499394434927 ops/sec 498.588087641442 ops/sec 1.01
macosX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 618247.4602462914 ops/sec 607141.1649394365 ops/sec 1.02
macosX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 109.07795051548935 ops/sec 107.43695655337476 ops/sec 1.02
macosX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 110.64269166508959 ops/sec 109.7812831488466 ops/sec 1.01
macosX64.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 371.69070605074586 ops/sec 369.10990719511346 ops/sec 1.01
macosX64.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 366.89214530813376 ops/sec 357.5052004879461 ops/sec 1.03
mingwX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 3442.059075342466 us/op 3548.5640213523134 us/op 0.97
mingwX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 3354.7761129568107 us/op 3468.628424657535 us/op 0.97
mingwX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 7464.100814814815 us/op 7618.2869465648855 us/op 0.98
mingwX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 7519.5988721804515 us/op 7765.591627906976 us/op 0.97
mingwX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 2468.520024752475 us/op 2495.563190954774 us/op 0.99
mingwX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 1.9743255153574548 us/op 2.1892058433976804 us/op 0.90
mingwX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 10690.470860215053 us/op 10752.24311827957 us/op 0.99
mingwX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 10197.322187499998 us/op 10449.288315789474 us/op 0.98
mingwX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 3418.417010309279 us/op 3400.754206896552 us/op 1.01
mingwX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 3398.268979591837 us/op 3508.3286896551717 us/op 0.97
mingwX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 287.17869098345744 ops/sec 288.8282242281345 ops/sec 0.99
mingwX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 292.0078550746905 ops/sec 289.1167014649221 ops/sec 1.01
mingwX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 132.71539927511466 ops/sec 131.07886556185593 ops/sec 1.01
mingwX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 130.59757838881276 ops/sec 129.36341239458744 ops/sec 1.01
mingwX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 401.39257805540035 ops/sec 400.15282456266357 ops/sec 1.00
mingwX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 501066.2608475623 ops/sec 492992.2223834451 ops/sec 1.02
mingwX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 94.01695879962955 ops/sec 93.52287753321602 ops/sec 1.01
mingwX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 96.66830959607428 ops/sec 95.94858778067656 ops/sec 1.01
mingwX64.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 291.7953278669171 ops/sec 293.2529576487629 ops/sec 1.00
mingwX64.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) 293.57382731130906 ops/sec 290.0971237181231 ops/sec 1.01

This comment was automatically generated by workflow using github-action-benchmark.

CC: @OptimumCode

@OptimumCode OptimumCode merged commit 6c5d8db into main Jul 2, 2024
18 checks passed
@OptimumCode OptimumCode deleted the separate-benchmarks branch July 2, 2024 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
repository enhancement Changes to make the repository look better
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant