-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Separate benchmarks for comparison from common bechmarks #150
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #150 +/- ##
=======================================
Coverage 90.68% 90.68%
=======================================
Files 130 130
Lines 3842 3842
Branches 1048 1048
=======================================
Hits 3484 3484
Misses 65 65
Partials 293 293 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'KMP JSON schema validator'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 0.50
.
Benchmark suite | Current: b57be65 | Previous: 88e2fb1 | Ratio |
---|---|---|---|
jvm.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
2605.658603613708 ops/s |
2653.595932185305 ops/s |
0.98 |
jvm.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
2565.4568858514504 ops/s |
2596.966861257367 ops/s |
0.99 |
jvm.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
2469.8908528423217 ops/s |
2479.338639230217 ops/s |
1.00 |
jvm.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
2358.8478936017023 ops/s |
2284.539603300548 ops/s |
1.03 |
jvm.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
896.2193200363384 ops/s |
941.7114536137551 ops/s |
0.95 |
jvm.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
917.3652112908449 ops/s |
922.0441016382626 ops/s |
0.99 |
jvm.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
3641.821813286976 ops/s |
3579.9316553633844 ops/s |
1.02 |
jvm.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
8044083.825494352 ops/s |
7621300.44650594 ops/s |
1.06 |
jvm.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
683.5218079038149 ops/s |
674.0463841540486 ops/s |
1.01 |
jvm.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
672.5670941160065 ops/s |
666.3336069371293 ops/s |
1.01 |
jvm.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
384.06815440893104 us/op |
391.9498227495268 us/op |
0.98 |
jvm.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
387.2138842149635 us/op |
382.56600177969256 us/op |
1.01 |
jvm.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
387.6856228290704 us/op |
393.5684466125768 us/op |
0.99 |
jvm.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
429.01818136369764 us/op |
426.70684804938884 us/op |
1.01 |
jvm.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1054.1006489117112 us/op |
1076.276657959926 us/op |
0.98 |
jvm.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1085.6996814756985 us/op |
1096.1003341788087 us/op |
0.99 |
jvm.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
276.59141658889035 us/op |
284.91422327214264 us/op |
0.97 |
jvm.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
0.12806074761112377 us/op |
0.12477345211023014 us/op |
1.03 |
jvm.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1483.1398087010261 us/op |
1476.3657089369349 us/op |
1.00 |
jvm.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1482.4918518206825 us/op |
1481.3692200056958 us/op |
1.00 |
linuxX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1753.5099510309278 us/op |
1617.3969675409837 us/op |
1.08 |
linuxX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1757.452131793961 us/op |
1703.4540571428574 us/op |
1.03 |
linuxX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
3858.502333463035 us/op |
3776.909825 us/op |
1.02 |
linuxX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
3920.84222578125 us/op |
3839.7078536121676 us/op |
1.02 |
linuxX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1125.9156673076925 us/op |
1126.848831171786 us/op |
1.00 |
linuxX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
0.8320370619436972 us/op |
0.8387993731003904 us/op |
0.99 |
linuxX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
6191.562040490798 us/op |
5500.8891117021285 us/op |
1.13 |
linuxX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
5857.01443 us/op |
5502.535467032967 us/op |
1.06 |
linuxX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1562.5010475590552 us/op |
1548.702371121352 us/op |
1.01 |
linuxX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1595.943885055644 us/op |
1593.0107977198695 us/op |
1.00 |
linuxX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
608.395496866347 ops/sec |
622.1350901250372 ops/sec |
0.98 |
linuxX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
566.4132159911586 ops/sec |
588.1531661342109 ops/sec |
0.96 |
linuxX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
258.1365425629106 ops/sec |
259.9879196093481 ops/sec |
0.99 |
linuxX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
256.26905654958244 ops/sec |
255.45221639373344 ops/sec |
1.00 |
linuxX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
891.9863656041161 ops/sec |
892.2849675464407 ops/sec |
1.00 |
linuxX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1186554.3960522749 ops/sec |
1199954.1195795832 ops/sec |
0.99 |
linuxX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
167.0979750759658 ops/sec |
177.95366701748884 ops/sec |
0.94 |
linuxX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
169.57470154964693 ops/sec |
180.69576999638036 ops/sec |
0.94 |
linuxX64.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
644.112655490751 ops/sec |
651.3011940200743 ops/sec |
0.99 |
linuxX64.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
623.2226478783192 ops/sec |
623.3692880507754 ops/sec |
1.00 |
macosArm64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1043.560785235602 us/op |
1325.9605106249999 us/op |
0.79 |
macosArm64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1136.9097669265034 us/op |
1317.886141011984 us/op |
0.86 |
macosArm64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
2544.8678464285717 us/op |
3004.6197703812313 us/op |
0.85 |
macosArm64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
2530.3239421465964 us/op |
2970.9433105421685 us/op |
0.85 |
macosArm64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
752.8182058130978 us/op |
841.7279331805682 us/op |
0.89 |
macosArm64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
0.5736408076258213 us/op |
0.634891532725674 us/op |
0.90 |
macosArm64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
3529.610580357142 us/op |
4609.096784474887 us/op |
0.77 |
macosArm64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
3669.9671195652177 us/op |
4869.894245714286 us/op |
0.75 |
macosArm64.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1001.5346264914053 us/op |
1141.4959121893492 us/op |
0.88 |
macosArm64.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1014.9402397979796 us/op |
1250.8724039340102 us/op |
0.81 |
macosArm64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
953.4353612172229 ops/sec |
794.8628645980135 ops/sec |
1.20 |
macosArm64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
882.1347019075071 ops/sec |
732.0228387143751 ops/sec |
1.21 |
macosArm64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
398.6530565353647 ops/sec |
336.54259684903155 ops/sec |
1.18 |
macosArm64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
396.3335416767539 ops/sec |
335.0481939684594 ops/sec |
1.18 |
macosArm64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1379.1583214000827 ops/sec |
1209.0794387942765 ops/sec |
1.14 |
macosArm64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1755772.763422075 ops/sec |
1574950.1674223018 ops/sec |
1.11 |
macosArm64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
260.1056756080779 ops/sec |
214.4373867323598 ops/sec |
1.21 |
macosArm64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
281.2879339712196 ops/sec |
224.9835488830835 ops/sec |
1.25 |
macosArm64.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
1009.0735939660741 ops/sec |
837.0111590713557 ops/sec |
1.21 |
macosArm64.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
982.3634025399876 ops/sec |
806.6204898450799 ops/sec |
1.22 |
macosX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
2797.157587811634 us/op |
2973.168883775811 us/op |
0.94 |
macosX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
3024.098404140127 us/op |
3350.707884539473 us/op |
0.90 |
macosX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
6108.410110967741 us/op |
6495.621247468354 us/op |
0.94 |
macosX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
6260.548285276074 us/op |
6899.0915493589755 us/op |
0.91 |
macosX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
2008.02796031746 us/op |
2364.0163889624723 us/op |
0.85 |
macosX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1.627560581158944 us/op |
1.8179743787817926 us/op |
0.90 |
macosX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
8944.784581249998 us/op |
9010.80259099099 us/op |
0.99 |
macosX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
8895.103347272725 us/op |
9141.43195625 us/op |
0.97 |
macosX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
2691.884683923706 us/op |
2933.309914457831 us/op |
0.92 |
macosX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
2707.4215760107822 us/op |
3001.459158064516 us/op |
0.90 |
macosX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
358.7136834475051 ops/sec |
349.530087997283 ops/sec |
1.03 |
macosX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
341.4295541681428 ops/sec |
336.15833230707284 ops/sec |
1.02 |
macosX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
165.00864938341365 ops/sec |
162.56661904420747 ops/sec |
1.02 |
macosX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
162.43808455208207 ops/sec |
159.57151156408517 ops/sec |
1.02 |
macosX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
503.499394434927 ops/sec |
498.588087641442 ops/sec |
1.01 |
macosX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
618247.4602462914 ops/sec |
607141.1649394365 ops/sec |
1.02 |
macosX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
109.07795051548935 ops/sec |
107.43695655337476 ops/sec |
1.02 |
macosX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
110.64269166508959 ops/sec |
109.7812831488466 ops/sec |
1.01 |
macosX64.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
371.69070605074586 ops/sec |
369.10990719511346 ops/sec |
1.01 |
macosX64.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
366.89214530813376 ops/sec |
357.5052004879461 ops/sec |
1.03 |
mingwX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
3442.059075342466 us/op |
3548.5640213523134 us/op |
0.97 |
mingwX64.CommonAvgTimeBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
3354.7761129568107 us/op |
3468.628424657535 us/op |
0.97 |
mingwX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
7464.100814814815 us/op |
7618.2869465648855 us/op |
0.98 |
mingwX64.CommonAvgTimeBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
7519.5988721804515 us/op |
7765.591627906976 us/op |
0.97 |
mingwX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
2468.520024752475 us/op |
2495.563190954774 us/op |
0.99 |
mingwX64.CommonAvgTimeBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
1.9743255153574548 us/op |
2.1892058433976804 us/op |
0.90 |
mingwX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
10690.470860215053 us/op |
10752.24311827957 us/op |
0.99 |
mingwX64.CommonAvgTimeBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
10197.322187499998 us/op |
10449.288315789474 us/op |
0.98 |
mingwX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
3418.417010309279 us/op |
3400.754206896552 us/op |
1.01 |
mingwX64.CommonAvgTimeBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
3398.268979591837 us/op |
3508.3286896551717 us/op |
0.97 |
mingwX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
287.17869098345744 ops/sec |
288.8282242281345 ops/sec |
0.99 |
mingwX64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
292.0078550746905 ops/sec |
289.1167014649221 ops/sec |
1.01 |
mingwX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
132.71539927511466 ops/sec |
131.07886556185593 ops/sec |
1.01 |
mingwX64.CommonThroughputBench.validateDetailed ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
130.59757838881276 ops/sec |
129.36341239458744 ops/sec |
1.01 |
mingwX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
401.39257805540035 ops/sec |
400.15282456266357 ops/sec |
1.00 |
mingwX64.CommonThroughputBench.validateFlag ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
501066.2608475623 ops/sec |
492992.2223834451 ops/sec |
1.02 |
mingwX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
94.01695879962955 ops/sec |
93.52287753321602 ops/sec |
1.01 |
mingwX64.CommonThroughputBench.validateVerbose ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
96.66830959607428 ops/sec |
95.94858778067656 ops/sec |
1.01 |
mingwX64.CommonThroughputBench.validate ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) |
291.7953278669171 ops/sec |
293.2529576487629 ops/sec |
1.00 |
mingwX64.CommonThroughputBench.validate ( {"objectPath":"openapi-invalid.json","schemaPath":"openapi_schema.json"} ) |
293.57382731130906 ops/sec |
290.0971237181231 ops/sec |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
CC: @OptimumCode
No description provided.