Configure self-hosted runner for benchmarks job #6975
Conversation
SiarheiFedartsou
force-pushed
the
sf-self-hosted-42
branch
from
7a95592
to
8f74005
Compare
SiarheiFedartsou
changed the title
| @@ -649,37 +649,17 @@ jobs: | |||
| GITHUB_REPOSITORY: ${{ github.repository }} | |||
| RUN_BIG_BENCHMARK: ${{ contains(github.event.pull_request.labels.*.name, 'Performance') }} | |||
| steps: | |||
| - name: Enable data.osm.pbf cache | |||
There was a problem hiding this comment.
It is not needed anymore - we don't clean cache on self-hosted runner, so job runs even faster without these caching actions...
|
This is cool. I have an ARM-based server instance that we could use for this, too. |
Let’s have it on this small Pi for the time being - it should be simpler as it is dedicated for only this task for now and we can completely isolate it from everything and we won’t be impacted by other things running on the same machine. It is not that fast though - current benchmarks require more than hour to finish(but eventually I hope we will optimize OSRM to make it much faster 😀) |
|
@DennisOSRM So don’t you mind merging it? 😀 |
|
I guess I was asking if it would make sense to give the server instance a try before we rely on self-hosted machine? That being said, 100% confidence in your sysop skills 😁 I can install the runner and configure it tomorrow. |
Let's try 😄 Do you need any guidance for runner configuration? It turned out to be super easy. But want to warn you: if you are running something sensitive on your machine in parallel I wouldn't do that... Just because in theory it means that you kind of open small vulnerability on your machine (it is indeed very small though if I understand correctly how it works...) |
|
I'd propose to eventually merge it for the time being. :) I want to create a couple of performance related PRs and would be really cool to have stable benchmarks in there. Later we can experiment with other things for sure. WDYT @DennisOSRM ? |
SiarheiFedartsou
force-pushed
the
sf-self-hosted-42
branch
from
bcc9e2c
to
6fbcb7c
Compare
|
I assume it is okay to merge it without explicit approval... Let me know in case of any concerns. It will be easy to revert in the case of something... |
Well, after a lot of attempts to make benchmarks stable when running GitHub-hosted runners I gave up and configured self-hosted runner which works on Raspberry Pi 5 right on my shelf:
It led to increased run time of benchmarks job, but results are much more stable: when running without code changes difference between master and PR is at glance <1% for most of benchmarks without jumps up to 10-20% as it happens with GitHub-hosted runner. When it comes to run time may be worth decreasing number of times we repeat benchmark: now it seems to be stable enough to run it say 3 times instead of current 5.
Important note: in general it is not recommended by GitHub to configure self-hosted runners for public repos due to security reasons. There is a risk that attacker can run malicious code on this runner simply creating PR from fork. I made some configuration changes to minimize security risks:
Benchmark Results
plain u32: 10975.8
aliased double: 15124.6
plain double: 15118.3
plain u32: 10921.3
aliased double: 14967
plain double: 14909.8
Total: 5097.95ms ± 4.80ms. Best: 5090.23ms
Min time: 3.06ms ± 0.06ms
Mean time: 38.92ms ± 0.03ms
Median time: 29.02ms ± 0.25ms
95th percentile: 133.90ms ± 0.38ms
99th percentile: 163.60ms ± 0.46ms
Max time: 173.97ms ± 0.97ms
Total: 5172.92ms ± 2.43ms. Best: 5167.75ms
Min time: 3.10ms ± 0.07ms
Mean time: 39.49ms ± 0.02ms
Median time: 29.28ms ± 0.13ms
95th percentile: 135.69ms ± 0.43ms
99th percentile: 166.30ms ± 0.33ms
Max time: 178.81ms ± 0.47ms
Total: 3014.91ms ± 1.89ms. Best: 3011.57ms
Min time: 2.54ms ± 0.04ms
Mean time: 23.01ms ± 0.02ms
Median time: 12.08ms ± 0.08ms
95th percentile: 75.56ms ± 0.15ms
99th percentile: 87.62ms ± 0.18ms
Max time: 100.58ms ± 0.27ms
Total: 3001.39ms ± 1.89ms. Best: 2997.53ms
Min time: 2.60ms ± 0.03ms
Mean time: 22.91ms ± 0.01ms
Median time: 12.04ms ± 0.10ms
95th percentile: 75.42ms ± 0.09ms
99th percentile: 87.15ms ± 0.26ms
Max time: 100.45ms ± 0.18ms
Total: 1561.91ms ± 9.10ms. Best: 1549.91ms
Min time: 1.27ms ± 0.01ms
Mean time: 1.56ms ± 0.01ms
Median time: 1.52ms ± 0.01ms
95th percentile: 1.95ms ± 0.01ms
99th percentile: 2.04ms ± 0.01ms
Max time: 9.30ms ± 7.40ms
Total: 1560.16ms ± 11.31ms. Best: 1539.46ms
Min time: 1.28ms ± 0.01ms
Mean time: 1.56ms ± 0.01ms
Median time: 1.52ms ± 0.01ms
95th percentile: 1.95ms ± 0.01ms
99th percentile: 2.03ms ± 0.01ms
Max time: 9.30ms ± 7.43ms
Total: 1572.73ms ± 12.87ms. Best: 1550.55ms
Min time: 1.29ms ± 0.01ms
Mean time: 1.57ms ± 0.01ms
Median time: 1.53ms ± 0.01ms
95th percentile: 1.96ms ± 0.01ms
99th percentile: 2.04ms ± 0.01ms
Max time: 9.35ms ± 7.41ms
Total: 1564.82ms ± 5.38ms. Best: 1556.50ms
Min time: 1.28ms ± 0.01ms
Mean time: 1.56ms ± 0.01ms
Median time: 1.52ms ± 0.01ms
95th percentile: 1.95ms ± 0.01ms
99th percentile: 2.04ms ± 0.01ms
Max time: 9.34ms ± 7.43ms
Total: 4592.01ms ± 11.41ms. Best: 4573.41ms
Min time: 1.81ms ± 0.04ms
Mean time: 4.59ms ± 0.01ms
Median time: 4.69ms ± 0.02ms
95th percentile: 6.05ms ± 0.01ms
99th percentile: 6.56ms ± 0.03ms
Max time: 13.33ms ± 6.36ms
Total: 4589.43ms ± 8.67ms. Best: 4574.23ms
Min time: 1.84ms ± 0.04ms
Mean time: 4.59ms ± 0.01ms
Median time: 4.67ms ± 0.01ms
95th percentile: 6.06ms ± 0.02ms
99th percentile: 6.58ms ± 0.04ms
Max time: 13.31ms ± 6.34ms
Total: 5578.59ms ± 11.68ms. Best: 5555.01ms
Min time: 1.83ms ± 0.05ms
Mean time: 5.58ms ± 0.01ms
Median time: 5.69ms ± 0.02ms
95th percentile: 7.61ms ± 0.02ms
99th percentile: 8.16ms ± 0.08ms
Max time: 14.76ms ± 5.95ms
Total: 5571.11ms ± 11.55ms. Best: 5548.72ms
Min time: 1.84ms ± 0.04ms
Mean time: 5.57ms ± 0.01ms
Median time: 5.69ms ± 0.02ms
95th percentile: 7.60ms ± 0.02ms
99th percentile: 8.16ms ± 0.10ms
Max time: 14.65ms ± 5.91ms
Total: 4623.34ms ± 9.48ms. Best: 4603.44ms
Min time: 2.53ms ± 0.05ms
Mean time: 4.62ms ± 0.01ms
Median time: 4.62ms ± 0.01ms
95th percentile: 6.27ms ± 0.01ms
99th percentile: 6.60ms ± 0.02ms
Max time: 13.71ms ± 7.02ms
Total: 4617.48ms ± 12.01ms. Best: 4598.98ms
Min time: 2.51ms ± 0.03ms
Mean time: 4.62ms ± 0.01ms
Median time: 4.61ms ± 0.01ms
95th percentile: 6.27ms ± 0.02ms
99th percentile: 6.58ms ± 0.03ms
Max time: 13.70ms ± 7.03ms
Total: 14182.95ms ± 7.53ms. Best: 14170.21ms
Min time: 5.94ms ± 0.04ms
Mean time: 14.18ms ± 0.01ms
Median time: 14.12ms ± 0.02ms
95th percentile: 21.59ms ± 0.04ms
99th percentile: 22.84ms ± 0.08ms
Max time: 29.17ms ± 5.71ms
Total: 14152.66ms ± 5.98ms. Best: 14143.36ms
Min time: 5.93ms ± 0.01ms
Mean time: 14.15ms ± 0.01ms
Median time: 14.12ms ± 0.04ms
95th percentile: 21.58ms ± 0.04ms
99th percentile: 22.72ms ± 0.02ms
Max time: 29.19ms ± 5.68ms
Total: 15894.23ms ± 9.98ms. Best: 15882.25ms
Min time: 2.40ms ± 0.21ms
Mean time: 15.89ms ± 0.01ms
Median time: 15.11ms ± 0.02ms
95th percentile: 28.06ms ± 0.02ms
99th percentile: 30.29ms ± 0.14ms
Max time: 32.51ms ± 1.35ms
Total: 15892.10ms ± 9.86ms. Best: 15871.82ms
Min time: 2.35ms ± 0.17ms
Mean time: 15.89ms ± 0.01ms
Median time: 15.13ms ± 0.03ms
95th percentile: 28.06ms ± 0.04ms
99th percentile: 30.17ms ± 0.11ms
Max time: 32.42ms ± 1.37ms
Total: 27016.58ms ± 10.78ms. Best: 27000.38ms
Min time: 2.38ms ± 0.13ms
Mean time: 27.02ms ± 0.01ms
Median time: 26.17ms ± 0.10ms
95th percentile: 43.99ms ± 0.04ms
99th percentile: 46.70ms ± 0.09ms
Max time: 49.09ms ± 0.15ms
Total: 26937.55ms ± 13.13ms. Best: 26920.20ms
Min time: 2.38ms ± 0.21ms
Mean time: 26.94ms ± 0.01ms
Median time: 26.11ms ± 0.06ms
95th percentile: 43.93ms ± 0.07ms
99th percentile: 46.47ms ± 0.10ms
Max time: 48.78ms ± 0.08ms
Stringstream: 14.1385ms
Vector: 9.52853ms
Stringstream: 14.4626ms
Vector: 9.41143ms
7.04594ms/req at 82 coordinate
0.0859261ms/coordinate
Radius 10m:
24.9416ms/req at 82 coordinate
0.304166ms/coordinate
7.07911ms/req at 82 coordinate
0.0863306ms/coordinate
Radius 10m:
25.06ms/req at 82 coordinate
0.30561ms/coordinate
4.3348ms/req at 82 coordinate
0.0528634ms/coordinate
Radius 10m:
16.1786ms/req at 82 coordinate
0.197301ms/coordinate
4.3798ms/req at 82 coordinate
0.0534122ms/coordinate
Radius 10m:
16.2303ms/req at 82 coordinate
0.197931ms/coordinate
std::vector 185206 ms
util::packed_vector 373497 ms
slowdown: 2.01666
random read:
std::vector 100579 ms
util::packed_vector 190735 ms
slowdown: 1.89638
std::vector 180202 ms
util::packed_vector 377634 ms
slowdown: 2.09561
random read:
std::vector 99301.9 ms
util::packed_vector 192446 ms
slowdown: 1.93799
ops: 120.33 ± 0.17 ops/s. best: 120.66ops/s.
total: 473.71 ± 0.68ms. best: 472.40ms.
avg: 8.31 ± 0.01ms
min: 0.25 ± 0.00ms
max: 40.76 ± 0.05ms
p99: 40.76 ± 0.05ms
500 matches, radius=10
ops: 34.59 ± 0.02 ops/s. best: 34.62ops/s.
total: 1850.46 ± 1.29ms. best: 1848.50ms.
avg: 28.91 ± 0.02ms
min: 0.24 ± 0.00ms
max: 410.13 ± 0.76ms
p99: 410.13 ± 0.76ms
500 matches, radius=20
ops: 8.17 ± 0.01 ops/s. best: 8.18ops/s.
total: 7956.25 ± 10.61ms. best: 7942.63ms.
avg: 122.40 ± 0.16ms
min: 0.49 ± 0.00ms
max: 2177.72 ± 7.13ms
p99: 2177.72 ± 7.13ms
Peak RAM: 56.000MB
ops: 119.36 ± 0.16 ops/s. best: 119.67ops/s.
total: 477.56 ± 0.65ms. best: 476.33ms.
avg: 8.38 ± 0.01ms
min: 0.25 ± 0.00ms
max: 41.22 ± 0.05ms
p99: 41.22 ± 0.05ms
500 matches, radius=10
ops: 34.18 ± 0.03 ops/s. best: 34.21ops/s.
total: 1872.57 ± 1.40ms. best: 1870.79ms.
avg: 29.26 ± 0.02ms
min: 0.24 ± 0.00ms
max: 416.68 ± 0.74ms
p99: 416.68 ± 0.74ms
500 matches, radius=20
ops: 8.05 ± 0.01 ops/s. best: 8.06ops/s.
total: 8076.21 ± 9.98ms. best: 8064.81ms.
avg: 124.25 ± 0.15ms
min: 0.50 ± 0.01ms
max: 2228.61 ± 7.23ms
p99: 2228.61 ± 7.23ms
Peak RAM: 56.000MB
ops: 204.73 ± 0.38 ops/s. best: 205.13ops/s.
total: 278.41 ± 0.51ms. best: 277.87ms.
avg: 4.88 ± 0.01ms
min: 0.21 ± 0.00ms
max: 27.18 ± 0.04ms
p99: 27.18 ± 0.04ms
500 matches, radius=10
ops: 72.96 ± 0.06 ops/s. best: 73.07ops/s.
total: 877.18 ± 0.66ms. best: 875.90ms.
avg: 13.71 ± 0.01ms
min: 0.22 ± 0.00ms
max: 162.66 ± 0.20ms
p99: 162.66 ± 0.20ms
500 matches, radius=20
ops: 15.32 ± 0.01 ops/s. best: 15.34ops/s.
total: 4242.81 ± 4.02ms. best: 4237.45ms.
avg: 65.27 ± 0.06ms
min: 0.29 ± 0.00ms
max: 848.61 ± 1.35ms
p99: 848.61 ± 1.35ms
Peak RAM: 51.500MB
ops: 205.18 ± 0.32 ops/s. best: 205.53ops/s.
total: 277.80 ± 0.42ms. best: 277.33ms.
avg: 4.87 ± 0.01ms
min: 0.21 ± 0.00ms
max: 27.10 ± 0.05ms
p99: 27.10 ± 0.05ms
500 matches, radius=10
ops: 73.15 ± 0.05 ops/s. best: 73.23ops/s.
total: 874.97 ± 0.65ms. best: 874.00ms.
avg: 13.67 ± 0.01ms
min: 0.22 ± 0.00ms
max: 162.30 ± 0.18ms
p99: 162.30 ± 0.18ms
500 matches, radius=20
ops: 15.38 ± 0.01 ops/s. best: 15.40ops/s.
total: 4226.39 ± 3.48ms. best: 4221.06ms.
avg: 65.02 ± 0.05ms
min: 0.29 ± 0.00ms
max: 845.94 ± 1.21ms
p99: 845.94 ± 1.21ms
Peak RAM: 51.500MB
ops: 21644.84 ± 48.40 ops/s. best: 21689.33ops/s.
total: 462.01 ± 1.04ms. best: 461.06ms.
avg: 0.05 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.15 ± 0.03ms
p99: 0.10 ± 0.00ms
10000 nearest, number_of_results=5
ops: 16023.30 ± 7.83 ops/s. best: 16031.10ops/s.
total: 624.09 ± 0.32ms. best: 623.79ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.16 ± 0.00ms
p99: 0.12 ± 0.00ms
10000 nearest, number_of_results=10
ops: 12280.18 ± 10.05 ops/s. best: 12288.43ops/s.
total: 814.32 ± 0.67ms. best: 813.77ms.
avg: 0.08 ± 0.00ms
min: 0.04 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.14 ± 0.00ms
Peak RAM: 35.000MB
ops: 21771.24 ± 38.76 ops/s. best: 21802.94ops/s.
total: 459.32 ± 0.82ms. best: 458.65ms.
avg: 0.05 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.15 ± 0.03ms
p99: 0.10 ± 0.00ms
10000 nearest, number_of_results=5
ops: 15973.04 ± 18.35 ops/s. best: 15990.34ops/s.
total: 626.06 ± 0.72ms. best: 625.38ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.16 ± 0.00ms
p99: 0.12 ± 0.00ms
10000 nearest, number_of_results=10
ops: 12197.35 ± 4.17 ops/s. best: 12201.73ops/s.
total: 819.85 ± 0.28ms. best: 819.56ms.
avg: 0.08 ± 0.00ms
min: 0.04 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.14 ± 0.00ms
Peak RAM: 35.000MB
ops: 21659.20 ± 40.41 ops/s. best: 21694.20ops/s.
total: 461.70 ± 0.86ms. best: 460.95ms.
avg: 0.05 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.15 ± 0.02ms
p99: 0.10 ± 0.00ms
10000 nearest, number_of_results=5
ops: 16026.48 ± 7.50 ops/s. best: 16036.94ops/s.
total: 623.97 ± 0.29ms. best: 623.56ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.16 ± 0.00ms
p99: 0.12 ± 0.00ms
10000 nearest, number_of_results=10
ops: 12289.95 ± 3.35 ops/s. best: 12293.74ops/s.
total: 813.67 ± 0.23ms. best: 813.42ms.
avg: 0.08 ± 0.00ms
min: 0.04 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.14 ± 0.00ms
Peak RAM: 35.000MB
ops: 21768.46 ± 32.44 ops/s. best: 21797.96ops/s.
total: 459.38 ± 0.69ms. best: 458.76ms.
avg: 0.05 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.14 ± 0.01ms
p99: 0.10 ± 0.00ms
10000 nearest, number_of_results=5
ops: 15977.18 ± 6.38 ops/s. best: 15986.80ops/s.
total: 625.89 ± 0.25ms. best: 625.52ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.16 ± 0.00ms
p99: 0.12 ± 0.00ms
10000 nearest, number_of_results=10
ops: 12194.27 ± 4.47 ops/s. best: 12199.51ops/s.
total: 820.06 ± 0.30ms. best: 819.71ms.
avg: 0.08 ± 0.00ms
min: 0.04 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.14 ± 0.00ms
Peak RAM: 35.000MB
ops: 286.78 ± 0.28 ops/s. best: 287.16ops/s.
total: 3431.23 ± 3.32ms. best: 3426.64ms.
avg: 3.49 ± 0.00ms
min: 0.44 ± 0.00ms
max: 5.96 ± 0.01ms
p99: 5.20 ± 0.01ms
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 336.11 ± 0.05 ops/s. best: 336.18ops/s.
total: 2975.26 ± 0.44ms. best: 2974.57ms.
avg: 2.98 ± 0.00ms
min: 0.08 ± 0.00ms
max: 7.13 ± 0.01ms
p99: 6.73 ± 0.01ms
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 591.78 ± 0.18 ops/s. best: 592.03ops/s.
total: 1662.79 ± 0.50ms. best: 1662.07ms.
avg: 1.69 ± 0.00ms
min: 0.35 ± 0.00ms
max: 2.54 ± 0.01ms
p99: 2.39 ± 0.00ms
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 653.45 ± 0.12 ops/s. best: 653.61ops/s.
total: 1530.33 ± 0.27ms. best: 1529.96ms.
avg: 1.53 ± 0.00ms
min: 0.06 ± 0.00ms
max: 4.53 ± 0.01ms
p99: 3.91 ± 0.01ms
Peak RAM: 84.000MB
ops: 285.44 ± 0.13 ops/s. best: 285.53ops/s.
total: 3447.35 ± 1.51ms. best: 3446.23ms.
avg: 3.50 ± 0.00ms
min: 0.43 ± 0.00ms
max: 5.97 ± 0.01ms
p99: 5.19 ± 0.02ms
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 336.96 ± 0.07 ops/s. best: 337.06ops/s.
total: 2967.75 ± 0.64ms. best: 2966.81ms.
avg: 2.97 ± 0.00ms
min: 0.08 ± 0.00ms
max: 7.12 ± 0.01ms
p99: 6.72 ± 0.01ms
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 584.31 ± 0.25 ops/s. best: 584.71ops/s.
total: 1684.05 ± 0.73ms. best: 1682.87ms.
avg: 1.71 ± 0.00ms
min: 0.36 ± 0.01ms
max: 2.55 ± 0.01ms
p99: 2.41 ± 0.01ms
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 651.83 ± 0.58 ops/s. best: 652.84ops/s.
total: 1534.15 ± 1.38ms. best: 1531.76ms.
avg: 1.53 ± 0.00ms
min: 0.06 ± 0.00ms
max: 4.53 ± 0.00ms
p99: 3.92 ± 0.00ms
Peak RAM: 84.000MB
ops: 147.56 ± 0.02 ops/s. best: 147.60ops/s.
total: 6668.35 ± 0.99ms. best: 6666.73ms.
avg: 6.78 ± 0.00ms
min: 0.42 ± 0.00ms
max: 14.14 ± 0.02ms
p99: 10.98 ± 0.01ms
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 141.41 ± 0.04 ops/s. best: 141.45ops/s.
total: 7071.59 ± 2.01ms. best: 7069.71ms.
avg: 7.07 ± 0.00ms
min: 0.07 ± 0.00ms
max: 16.42 ± 0.01ms
p99: 15.00 ± 0.04ms
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 203.74 ± 0.29 ops/s. best: 204.18ops/s.
total: 4829.74 ± 6.76ms. best: 4819.17ms.
avg: 4.91 ± 0.01ms
min: 0.36 ± 0.00ms
max: 11.50 ± 0.02ms
p99: 8.42 ± 0.04ms
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 177.29 ± 0.04 ops/s. best: 177.35ops/s.
total: 5640.58 ± 1.41ms. best: 5638.47ms.
avg: 5.64 ± 0.00ms
min: 0.05 ± 0.00ms
max: 12.75 ± 0.03ms
p99: 11.63 ± 0.16ms
Peak RAM: 73.484MB
ops: 147.79 ± 0.02 ops/s. best: 147.83ops/s.
total: 6657.89 ± 0.95ms. best: 6656.43ms.
avg: 6.77 ± 0.00ms
min: 0.42 ± 0.01ms
max: 14.19 ± 0.02ms
p99: 10.97 ± 0.01ms
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 141.64 ± 0.02 ops/s. best: 141.66ops/s.
total: 7060.39 ± 0.81ms. best: 7059.28ms.
avg: 7.06 ± 0.00ms
min: 0.07 ± 0.00ms
max: 16.42 ± 0.03ms
p99: 15.02 ± 0.03ms
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 205.43 ± 0.07 ops/s. best: 205.49ops/s.
total: 4789.97 ± 1.68ms. best: 4788.57ms.
avg: 4.87 ± 0.00ms
min: 0.36 ± 0.00ms
max: 11.51 ± 0.02ms
p99: 8.37 ± 0.01ms
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 177.59 ± 0.02 ops/s. best: 177.61ops/s.
total: 5631.04 ± 0.53ms. best: 5630.36ms.
avg: 5.63 ± 0.00ms
min: 0.05 ± 0.00ms
max: 12.75 ± 0.05ms
p99: 11.52 ± 0.01ms
Peak RAM: 73.484MB
ops: 1129.94 ± 4.76 ops/s. best: 1133.66ops/s.
total: 221.26 ± 0.94ms. best: 220.52ms.
avg: 0.89 ± 0.00ms
min: 0.69 ± 0.00ms
max: 1.21 ± 0.14ms
p99: 1.08 ± 0.02ms
250 tables, 25 coordinates
ops: 125.34 ± 0.04 ops/s. best: 125.40ops/s.
total: 1994.55 ± 0.60ms. best: 1993.63ms.
avg: 7.98 ± 0.00ms
min: 7.27 ± 0.00ms
max: 8.63 ± 0.02ms
p99: 8.54 ± 0.00ms
250 tables, 50 coordinates
ops: 61.16 ± 0.01 ops/s. best: 61.17ops/s.
total: 4087.91 ± 0.56ms. best: 4087.22ms.
avg: 16.35 ± 0.00ms
min: 15.37 ± 0.01ms
max: 17.46 ± 0.02ms
p99: 17.22 ± 0.01ms
Peak RAM: 63.500MB
ops: 1130.98 ± 4.79 ops/s. best: 1134.73ops/s.
total: 221.05 ± 0.95ms. best: 220.32ms.
avg: 0.88 ± 0.00ms
min: 0.69 ± 0.00ms
max: 1.21 ± 0.14ms
p99: 1.09 ± 0.02ms
250 tables, 25 coordinates
ops: 126.08 ± 0.02 ops/s. best: 126.12ops/s.
total: 1982.83 ± 0.32ms. best: 1982.22ms.
avg: 7.93 ± 0.00ms
min: 7.22 ± 0.01ms
max: 8.56 ± 0.00ms
p99: 8.49 ± 0.01ms
250 tables, 50 coordinates
ops: 61.46 ± 0.00 ops/s. best: 61.47ops/s.
total: 4067.37 ± 0.29ms. best: 4067.08ms.
avg: 16.27 ± 0.00ms
min: 15.29 ± 0.01ms
max: 17.38 ± 0.01ms
p99: 17.14 ± 0.01ms
Peak RAM: 63.000MB
ops: 228.86 ± 0.24 ops/s. best: 229.22ops/s.
total: 1092.39 ± 1.16ms. best: 1090.66ms.
avg: 4.37 ± 0.00ms
min: 3.45 ± 0.00ms
max: 5.64 ± 0.01ms
p99: 5.43 ± 0.02ms
250 tables, 25 coordinates
ops: 24.19 ± 0.02 ops/s. best: 24.22ops/s.
total: 10332.89 ± 7.15ms. best: 10320.72ms.
avg: 41.33 ± 0.03ms
min: 38.13 ± 0.13ms
max: 45.25 ± 0.06ms
p99: 44.67 ± 0.20ms
250 tables, 50 coordinates
ops: 11.21 ± 0.01 ops/s. best: 11.23ops/s.
total: 22294.08 ± 27.15ms. best: 22256.76ms.
avg: 89.18 ± 0.11ms
min: 85.19 ± 0.21ms
max: 94.36 ± 0.14ms
p99: 93.14 ± 0.22ms
Peak RAM: 62.734MB
ops: 230.15 ± 0.20 ops/s. best: 230.41ops/s.
total: 1086.23 ± 0.93ms. best: 1085.01ms.
avg: 4.34 ± 0.00ms
min: 3.45 ± 0.00ms
max: 5.62 ± 0.01ms
p99: 5.41 ± 0.02ms
250 tables, 25 coordinates
ops: 24.17 ± 0.01 ops/s. best: 24.19ops/s.
total: 10344.61 ± 5.30ms. best: 10336.17ms.
avg: 41.38 ± 0.02ms
min: 38.31 ± 0.06ms
max: 45.33 ± 0.29ms
p99: 44.77 ± 0.01ms
250 tables, 50 coordinates
ops: 11.19 ± 0.00 ops/s. best: 11.20ops/s.
total: 22344.34 ± 6.47ms. best: 22330.88ms.
avg: 89.38 ± 0.03ms
min: 85.34 ± 0.04ms
max: 94.85 ± 0.10ms
p99: 93.50 ± 0.09ms
Peak RAM: 62.734MB
ops: 326.02 ± 0.43 ops/s. best: 326.41ops/s.
total: 766.82 ± 1.01ms. best: 765.91ms.
avg: 3.07 ± 0.00ms
min: 1.60 ± 0.01ms
max: 4.43 ± 0.00ms
p99: 4.23 ± 0.01ms
250 trips, 5 coordinates
ops: 210.23 ± 0.03 ops/s. best: 210.28ops/s.
total: 1189.20 ± 0.17ms. best: 1188.87ms.
avg: 4.76 ± 0.00ms
min: 2.92 ± 0.01ms
max: 5.99 ± 0.01ms
p99: 5.96 ± 0.01ms
Peak RAM: 73.500MB
ops: 326.08 ± 0.45 ops/s. best: 326.40ops/s.
total: 766.70 ± 1.07ms. best: 765.94ms.
avg: 3.07 ± 0.00ms
min: 1.60 ± 0.01ms
max: 4.43 ± 0.01ms
p99: 4.23 ± 0.00ms
250 trips, 5 coordinates
ops: 210.35 ± 0.04 ops/s. best: 210.41ops/s.
total: 1188.48 ± 0.24ms. best: 1188.15ms.
avg: 4.75 ± 0.00ms
min: 2.93 ± 0.01ms
max: 6.00 ± 0.02ms
p99: 5.95 ± 0.01ms
Peak RAM: 73.500MB
ops: 106.34 ± 0.06 ops/s. best: 106.42ops/s.
total: 2350.97 ± 1.36ms. best: 2349.29ms.
avg: 9.40 ± 0.01ms
min: 5.44 ± 0.01ms
max: 12.32 ± 0.01ms
p99: 12.19 ± 0.01ms
250 trips, 5 coordinates
ops: 68.54 ± 0.05 ops/s. best: 68.66ops/s.
total: 3647.27 ± 2.43ms. best: 3640.95ms.
avg: 14.59 ± 0.01ms
min: 10.05 ± 0.01ms
max: 17.73 ± 0.04ms
p99: 17.47 ± 0.03ms
Peak RAM: 69.000MB
ops: 107.85 ± 0.08 ops/s. best: 107.91ops/s.
total: 2318.14 ± 1.70ms. best: 2316.69ms.
avg: 9.27 ± 0.01ms
min: 5.44 ± 0.01ms
max: 12.09 ± 0.03ms
p99: 11.97 ± 0.01ms
250 trips, 5 coordinates
ops: 69.49 ± 0.01 ops/s. best: 69.50ops/s.
total: 3597.85 ± 0.42ms. best: 3596.97ms.
avg: 14.39 ± 0.00ms
min: 10.01 ± 0.01ms
max: 17.60 ± 0.04ms
p99: 17.19 ± 0.01ms
Peak RAM: 69.000MB
638.492ms
0.638492ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
782.914ms
0.782914ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
246.145ms
0.246145ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
216.298ms
0.216298ms/req
634.704ms
0.634704ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
777.698ms
0.777698ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
246.812ms
0.246812ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
217.509ms
0.217509ms/req
808.868ms
0.808868ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
1040.59ms
1.04059ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
414.111ms
0.414111ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
461.104ms
0.461104ms/req
804.942ms
0.804942ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
1036.58ms
1.03658ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
412.784ms
0.412784ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
460.433ms
0.460433ms/req
227.443ms -> 0.0227443 ms/query
10 results:
258.195ms -> 0.0258195 ms/query
227.144ms -> 0.0227144 ms/query
10 results:
257.936ms -> 0.0257936 ms/query