Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Meaning of p0...p100 undocumented #306

Closed
dabrahams opened this issue Jan 29, 2025 · 3 comments
Closed

Meaning of p0...p100 undocumented #306

dabrahams opened this issue Jan 29, 2025 · 3 comments
Assignees
@dabrahams

Comments

@dabrahams
Copy link

The README doesn't tell me how to interpret these columns.
@hassila
Copy link
Contributor

hassila commented Jan 31, 2025

@dabrahams please check out the documentation here:
https://swiftpackageindex.com/ordo-one/package-benchmark/1.27.4/documentation/benchmark

Specifically this page:

https://swiftpackageindex.com/ordo-one/package-benchmark/1.27.4/documentation/benchmark/aboutpercentiles

@dabrahams
Copy link
Author

Thank you; I see the docs are referenced from the README but maybe it would be helpful to drop that link in next to the example outputs.

@hassila hassila mentioned this issue Feb 11, 2025
Merged
3 tasks
hassila added a commit that referenced this issue Feb 11, 2025
@hassila @axelandersson @dimlio
feat(minor): Enable Swift 6 mode and support specifying units per met…
2bb8a39 
…ric (#309)

## Description

Swift 6 benchmark targets should now be possible with:

```swift
let benchmarks: @sendable () -> Void = {
...
}
```

Make it possible to specify output units for the text output for a given
metric by specifying it in the configuration.

E.g.
```swift
    Benchmark.defaultConfiguration.units = [.peakMemoryResident: .mega, .peakMemoryVirtual: .giga]
```

Also add the ability to override the time units from the command line
using `--time-units`.

E.g.
```bash
swift package benchmark --time-units microseconds
```

This update also displays overflowing numeric values in scientific
notation in the text output:
```
╒═══════════════════════════════════════════════════════════════════════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╕
│ Test                                                                      │        p0 │       p25 │       p50 │       p75 │       p90 │       p99 │      p100 │   Samples │
╞═══════════════════════════════════════════════════════════════════════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╡
│ Samples:All metrics, full concurrency, async (ns) *                       │  8.08e+07 │  8.34e+07 │  8.38e+07 │  8.40e+07 │  8.41e+07 │  8.46e+07 │  8.49e+07 │       183 │
├───────────────────────────────────────────────────────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Samples:Counter, custom metric thresholds (ns) *                          │      2375 │      2417 │      2417 │      2459 │      2501 │      2709 │      4334 │      4753 │
├───────────────────────────────────────────────────────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Samples:Extended + custom metrics (ns) *                                  │      1875 │      2000 │      2000 │      2041 │      2167 │      2667 │      8750 │       707 │
├───────────────────────────────────────────────────────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Samples:Extended metrics (ns) *                                           │      1750 │      1875 │      1875 │      1917 │      1958 │      2209 │      3250 │       705 │
╘═══════════════════════════════════════════════════════════════════════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╛
```

Also updates documentation.

Addresses:
#306 
and
#237
#293
#277
#258

---------

Co-authored-by: Axel Andersson <axel@ordo.one>
Co-authored-by: dimlio <122263440+dimlio@users.noreply.github.com>
@hassila
Copy link
Contributor

hassila commented Feb 11, 2025

Documentation updated in #309

@hassila hassila closed this as completed Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dabrahams dabrahams

Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dabrahams @hassila