Skip to content

Commit

Permalink
Improve flamegraph profiling instructions (#12521)
Browse files Browse the repository at this point in the history
* Improve flamegraph profiling instructions

* Update docs/source/library-user-guide/profiling.md

Co-authored-by: kamille <3144148605@qq.com>

---------

Co-authored-by: kamille <3144148605@qq.com>
alamb and Rachelint authored Sep 18, 2024

Unverified

No user is associated with the committer email.
1 parent 5abef41 commit 21e35ee
Showing 2 changed files with 517 additions and 16 deletions.
491 changes: 491 additions & 0 deletions docs/source/_static/images/flamegraph.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 26 additions & 16 deletions docs/source/library-user-guide/profiling.md
Original file line number Diff line number Diff line change
@@ -25,34 +25,44 @@ The section contains examples how to perform CPU profiling for Apache DataFusion

### Building a flamegraph

- [cargo-flamegraph](https://github.com/flamegraph-rs/flamegraph)
[Video: how to CPU profile DataFusion with a Flamegraph](https://youtu.be/2z11xtYw_xs)

Test:
A flamegraph is a visual representation of which functions are being run
You can create flamegraphs in many ways; The instructions below are for
[cargo-flamegraph](https://github.com/flamegraph-rs/flamegraph) which results
in images such as this:

```bash
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --unit-test datafusion -- dataframe::tests::test_array_agg
![Flamegraph](../_static/images/flamegraph.svg)

To create a flamegraph, you need to install the `flamegraph` tool:

```shell
cargo install flamegraph
```

Benchmark:
Then you can run the flamegraph tool with the `--` separator to pass arguments
to the binary you want to profile.

```bash
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --bench sql_planner -- --bench
Example: Flamegraph for `datafusion-cli` executing `q28.sql`. Note this
must be run as root on Mac OSx to access DTrace.

```shell
sudo flamegraph -- datafusion-cli -f q28.sq
```

Open `flamegraph.svg` file with the browser
You can also invoke the flamegraph tool with `cargo` to profile a specific test or benchmark.

- dtrace with DataFusion CLI
Example: Flamegraph for a specific test:

```bash
git clone https://github.com/brendangregg/FlameGraph.git /tmp/fg
cd datafusion-cli
CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release
echo "select * from table;" >> test.sql
sudo dtrace -c './target/debug/datafusion-cli -f test.sql' -o out.stacks -n 'profile-997 /execname == "datafusion-cli"/ { @[ustack(100)] = count(); }'
/tmp/fg/FlameGraph/stackcollapse.pl out.stacks | /tmp/fg/FlameGraph/flamegraph.pl > flamegraph.svg
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --unit-test datafusion -- dataframe::tests::test_array_agg
```

Open `flamegraph.svg` file with the browser
Example: Flamegraph for a benchmark

```bash
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --bench sql_planner -- --bench
```

### CPU profiling with XCode Instruments

0 comments on commit 21e35ee

Please sign in to comment.