Add console-subscriber

[adam-singer]

Description

console-subscriber provides an instrumentation layer for async tokio applications. console provides viewing tasks, resources and histogram information.

Tokio instrumentation is in unstable phase and requires additional build/rust flags to be enabled. These configuration are not enabled unless explicitly opted into, thus disabled by default in production/dev. Use the tokio_unstable cfg and enable_tokio_console features flags to enabled.

Example

Run native link with basic cas config.

cargo --config 'build.rustflags=["--cfg", "tokio_unstable"]' run --features enable_tokio_console -- nativelink-config/examples/basic_cas.json

New terminal (same host) to launch the tokio console application. See tokio-console for details on how to run.

git clone https://github.com/tokio-rs/console.git
cd console/tokio-console
cargo run

NOTE: Skipping documentation on instrumentation util proper place is found and other tips/tooling available, such as spans and profiling.

Fixes # (issue)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to
  not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please also list any relevant details for your test configuration

Checklist

• Updated documentation if needed
• Tests added/amended
• bazel test //... passes locally
• PR is contained in a single commit, using git amend see some docs

---

This change is 

[allada]

Reviewable status: 0 of 1 LGTMs obtained, and pending CI: Bazel Dev / large-ubuntu-22.04, Local / large-ubuntu-22.04, Remote / large-ubuntu-22.04, asan / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), large-ubuntu-20.04, large-ubuntu-22.04

a discussion (no related file):
I have played with this too and think we can enable this using a feature. We can just use #[cfg(feature = enable_tokio_console]) or something on the few additional lines of code.

---

[adam-singer]

Reviewable status: 0 of 1 LGTMs obtained

a discussion (no related file):

Previously, allada (Nathan (Blaise) Bruer) wrote…

I have played with this too and think we can enable this using a feature. We can just use #[cfg(feature = enable_tokio_console]) or something on the few additional lines of code.

Added feature flag. Updated the github description with an example but that isn't being reflective in reviewable for some reason.

Done.

---

[aaronmondal]

Reviewed 4 of 4 files at r2, 1 of 1 files at r3, all commit messages.
Reviewable status: 0 of 1 LGTMs obtained

---

-- commits line 5 at r3:
nit: Maybe this is a reviewable bug but this looks like it's a second commit.

---

Cargo.toml line 50 at r2 (raw file):

tracing = "0.1.40"
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
console-subscriber = { version = "0.2.0" }

nit: The other imports are sorted alphabetically

---

src/bin/cas.rs line 673 at r2 (raw file):

                    .with_thread_names(true)
                    .with_filter(env_filter),
            )

nit: This looks like it could be deduplicated with a let fmt_layer = tracing_subscriber::fmt::layer()....with_env_filter(env_filter).

[aaronmondal]

Reviewable status: 1 of 1 LGTMs obtained

[aaronmondal]

Reviewable status: 1 of 1 LGTMs obtained

[allada]

Reviewable status: 1 of 1 LGTMs obtained

[adam-singer]

Reviewable status: 1 of 1 LGTMs obtained

---

-- commits line 5 at r3:

Previously, aaronmondal (Aaron Siddhartha Mondal) wrote…

nit: Maybe this is a reviewable bug but this looks like it's a second commit.

I foobar'd something, not a reviewable bug

---

src/bin/cas.rs line 673 at r2 (raw file):

Previously, aaronmondal (Aaron Siddhartha Mondal) wrote…

nit: This looks like it could be deduplicated with a let fmt_layer = tracing_subscriber::fmt::layer()....with_env_filter(env_filter).

I spent a lot of time trying to wedge that in, for some reason tracing_subscriber::fmt() builder does not seem to take a Layer or Layered (forget which one is produced). Instead I figured we might revisit this code if/when we add span tracing or other instrumentation features from tokio. Also gives me personally a bit more time to dig into their docs, its fairly rich but also seems this side of stuff isn't near a 1.0.0 yet.

[adam-singer]

Reviewable status: 1 of 1 LGTMs obtained, and pending CI: Analyze (javascript-typescript), Analyze (python), Bazel Dev / ubuntu-22.04, Cargo Dev / ubuntu-22.04, Local / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), pre-commit-checks, publish-image, ubuntu-20.04, ubuntu-20.04 / stable, ubuntu-22.04, ubuntu-22.04 / stable, windows-2022 / stable

---

-- commits line 5 at r3:

Previously, adam-singer (Adam Singer) wrote…

I foobar'd something, not a reviewable bug

Done.

---

Cargo.toml line 50 at r2 (raw file):

Previously, aaronmondal (Aaron Siddhartha Mondal) wrote…

nit: The other imports are sorted alphabetically

Done.

[adam-singer]

Reviewable status: 1 of 1 LGTMs obtained, and pending CI: Analyze (javascript-typescript), Analyze (python), Bazel Dev / ubuntu-22.04, Cargo Dev / ubuntu-22.04, Local / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), pre-commit-checks, publish-image, ubuntu-20.04, ubuntu-20.04 / stable, ubuntu-22.04, ubuntu-22.04 / stable, windows-2022 / stable

[adam-singer]

Reviewed 6 of 6 files at r4, all commit messages.
Reviewable status: complete! 1 of 1 LGTMs obtained