subscriber: update sharded-slab to 0.1, pool hashmap allocations #1062
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
|
i'll also backport this to v0.1.x once it merges. |
this fixes an issue where the first time a slot is used, it starts with 0 refs instead of 1 ref, because the ref count is set to 1 in `clear` instead. Signed-off-by: Eliza Weisman <eliza@buoyant.io>
hawkw
commented
Oct 22, 2020
davidbarsky
reviewed
Oct 22, 2020
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
hawkw
added a commit
that referenced
this pull request
Oct 22, 2020
`tracing-subscriber` can conditionally use `parking_lot`'s locks rather than `std::sync`'s locks, when a feature flag is enabled. Because `parking_lot`'s locks do not poison on panics, while `std::sync`'s locks do, the APIs differ slightly. Currently, we handle this by wrapping the `parking_lot` `RwLock` in a newtype that returns `Result`s, like `std::sync::RwLock` does, but always returns `Ok`. Then, potentially poisoned locks are handled at the callsite. However, this is unnecessary (and potentially incorrect). Because locks may or may not be poisoned depending on the feature flag, `tracing-subscriber` can't rely on poisoning for correctness. Doing so would mean potentially different behavior between `std::sync` and `parking_lot`. Currently, because a number of `tracing-subscriber` functions are called in `Drop` impls, we don't panic when a lock is poisoned, to avoid double panics. Most cases where a lock might be poisoned are handled by returning early with a default value. However, with `parking_lot` enabled, this will never happen, and it adds complexity at the callsite. This branch changes our approach to wrap `std::sync`'s locks and use `PoisonError::into_inner` to ignore poisoning. This results in more consistent behavior between the `parking_lot` feature and `std`, and also probably improves performance a bit with `parking_lot` --- we don't have to have extra code for handling an `Err` case that will never happen (although rustc might be smart enough to optimize it away). Also, the callsites are simpler since they never have to handle poisoning, and we can't accidentally `unwrap` in a `Drop` impl. I also removed the janky user-space thread-local implementation used by `CurrentSpan`, and changed it to just use the `thread-local` crate. We already depend on that crate for the `Registry`, and my DIY implementation has some performance issues. So, I removed it. Depends on #1062
clippy was INNOCENT Signed-off-by: Eliza Weisman <eliza@buoyant.io>
…eliza/update-slab
davidbarsky
approved these changes
Oct 22, 2020
hawkw
added a commit
that referenced
this pull request
Oct 22, 2020
This backports #1062 to v0.1.6. This has already been approved on master. hawkw/sharded-slab#45 changes `sharded-slab` so that the per-shard metadata is allocated only when a new shard is created, rather than all up front when the slab is created. This fixes the very large amount of memory allocated by simply creating a new `Registry` without actually collecting any traces. This branch updates `tracing-subscriber` to depend on `sharded-slab` 0.1.0, which includes the upstream fix. In addition, this branch the registry from using `sharded_slab::Slab` to `sharded_slab::Pool`. This allows us to clear hashmap allocations for extensions in-place, retaining the already allocated maps. This should improve `new_span` performance a bit. Fixes #1005 Signed-off-by: Eliza Weisman <eliza@buoyant.io>
hawkw
added a commit
that referenced
this pull request
Oct 22, 2020
This backports #1062 to v0.1.6. This has already been approved on master. hawkw/sharded-slab#45 changes `sharded-slab` so that the per-shard metadata is allocated only when a new shard is created, rather than all up front when the slab is created. This fixes the very large amount of memory allocated by simply creating a new `Registry` without actually collecting any traces. This branch updates `tracing-subscriber` to depend on `sharded-slab` 0.1.0, which includes the upstream fix. In addition, this branch the registry from using `sharded_slab::Slab` to `sharded_slab::Pool`. This allows us to clear hashmap allocations for extensions in-place, retaining the already allocated maps. This should improve `new_span` performance a bit. Fixes #1005 Signed-off-by: Eliza Weisman <eliza@buoyant.io>
dvdplm
added a commit
to dvdplm/tracing
that referenced
this pull request
Oct 23, 2020
* upstream/master: subscriber: update sharded-slab to 0.1, pool hashmap allocations (tokio-rs#1062) subscriber: remove deprecated type, structs, and methods tokio-rs#1030 core: rename Subscriber to Collect (tokio-rs#1015) chore: fix rustdoc warning in tracing-subscriber (tokio-rs#1061)
dvdplm
added a commit
to dvdplm/tracing
that referenced
this pull request
Oct 23, 2020
* dp-target-is-cow: subscriber: update sharded-slab to 0.1, pool hashmap allocations (tokio-rs#1062) subscriber: remove deprecated type, structs, and methods tokio-rs#1030 core: rename Subscriber to Collect (tokio-rs#1015) chore: fix rustdoc warning in tracing-subscriber (tokio-rs#1061)
dvdplm
added a commit
to dvdplm/tracing
that referenced
this pull request
Oct 23, 2020
…spatch-as-owned-tokio-rs#455 * upstream/master: subscriber: update sharded-slab to 0.1, pool hashmap allocations (tokio-rs#1062) subscriber: remove deprecated type, structs, and methods tokio-rs#1030 core: rename Subscriber to Collect (tokio-rs#1015) chore: fix rustdoc warning in tracing-subscriber (tokio-rs#1061)
hawkw
added a commit
that referenced
this pull request
Oct 24, 2020
## Motivation `tracing-subscriber` can conditionally use `parking_lot`'s locks rather than `std::sync`'s locks, when a feature flag is enabled. Because `parking_lot`'s locks do not poison on panics, while `std::sync`'s locks do, the APIs differ slightly. Currently, we handle this by wrapping the `parking_lot` `RwLock` in a newtype that returns `Result`s, like `std::sync::RwLock` does, but always returns `Ok`. Then, potentially poisoned locks are handled at the callsite. However, this is unnecessary (and potentially incorrect). Because locks may or may not be poisoned depending on the feature flag, `tracing-subscriber` can't rely on poisoning for correctness. Doing so would mean potentially different behavior between `std::sync` and `parking_lot`. Currently, because a number of `tracing-subscriber` functions are called in `Drop` impls, we don't panic when a lock is poisoned, to avoid double panics. Most cases where a lock might be poisoned are handled by returning early with a default value. However, with `parking_lot` enabled, this will never happen, and it adds complexity at the callsite. ## Solution This branch changes our approach to wrap `std::sync`'s locks and use `PoisonError::into_inner` to ignore poisoning. This results in more consistent behavior between the `parking_lot` feature and `std`, and also probably improves performance a bit with `parking_lot` --- we don't have to have extra code for handling an `Err` case that will never happen (although rustc might be smart enough to optimize it away). Also, the callsites are simpler since they never have to handle poisoning, and we can't accidentally `unwrap` in a `Drop` impl. I also removed the janky user-space thread-local implementation used by `CurrentSpan`, and changed it to just use the `thread-local` crate. We already depend on that crate for the `Registry`, and my DIY implementation has some performance issues. So, I removed it. Depends on #1062
kaffarell
pushed a commit
to kaffarell/tracing
that referenced
this pull request
May 22, 2024
This backports tokio-rs#1062 to v0.1.6. This has already been approved on master. hawkw/sharded-slab#45 changes `sharded-slab` so that the per-shard metadata is allocated only when a new shard is created, rather than all up front when the slab is created. This fixes the very large amount of memory allocated by simply creating a new `Registry` without actually collecting any traces. This branch updates `tracing-subscriber` to depend on `sharded-slab` 0.1.0, which includes the upstream fix. In addition, this branch the registry from using `sharded_slab::Slab` to `sharded_slab::Pool`. This allows us to clear hashmap allocations for extensions in-place, retaining the already allocated maps. This should improve `new_span` performance a bit. Fixes tokio-rs#1005 Signed-off-by: Eliza Weisman <eliza@buoyant.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
hawkw/sharded-slab#45 changes
sharded-slabso that the per-shardmetadata is allocated only when a new shard is created, rather than all
up front when the slab is created. This fixes the very large amount of
memory allocated by simply creating a new
Registrywithout actuallycollecting any traces.
Solution
This branch updates
tracing-subscriberto depend onsharded-slab0.1.0, which includes the upstream fix.
In addition, this branch the registry from using
sharded_slab::Slabtosharded_slab::Pool. This allows us to clear hashmap allocations forextensions in-place, retaining the already allocated maps. This should
improve
new_spanperformance a bit.Fixes #1005