This repository has been archived by the owner on Nov 15, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Turn on the logger's interest cache #11264
Merged
paritytech-processbot
merged 2 commits into
paritytech:master
from
koute:master_faster_not_logging_final
Apr 26, 2022
Merged
Turn on the logger's interest cache #11264
paritytech-processbot
merged 2 commits into
paritytech:master
from
koute:master_faster_not_logging_final
Apr 26, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
koute
added
A0-please_review
Pull request needs code review.
I9-optimisation
An enhancement to provide better overall performance in terms of time-to-completion for a task.
B0-silent
Changes should not be mentioned in any release notes
C1-low
PR touches the given topic and has a low impact on builders.
D3-trivial 🧸
PR contains trivial changes in a runtime directory that do not require an audit
labels
Apr 22, 2022
bkchr
approved these changes
Apr 22, 2022
gilescope
approved these changes
Apr 25, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great to see it land at last! Good things come to those that wait.
bot merge |
Error: "Check reviews" status is not passing for paritytech/polkadot#5393 |
bot merge |
2 tasks
godcodehunter
pushed a commit
to sensoriumxr/substrate
that referenced
this pull request
Jun 22, 2022
DaviRain-Su
pushed a commit
to octopus-network/substrate
that referenced
this pull request
Aug 23, 2022
ark0f
pushed a commit
to gear-tech/substrate
that referenced
this pull request
Feb 27, 2023
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Labels
A0-please_review
Pull request needs code review.
B0-silent
Changes should not be mentioned in any release notes
C1-low
PR touches the given topic and has a low impact on builders.
D3-trivial 🧸
PR contains trivial changes in a runtime directory that do not require an audit
I9-optimisation
An enhancement to provide better overall performance in terms of time-to-completion for a task.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
polkadot companion: paritytech/polkadot#5393
This lowers the CPU overhead of our logging machinery when any
trace
log is enabled down to ~3% of what it is without anytrace
logs enabled.Background
The
tracing-log
crate has a bunch of relatively expensive filtering machinery which decides whether a given log should be printed out or not. This machinery has a fast path: if the current most verbose logging level is set to X then the logger can just reject every log which uses a more verbose level and skip the expensive checks.For example, if we have set
RUST_LOG
tofoo=info,bar=warn
then the most verbose log level isinfo
, so anytrace
logs can be quickly rejected because we know that a log must be at leastinfo
to be printable. Now, if we have setRUST_LOG
tofoo=info,bar=trace
then the most verbose log level istrace
, so now for anytrace
log the logger must actually check whether that log came fromfoo
or frombar
to decide whether it should be printed out, and this is expensive when you're generating thousands oftrace
logs every second (even if they're not printed out!).So this PR enables a built-in interest cache which I've previously added to
tracing-log
. This cache allows the logger to essentially skip the heavyweight filtering machinery by just doing the work once, and then just querying the cache on any subsequent call. According to the measurements I've done a long time ago this should cut down the CPU usage of the logger to ~3% when you have anytrace
log enabled (even if it's for a target which doesn't actually exist and will never actually match anything).(This is the second time I'm making this PR; the previous PR couldn't be merged since the
tracing-log
crate was not released on crates.io; but that finally changed today with the release oftracing-log
0.1.3.)