[Data-Pipeline] [APMSP-1240] Add concentrator #570
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BenchmarksComparisonBenchmark execution time: 2024-08-22 13:04:11 Comparing candidate commit b80115d in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 49 metrics, 2 unstable metrics. scenario:benching deserializing traces from msgpack to their internal representation
CandidateCandidate benchmark detailsGroup 1
Group 2
Group 3
Group 4
Group 5
Group 6
Group 7
Group 8
Group 9
Group 10
Group 11
BaselineOmitted due to size. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #570 +/- ##
==========================================
+ Coverage 71.73% 72.90% +1.17%
==========================================
Files 238 241 +3
Lines 32941 34364 +1423
==========================================
+ Hits 23631 25054 +1423
Misses 9310 9310
|
VianneyRuhlmann
force-pushed
the
vianney/data-pipeline/add-stats-bucket
branch
from
401f75c
to
fabcd74
Compare
VianneyRuhlmann
force-pushed
the
vianney/data-pipeline/add-stats-bucket
branch
from
33cd1cf
to
6778da6
Compare
VianneyRuhlmann
changed the base branch from
vianney/data-pipeline/stats-computation
to
main
VianneyRuhlmann
force-pushed
the
vianney/data-pipeline/add-stats-bucket
branch
from
89bc718
to
c750b0d
Compare
VianneyRuhlmann
force-pushed
the
vianney/data-pipeline/add-stats-bucket
branch
from
c750b0d
to
1a295e5
Compare
ekump
reviewed
Aug 5, 2024
| @@ -410,6 +412,13 @@ pub fn compute_top_level_span(trace: &mut [pb::Span]) { | |||
| } | |||
| } | |||
|
|
|||
| pub fn has_top_level(span: &pb::Span) -> bool { | |||
There was a problem hiding this comment.
These public functions should have unit tests
ekump
reviewed
Aug 5, 2024
| } | ||
|
|
||
| pub fn add_span(&mut self, span: &pb::Span) -> Result<()> { | ||
| if !(trace_utils::has_top_level(span) |
There was a problem hiding this comment.
minor: I think this if statement is complex enough that it should be moved into a separate function for readability / maintainability purposes. Something along the lines of:
fn should_ignore_span(span: &Span, compute_stats_by_span_kind: bool) -> bool {
!(trace_utils::has_top_level(span)
|| trace_utils::is_measured(span)
|| (self.compute_stats_by_span_kind && compute_stats_for_span_kind))
|| trace_utils::is_partial_snapshot(span)
}
VianneyRuhlmann
changed the title
bantonsson
approved these changes
Aug 20, 2024
| pub is_synthetics_request: bool, | ||
| pub peer_tags: Vec<Tag>, | ||
| pub is_trace_root: bool, | ||
| } |
There was a problem hiding this comment.
This is not for this PR, but maybe the AggregationKey should cache a precomputed hash value (since it's immutable anyway), and implement Hash by itself.
VianneyRuhlmann
force-pushed
the
vianney/data-pipeline/add-stats-bucket
branch
from
c6b0466
to
60be546
Compare
VianneyRuhlmann
force-pushed
the
vianney/data-pipeline/add-stats-bucket
branch
2 times, most recently
from
1133a4e
to
ea7bee7
Compare
Change concentrator name to be consistent with the trace agent
VianneyRuhlmann
force-pushed
the
vianney/data-pipeline/add-stats-bucket
branch
from
ea7bee7
to
a517d6b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Implement the stats concentrator from the trace agent, to allow stats computation in the data pipeline.
Motivation
Is required to allow stats computation in libdatadog