-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add request tracing framework #7648
Add request tracing framework #7648
Conversation
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @suranjay picking this up. What is the overhead of Tracer framework disabled by default? Do we have some nos.?
server/src/main/java/org/opensearch/common/util/concurrent/ThreadContext.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/tracing/OTelResourceProvider.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/common/settings/ClusterSettings.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/tracing/OTelResourceProvider.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please consider re-wiring dependencies. It seems too tightly coupled with Otel. Please also consider moving this to a module
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
@@ Coverage Diff @@
## main #7648 +/- ##
============================================
+ Coverage 70.90% 70.97% +0.07%
- Complexity 56710 56866 +156
============================================
Files 4733 4752 +19
Lines 268333 268599 +266
Branches 39322 39336 +14
============================================
+ Hits 190254 190645 +391
+ Misses 61996 61888 -108
+ Partials 16083 16066 -17
|
Gradle Check (Jenkins) Run Completed with:
|
764862b
to
60c2a47
Compare
Gradle Check (Jenkins) Run Completed with:
|
60c2a47
to
7f5a378
Compare
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: suranjay <surajkumar.tu@gmail.com>
Signed-off-by: suranjay <surajkumar.tu@gmail.com>
Gradle Check (Jenkins) Run Completed with:
|
1 similar comment
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: suranjay <surajkumar.tu@gmail.com>
Gradle Check (Jenkins) Run Completed with:
|
@shwetathareja There is no overhead with of Tracer framework in disabled state. We have verified this by doing performance benchmarking using OSB |
Signed-off-by: suranjay <surajkumar.tu@gmail.com>
Gradle Check (Jenkins) Run Completed with:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @suranjay . I am still going over the changes, published some feedback after first pass. Btw, you can change the status to actual PR from DRAFT.
modules/tracer-otel/src/main/java/org/opensearch/tracing/DefaultTracer.java
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/main/java/org/opensearch/tracing/OTelResourceProvider.java
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/main/java/org/opensearch/tracing/DefaultTracer.java
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/main/java/org/opensearch/tracing/DefaultTracer.java
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/main/java/org/opensearch/tracing/DefaultTracer.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few files left from TracerManager.....
modules/tracer-otel/src/main/java/org/opensearch/tracing/DefaultSpan.java
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/main/java/org/opensearch/tracing/DefaultSpan.java
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/main/java/org/opensearch/tracing/DefaultTracer.java
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/main/java/org/opensearch/tracing/DefaultTracer.java
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/main/plugin-metadata/plugin-security.policy
Outdated
Show resolved
Hide resolved
modules/tracer-otel/src/test/java/org/opensearch/tracing/exporter/MockAppender.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/common/util/FeatureFlags.java
Outdated
Show resolved
Hide resolved
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
server/src/main/java/org/opensearch/common/util/concurrent/ThreadContextStatePropagator.java
Show resolved
Hide resolved
Thanks @suranjay, LGTM! |
Thanks @suranjay ! |
(PS: the build should be fixed by #8290) |
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: suranjay <surajkumar.tu@gmail.com>
3e1e031
to
7637b37
Compare
Signed-off-by: suranjay <surajkumar.tu@gmail.com>
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-7648-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 03bc19220a711853660652a4761c05285f597884
# Push it to GitHub
git push --set-upstream origin backport/backport-7648-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x Then, create a pull request where the |
* Add request tracing framework Signed-off-by: suranjay <surajkumar.tu@gmail.com> * Introduce ThreadContextStatePropagator to propagate request and transient headers within ThreadContext Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: suranjay <surajkumar.tu@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
* Add request tracing framework Signed-off-by: suranjay <surajkumar.tu@gmail.com> * Introduce ThreadContextStatePropagator to propagate request and transient headers within ThreadContext Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: suranjay <surajkumar.tu@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
* Add request tracing framework Signed-off-by: suranjay <surajkumar.tu@gmail.com> * Introduce ThreadContextStatePropagator to propagate request and transient headers within ThreadContext Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: suranjay <surajkumar.tu@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
* Add request tracing framework Signed-off-by: suranjay <surajkumar.tu@gmail.com> * Introduce ThreadContextStatePropagator to propagate request and transient headers within ThreadContext Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: suranjay <surajkumar.tu@gmail.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Description
This change adds request tracing framework in OpenSearch. It also integrates Opentelemetry with OpenSearch as it uses the framework for implementing tracing capabilities in OpenSearch.
The tracing framework automatically handles context propagation between different threads, tasks and nodes.
Note: The integrations tests will be covered in the subsequent PR to limit the changes in this PR.
Related Issues
Resolves #7543, #7544
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.