fix: httpjson callables to trace attempts (started, failed) #3300

diegomarquezp · 2024-10-21T20:21:18Z

Approach

This PR uses the approach suggested by the same bug.

We should move creating the TracedUnaryCallable to the last step for HttpJson. This would require refactoring this file: https://github.com/googleapis/sdk-platform-java/blob/7902a41c87240d607179d07c28cce462ea135c5f/gax-java/gax-httpjson/src/main/java/com/google/api/gax/httpjson/HttpJsonCallableFactory.java

Confirmation that it works

The confirmation that it works is in the modified tests (e.g. retry()). When using the proposed test version against the version of HttpJsonCallableFactory from the main branch, it will fail because it will not record attempts started or failed whatsoever:

The image above shows all tests having failed when using the production files from main, implying that no tracer attempts are recorded for any of the tests.

…trics

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java

lqiu96 · 2024-10-22T18:06:47Z

I think you can also remove a bunch of the @Disabled annotations in the showcase ITs as well.

sdk-platform-java/showcase/gapic-showcase/src/test/java/com/google/showcase/v1beta1/it/ITOtelMetrics.java

Line 323 in 7512cfa

    
           @Disabled("https://github.com/googleapis/sdk-platform-java/issues/2503")

I think they should pass with your PR.

…eapis/sdk-platform-java into fix-httpjson-retry-metrics

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java

lqiu96

~~Can you try and re-enable the Disabled ITOtelMetric tests and see if it works? LGTM otherwise~~

Edit: Spoke too soon.

showcase/gapic-showcase/src/test/java/com/google/showcase/v1beta1/it/ITOtelMetrics.java

…metrics' into fix-httpjson-retry-metrics

…eapis/sdk-platform-java into fix-httpjson-retry-metrics

…trics

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java

lqiu96 · 2024-10-28T14:06:32Z

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java

+    // the number of attempts varies. Here we just make sure that all of them except the last are
+    // considered as failed


qq, isn't the last one also considered a failed attempt? Why is the tracerAttemptsFailed != tracerAttempts?

Also, can we add the operationfailed == true check here as well?

Why is the tracerAttemptsFailed != tracerAttempts?

I'm not sure and I agree they should be the same value since this test just fetches an exception until it can't retry anymore (retries exhausted).

from discussion with @blakeli0, @lqiu96 and @zhumin8, let's double check TestApiTracer, confirm with showcase.

showcase has another tracer we can use to cross check TestApiTracer

lqiu96

LGTM. Added a few questions/ nits

blakeli0 · 2024-10-30T22:02:37Z

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/testing/TestApiTracer.java

+ */
+public class TestApiTracer extends BaseApiTracer {
+
+  private final AtomicInteger tracerAttempts;


I think they can be called just attempts, same for the fields below.

renamed to attemptsStarted, removed the tracer prefix from all fields

blakeli0 · 2024-10-30T22:05:14Z

...ava/gax-httpjson/src/test/java/com/google/api/gax/httpjson/testing/TestApiTracerFactory.java

+          tracerOperationFailed,
+          tracerFailedRetriesExhausted);
+
+  public AtomicInteger getTracerAttempts() {


I think it's better to get the tracer instance, and use the getters from the tracer in unit tests.

blakeli0 · 2024-10-30T22:05:47Z

...ava/gax-httpjson/src/test/java/com/google/api/gax/httpjson/testing/TestApiTracerFactory.java

+ * exhausted.
+ */
+public class TestApiTracerFactory implements ApiTracerFactory {
+  private final AtomicInteger tracerAttempts = new AtomicInteger();


These can be initialized within the tracer for better encapsulation.

I'd like to initialize them in the tracer itself. However the clients only expect tracer factories, so there is no way to have control over how the actual tracers are instantiated.
I'll double check on this.

sdk-platform-java/gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java

Lines 121 to 128 in 2df2e9f

clientContext =

ClientContext.newBuilder()

// we use a custom tracer to confirm whether the retrials are being recorded.

.setTracerFactory(tracerFactory)

.setExecutor(executor)

.setClock(fakeClock)

.setDefaultCallContext(HttpJsonCallContext.createDefault())

.build();

Ah nvm, I realized that accessing the ApiTracer instance should be enough.

blakeli0 · 2024-10-30T22:09:23Z

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/testing/TestApiTracer.java

+  }
+
+  @Override
+  public void attemptStarted(int attemptNumber) {


Use attemptStarted(Object request, int attemptNumber) instead

blakeli0 · 2024-10-30T22:15:52Z

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/testing/TestApiTracer.java

I like the idea of a test tracer in general. On the other hand, we should only implement what is needed for our use cases, so that we have lower chance to run into issues that are not related to our PR. For example, maybe we don't need to implement attemptFailedRetriesExhausted.

@lqiu96 figured that the retries exhausted event records the missing failed attempt in our tests:

It looks like attempt_count is recorded when this is called:

sdk-platform-java/gax-java/gax/src/main/java/com/google/api/gax/tracing/MetricsTracer.java

Line 204 in 4d4c798

metricsRecorder.recordAttemptCount(1, attributes);

. This seems like the number recorded in Otel should be correct.
I'm guessing your TestApiTracer would need to increment the attemptCount when attemptFailedRetriesExhausted is called as well to match behavior.

I'll keep this implementation and its retries exhausted flag.

blakeli0 · 2024-10-30T22:16:13Z

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/testing/TestApiTracer.java

+ * Test tracer that keeps count of different events. See {@link TestApiTracerFactory} for more
+ * details.
+ */
+public class TestApiTracer extends BaseApiTracer {


We should use the interface ApiTracer instead.

This reverts commit e57b4db.

lqiu96 · 2024-11-06T22:15:09Z

FYI @diegomarquezp
CI error:

Error:  Failures: 
Error:    RetryingTest.retryKeepFailing:360 expected to be true

sonarcloud · 2024-11-11T22:59:23Z

Quality Gate passed for 'gapic-generator-java-root'

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

sonarcloud · 2024-11-11T23:03:00Z

Quality Gate passed for 'java_showcase_integration_tests'

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

blakeli0

LGTM. Please make sure @lqiu96 has no other concerns before merging.

diegomarquezp · 2024-11-13T23:28:43Z

From @lqiu96 (thanks!):

I have a suspicion as to why the race condition exists. I took a look into the code and I think I understand what you were explaining in the call.
My suspicion of the cause of the race condition:

sdk-platform-java/gax-java/gax/src/main/java/com/google/api/gax/retrying/BasicRetryingFuture.java

Line 205 in bb2a3be

super.set(response);

This sets the result to the future which frees up the thread that is running the test. (basically .get() no longer blocks and if the thread schedules the test to run, it will). The tracerFinisher code which sets the operation failure status runs after this

This means that the tests removed in 065b01f were actually expected behavior and is due to the way we set up the tests.

🤖 I have created a release *beep* *boop* --- <details><summary>2.50.0</summary> ## [2.50.0](v2.49.0...v2.50.0) (2024-11-14) ### Features * Add experimental S2A integration in client libraries grpc transport ([#3326](#3326)) ([1138ca6](1138ca6)) * enable selective generation based on service config include list ([#3323](#3323)) ([0cddadb](0cddadb)) * introduce `java.time` to java-core ([#3330](#3330)) ([f202c3b](f202c3b)) * Update Gapic-Generator to generate libraries using `java.time` methods ([#3321](#3321)) ([b21c9a4](b21c9a4)) ### Bug Fixes * Fix flaky test ScheduledRetryingExecutorTest.testCancelOuterFutureAfterStart ([#3335](#3335)) ([e73740d](e73740d)) * httpjson callables to trace attempts (started, failed) ([#3300](#3300)) ([15a64ee](15a64ee)) * instantiate GaxProperties at build time to ensure we get the protobuf version ([#3365](#3365)) ([bb2a3be](bb2a3be)) * protobuf version not always getting set in headers ([#3322](#3322)) ([7f6e470](7f6e470)) * use BuildKit instead of legacy builder to build the Hermetic Build images ([#3338](#3338)) ([222fb45](222fb45)) ### Dependencies * update google auth library dependencies to v1.30.0 ([#3367](#3367)) ([a31c682](a31c682)) * update grpc dependencies to v1.68.1 ([#3240](#3240)) ([c8e3941](c8e3941)) ### Documentation * fix list num ([#3356](#3356)) ([b7d6296](b7d6296)) * **hermetic-build:** indicate usage of Docker Buildkit in development guide ([#3337](#3337)) ([01e742d](01e742d)) * modify hermetic build docs ([#3331](#3331)) ([25023af](25023af)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.api.grpc:proto-google-common-protos](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.49.0` -> `2.50.0` | | [com.google.cloud:google-cloud-core-http](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.48.0` -> `2.49.0` | | [com.google.cloud:google-cloud-spanner](https://github.com/googleapis/java-spanner) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `6.82.0` -> `6.83.0` | | [com.google.cloud:google-cloud-logging](https://github.com/googleapis/java-logging) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `3.20.7` -> `3.21.0` | | [com.google.cloud:google-cloud-datastore](https://github.com/googleapis/java-datastore) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.24.3` -> `2.25.1` | | [com.google.cloud:google-cloud-core](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.48.0` -> `2.49.0` | | [com.google.api:gax](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.58.0` -> `2.59.0` | | [com.autonomousapps.dependency-analysis](https://github.com/autonomousapps/dependency-analysis-android-gradle-plugin) | plugin | misk/gradle/libs.versions.toml | gradle | patch | `2.6.0` -> `2.6.1` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.43.0` -> `1.44.1` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.43.0` -> `1.44.1` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.32` -> `2.29.34` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.32` -> `2.29.34` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.32` -> `2.29.34` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.32` -> `2.29.34` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.32` -> `2.29.34` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.29.32` -> `2.29.34` | | [com.amazonaws:aws-java-sdk-sqs](https://aws.amazon.com/sdkforjava) ([source](https://github.com/aws/aws-sdk-java)) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `1.12.779` -> `1.12.780` | | [com.amazonaws:aws-java-sdk-s3](https://aws.amazon.com/sdkforjava) ([source](https://github.com/aws/aws-sdk-java)) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `1.12.779` -> `1.12.780` | | [com.amazonaws:aws-java-sdk-dynamodb](https://aws.amazon.com/sdkforjava) ([source](https://github.com/aws/aws-sdk-java)) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `1.12.779` -> `1.12.780` | | [com.amazonaws:aws-java-sdk-core](https://aws.amazon.com/sdkforjava) ([source](https://github.com/aws/aws-sdk-java)) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `1.12.779` -> `1.12.780` | --- ### Release Notes <details> <summary>googleapis/sdk-platform-java (com.google.api.grpc:proto-google-common-protos)</summary> ### [`v2.50.0`](https://github.com/googleapis/sdk-platform-java/blob/HEAD/CHANGELOG.md#2500-2024-11-14) ##### Features - Add experimental S2A integration in client libraries grpc transport ([#3326](googleapis/sdk-platform-java#3326)) ([1138ca6](googleapis/sdk-platform-java@1138ca6)) - enable selective generation based on service config include list ([#3323](googleapis/sdk-platform-java#3323)) ([0cddadb](googleapis/sdk-platform-java@0cddadb)) - introduce `java.time` to java-core ([#3330](googleapis/sdk-platform-java#3330)) ([f202c3b](googleapis/sdk-platform-java@f202c3b)) - Update Gapic-Generator to generate libraries using `java.time` methods ([#3321](googleapis/sdk-platform-java#3321)) ([b21c9a4](googleapis/sdk-platform-java@b21c9a4)) ##### Bug Fixes - Fix flaky test ScheduledRetryingExecutorTest.testCancelOuterFutureAfterStart ([#3335](googleapis/sdk-platform-java#3335)) ([e73740d](googleapis/sdk-platform-java@e73740d)) - httpjson callables to trace attempts (started, failed) ([#3300](googleapis/sdk-platform-java#3300)) ([15a64ee](googleapis/sdk-platform-java@15a64ee)) - instantiate GaxProperties at build time to ensure we get the protobuf version ([#3365](googleapis/sdk-platform-java#3365)) ([bb2a3be](googleapis/sdk-platform-java@bb2a3be)) - protobuf version not always getting set in headers ([#3322](googleapis/sdk-platform-java#3322)) ([7f6e470](googleapis/sdk-platform-java@7f6e470)) - use BuildKit instead of legacy builder to build the Hermetic Build images ([#3338](googleapis/sdk-platform-java#3338)) ([222fb45](googleapis/sdk-platform-java@222fb45)) ##### Dependencies - update google auth library dependencies to v1.30.0 ([#3367](googleapis/sdk-platform-java#3367)) ([a31c682](googleapis/sdk-platform-java@a31c682)) - update grpc dependencies to v1.68.1 ([#3240](googleapis/sdk-platform-java#3240)) ([c8e3941](googleapis/sdk-platform-java@c8e3941)) ##### Documentation - fix list num ([#3356](googleapis/sdk-platform-java#3356)) ([b7d6296](googleapis/sdk-platform-java@b7d6296)) - **hermetic-build:** indicate usage of Docker Buildkit in development guide ([#3337](googleapis/sdk-platform-java#3337)) ([01e742d](googleapis/sdk-platform-java@01e742d)) - modify hermetic build docs ([#3331](googleapis/sdk-platform-java#3331)) ([25023af](googleapis/sdk-platform-java@25023af)) </details> <details> <summary>googleapis/java-spanner (com.google.cloud:google-cloud-spanner)</summary> ### [`v6.83.0`](https://github.com/googleapis/java-spanner/blob/HEAD/CHANGELOG.md#6830-2024-12-13) ##### Features - Add Metrics host for built in metrics ([#3519](googleapis/java-spanner#3519)) ([4ed455a](googleapis/java-spanner@4ed455a)) - Add opt-in for using multiplexed sessions for blind writes ([#3540](googleapis/java-spanner#3540)) ([216f53e](googleapis/java-spanner@216f53e)) - Add UUID in Spanner TypeCode enum ([41f83dc](googleapis/java-spanner@41f83dc)) - Introduce java.time variables and methods ([#3495](googleapis/java-spanner#3495)) ([8a7d533](googleapis/java-spanner@8a7d533)) - **spanner:** Support multiplexed session for Partitioned operations ([#3231](googleapis/java-spanner#3231)) ([4501a3e](googleapis/java-spanner@4501a3e)) - Support 'set local' for retry_aborts_internally ([#3532](googleapis/java-spanner#3532)) ([331942f](googleapis/java-spanner@331942f)) ##### Bug Fixes - **deps:** Update the Java code generator (gapic-generator-java) to 2.51.0 ([41f83dc](googleapis/java-spanner@41f83dc)) ##### Dependencies - Update sdk platform java dependencies ([#3549](googleapis/java-spanner#3549)) ([6235f0f](googleapis/java-spanner@6235f0f)) </details> <details> <summary>googleapis/java-logging (com.google.cloud:google-cloud-logging)</summary> ### [`v3.21.0`](https://github.com/googleapis/java-logging/blob/HEAD/CHANGELOG.md#3210-2024-12-13) ##### Features - Introduce `java.time` methods ([#1729](googleapis/java-logging#1729)) ([323eb33](googleapis/java-logging@323eb33)) ##### Bug Fixes - **deps:** Update the Java code generator (gapic-generator-java) to 2.51.0 ([04d8868](googleapis/java-logging@04d8868)) ##### Dependencies - Update dependency io.opentelemetry:opentelemetry-bom to v1.45.0 ([#1638](googleapis/java-logging#1638)) ([7e007d4](googleapis/java-logging@7e007d4)) - Update sdk platform java dependencies ([#1736](googleapis/java-logging#1736)) ([88b4cdf](googleapis/java-logging@88b4cdf)) </details> <details> <summary>googleapis/java-datastore (com.google.cloud:google-cloud-datastore)</summary> ### [`v2.25.1`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2251-2024-12-13) ##### Bug Fixes - **deps:** Update the Java code generator (gapic-generator-java) to 2.51.0 ([106ee4d](googleapis/java-datastore@106ee4d)) ##### Dependencies - Update sdk platform java dependencies ([#1685](googleapis/java-datastore#1685)) ([4372350](googleapis/java-datastore@4372350)) ### [`v2.25.0`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2250-2024-12-11) ##### Features - Introduce `java.time` methods and variables ([#1671](googleapis/java-datastore#1671)) ([5a78a80](googleapis/java-datastore@5a78a80)) ##### Dependencies - Update dependency com.google.cloud:gapic-libraries-bom to v1.48.0 ([#1605](googleapis/java-datastore#1605)) ([5c6a678](googleapis/java-datastore@5c6a678)) ##### Documentation - Update gapic upgrade installation instructions ([#1677](googleapis/java-datastore#1677)) ([b3fbfcc](googleapis/java-datastore@b3fbfcc)) </details> <details> <summary>autonomousapps/dependency-analysis-android-gradle-plugin (com.autonomousapps.dependency-analysis)</summary> ### [`v2.6.1`](https://github.com/autonomousapps/dependency-analysis-android-gradle-plugin/blob/HEAD/CHANGELOG.md#Version-261) - \[Fix]: `superClassName` can be null (Object has no superclass). </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.44.1`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.44.1): 1.44.1 ##### Components ##### Continuous Integration Visibility - 🐛 Fix tracing JUnit5 tests in Maven projects with multiple forks ([#8089](DataDog/dd-trace-java#8089) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) ### [`v1.44.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.44.0): 1.44.0 ##### Known Issues > \[!WARNING]\ > This release contains a known issue that causes failures when using Test Optimization to trace JUnit 5 tests in a Maven project where Maven Surefire is configured with `forkCount` > 1. > The issue is fixed in v1.44.1 ##### Breaking Changes > \[!WARNING]\ > Support for `X-Forwarded` header is dropped from default client IP resolution. > It can still be re-activated using the `dd.trace.client-ip-header=x-forwarded` system property, or the `DD_TRACE_CLIENT_IP_HEADER=x-forwarded` environment variable. See [#7946](DataDog/dd-trace-java#7946). ##### Components ##### Application Security Management (IAST) - ✨ Set unexpected IAST exceptions to debug log level ([#8044](DataDog/dd-trace-java#8044) - [@smola](https://github.com/smola)) - ✨ Increase IAST propagation to StringBuffer subSequence ([#8038](DataDog/dd-trace-java#8038) - [@Mariovido](https://github.com/Mariovido)) - ✨ Increase IAST propagation to StringBuilder subSequence ([#8026](DataDog/dd-trace-java#8026) - [@Mariovido](https://github.com/Mariovido)) - ✨ Add IAST propagation to String valueOf ([#8013](DataDog/dd-trace-java#8013) - [@Mariovido](https://github.com/Mariovido)) - ✨ Increase IAST propagation to StringBuilder append ([#8010](DataDog/dd-trace-java#8010) - [@Mariovido](https://github.com/Mariovido)) - ✨ Expand SSRF support in IAST to apache-httpclient-5 and apache-httpasyncclient-4 ([#7920](DataDog/dd-trace-java#7920) - [@Mariovido](https://github.com/Mariovido)) ##### Build & Tooling - ✨ Generate Muzzle classes for Groovy instrumentations ([#8004](DataDog/dd-trace-java#8004) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) ##### Continuous Integration Visibility - ✨ Support distributed traces in tests ([#8078](DataDog/dd-trace-java#8078) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Implement fail-fast tests ordering for JUnit 5 ([#8055](DataDog/dd-trace-java#8055) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Mark JUnit 5 setup and teardown action spans as failed if there is an error ([#8033](DataDog/dd-trace-java#8033) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add tracing of setup and teardown actions in JUnit 4 ([#8030](DataDog/dd-trace-java#8030) - [@daniel-mohedano](https://github.com/daniel-mohedano)) ##### Crash tracking - ✨ Improve crash tracking install logging ([#8045](DataDog/dd-trace-java#8045) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Data Streams Monitoring - 🐛 Add Data Streams support in AWS SQS without raw message delivery ([#8071](DataDog/dd-trace-java#8071) - [@piochelepiotr](https://github.com/piochelepiotr)) - ✨ Add new tag for enabled products / features to DSM checkpoints ([#8051](DataDog/dd-trace-java#8051) - [@kr-igor](https://github.com/kr-igor)) - 💡 Instrument self hosted Kafka connectors ([#7959](DataDog/dd-trace-java#7959) - [@piochelepiotr](https://github.com/piochelepiotr)) ##### Dynamic Instrumentation - ✨ Add Micronaut 4 support for code origin for spans ([#8039](DataDog/dd-trace-java#8039) - [@jpbempel](https://github.com/jpbempel)) - ✨ Refactor probe matching for methods ([#8021](DataDog/dd-trace-java#8021) - [@jpbempel](https://github.com/jpbempel)) - ✨ Update the CodeOriginProbe fingerprint to not rely on a stack walk ([#8016](DataDog/dd-trace-java#8016) - [@evanchooly](https://github.com/evanchooly)) - ✨ Implement code origin support for grpc server entry spans ([#7942](DataDog/dd-trace-java#7942) - [@evanchooly](https://github.com/evanchooly)) ##### GraalVM native-image - 🐛 Update Graal build-time instrumentation config for TracePropagationStyle ([#8065](DataDog/dd-trace-java#8065) - [@MattAlp](https://github.com/MattAlp)) - 🐛 Fix NoClassDefFoundError: Could not initialize class DDSpanLink$EncoderHolder in Graal native-image ([#8036](DataDog/dd-trace-java#8036) - [@mcculls](https://github.com/mcculls)) - 🐛🧹 Fix native-image generation of reactive applications ([#8012](DataDog/dd-trace-java#8012) - [@mcculls](https://github.com/mcculls)) ##### OpenTracing - 🧹 Custom ScopeManagers are deprecated and will be removed in a future release of dd-trace-ot ([#8058](DataDog/dd-trace-java#8058) - [@mcculls](https://github.com/mcculls)) ##### Tracer core - ✨🧪 Service naming: split by jee deployment ([#8064](DataDog/dd-trace-java#8064) - [@amarziali](https://github.com/amarziali)) - ✨ Exclude jboss mdb proxies from instrumenting ([#8061](DataDog/dd-trace-java#8061) - [@amarziali](https://github.com/amarziali)) - ✨ Add a built-in trace interceptor for keeping traces depending of their latency ([#8040](DataDog/dd-trace-java#8040) - [@cecile75](https://github.com/cecile75)) - 💡 Introduce marker mechanism for eagerly initializing helpers ([#8028](DataDog/dd-trace-java#8028) - [@mcculls](https://github.com/mcculls)) - 💡 Add JSON component ([#7973](DataDog/dd-trace-java#7973) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨⚠️ Remove support for X-Forwarded in client IP resolution ([#7946](DataDog/dd-trace-java#7946) - [@smola](https://github.com/smola)) ##### Instrumentations ##### Apache HttpComponents - ✨ Expand SSRF support in IAST to apache-httpclient-5 and apache-httpasyncclient-4 ([#7920](DataDog/dd-trace-java#7920) - [@Mariovido](https://github.com/Mariovido)) ##### gRPC instrumentation - 🐛 Use lower priorities for grpc server errors ([#8043](DataDog/dd-trace-java#8043) - [@amarziali](https://github.com/amarziali)) ##### JDBC instrumentation - ✨ Add trace injection for prepared statements in Postgres ([#7940](DataDog/dd-trace-java#7940) - [@nenadnoveljic](https://github.com/nenadnoveljic)) ##### JMS instrumentation - 🐛 Protect mdb from instrumenting multiple time the same event ([#8062](DataDog/dd-trace-java#8062) - [@amarziali](https://github.com/amarziali)) ##### Kafka instrumentation - 💡 Instrument self hosted Kafka connectors ([#7959](DataDog/dd-trace-java#7959) - [@piochelepiotr](https://github.com/piochelepiotr)) ##### OpenTelemetry instrumentation - 🐛 Support using OpenTelemetry Event API inside `@WithSpan` annotated method ([#8019](DataDog/dd-trace-java#8019) - [@mcculls](https://github.com/mcculls)) ##### Reactor instrumentation - 🐛🧹 Fix native-image generation of reactive applications ([#8012](DataDog/dd-trace-java#8012) - [@mcculls](https://github.com/mcculls)) ##### Spring instrumentation - 🐛 Avoid double instrumenting lambdas on latest spring scheduling ([#8005](DataDog/dd-trace-java#8005) - [@amarziali](https://github.com/amarziali)) ##### All other instrumentations - 🐛 Twilio: allow service name flattening ([#8025](DataDog/dd-trace-java#8025) - [@amarziali](https://github.com/amarziali)) - ✨ Instrument Mulesoft 4.5.0+ ([#7981](DataDog/dd-trace-java#7981) - [@amarziali](https://github.com/amarziali)) </details> <details> <summary>aws/aws-sdk-java (com.amazonaws:aws-java-sdk-sqs)</summary> ### [`v1.12.780`](https://github.com/aws/aws-sdk-java/blob/HEAD/CHANGELOG.md#112780-2024-12-11) [Compare Source](aws/aws-sdk-java@1.12.779...1.12.780) #### **Amazon Simple Storage Service** - ### Bugfixes - AWS SDK for Java 1.x now includes additional validation for Amazon S3 client APIs to handle scenarios where an empty string ('') is passed as the key argument to the following operations: PutObject, DeleteObject, ListObjects, GetObjectMetaData, ListObjectsV2, SetObjectTagging, GetObjectTagging, SetObjectAcl, GetObjectAcl, SetObjectLegalHold, GetObjectLegalHold, CopyObject, CopyPart, SelectObjectContent, SetObjectRetention, GetObjectRetention, AbortMultipartUpload, CompleteMultipartUpload, InitiateMultipartUpload, ListParts, UploadPart, RestoreObjectV2, and RestoreObject. The SDK will validate the key argument and throw an exception if it is an empty string, ensuring correct and expected behavior. </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 69831bc62ea4d80cdcd42cef2aa9bd8eda28ae8c

diegomarquezp added 3 commits October 21, 2024 16:07

add test to confirm tracer attempts failed

6635019

simplify test for retries

944e8d6

Merge remote-tracking branch 'origin/main' into fix-httpjson-retry-me…

1edf557

…trics

product-auto-label bot added the size: s Pull request size is small. label Oct 21, 2024

extend tracer tests to every test

4db1d2b

product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Oct 21, 2024

diegomarquezp added 2 commits October 21, 2024 16:59

fix test vars initialization

9ab23d5

add retrial exhaustion check

6364aab

lqiu96 reviewed Oct 22, 2024

View reviewed changes

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java Outdated Show resolved Hide resolved

diegomarquezp added 3 commits October 22, 2024 19:48

Merge branch 'main' into fix-httpjson-retry-metrics

ae1eb44

restore ApiException expected in test

1ebc3a2

Merge branch 'fix-httpjson-retry-metrics' of https://github.com/googl…

25e778f

…eapis/sdk-platform-java into fix-httpjson-retry-metrics

lqiu96 reviewed Oct 24, 2024

View reviewed changes

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java Outdated Show resolved Hide resolved

enable Disabled ITs

b363a29

lqiu96 reviewed Oct 24, 2024

View reviewed changes

try fixing multipleattempts test

5c11e10

diegomarquezp commented Oct 24, 2024

View reviewed changes

showcase/gapic-showcase/src/test/java/com/google/showcase/v1beta1/it/ITOtelMetrics.java Show resolved Hide resolved

diegomarquezp closed this Oct 24, 2024

diegomarquezp reopened this Oct 24, 2024

diegomarquezp added 3 commits October 24, 2024 18:10

Merge branch 'main' into fix-httpjson-retry-metrics

9b48dbe

fix multiple attempts httpjson test (thanks @lqiu96)

c8e8a7d

Merge remote-tracking branch 'refs/remotes/origin/fix-httpjson-retry-…

af3adb5

…metrics' into fix-httpjson-retry-metrics

diegomarquezp marked this pull request as ready for review October 25, 2024 23:32

diegomarquezp added 5 commits October 25, 2024 19:50

extract tracer factory to its own method

035be17

add comments

8e2a58a

Merge branch 'fix-httpjson-retry-metrics' of https://github.com/googl…

8f3331e

…eapis/sdk-platform-java into fix-httpjson-retry-metrics

Merge remote-tracking branch 'origin/main' into fix-httpjson-retry-me…

f647320

…trics

licence

2df2e9f

lqiu96 reviewed Oct 28, 2024

View reviewed changes

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java Outdated Show resolved Hide resolved

lqiu96 reviewed Oct 28, 2024

View reviewed changes

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java Outdated Show resolved Hide resolved

lqiu96 reviewed Oct 28, 2024

View reviewed changes

gax-java/gax-httpjson/src/test/java/com/google/api/gax/httpjson/RetryingTest.java Outdated Show resolved Hide resolved

lqiu96 reviewed Oct 28, 2024

View reviewed changes

formalize tests

e7bc1c3

blakeli0 reviewed Oct 30, 2024

View reviewed changes

diegomarquezp added 7 commits October 31, 2024 17:25

rename vars, extend ApiTracer

e2385ad

move initialization to contained class

58e874a

fix order of methods

87f807a

remove retries exhausted

e57b4db

Revert "remove retries exhausted"

58a0daa

This reverts commit e57b4db.

record attempt failed when retries are exhausted

325eba4

adjust retries count in retryingtest

2120960

diegomarquezp requested review from blakeli0 and lqiu96 November 5, 2024 23:19

diegomarquezp added 3 commits November 11, 2024 10:53

Merge branch 'main' into fix-httpjson-retry-metrics

0630fdf

empty commit to retry tests

9697e8a

remove flaky tests

065b01f

blakeli0 approved these changes Nov 12, 2024

View reviewed changes

lqiu96 approved these changes Nov 13, 2024

View reviewed changes

diegomarquezp merged commit 15a64ee into main Nov 13, 2024
51 of 55 checks passed

diegomarquezp deleted the fix-httpjson-retry-metrics branch November 13, 2024 23:29

release-please bot mentioned this pull request Nov 13, 2024

chore(main): release 2.50.0 #3327

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: httpjson callables to trace attempts (started, failed) #3300

fix: httpjson callables to trace attempts (started, failed) #3300

diegomarquezp commented Oct 21, 2024 •

edited

Loading

lqiu96 commented Oct 22, 2024

lqiu96 left a comment •

edited

Loading

lqiu96 Oct 28, 2024

diegomarquezp Oct 30, 2024

diegomarquezp Oct 30, 2024

diegomarquezp Oct 30, 2024

lqiu96 left a comment

blakeli0 Oct 30, 2024

diegomarquezp Nov 4, 2024

blakeli0 Oct 30, 2024

diegomarquezp Nov 4, 2024

blakeli0 Oct 30, 2024

diegomarquezp Oct 31, 2024

diegomarquezp Oct 31, 2024

diegomarquezp Nov 4, 2024

blakeli0 Oct 30, 2024

diegomarquezp Nov 4, 2024

blakeli0 Oct 30, 2024

diegomarquezp Nov 4, 2024

diegomarquezp Nov 4, 2024

blakeli0 Oct 30, 2024

diegomarquezp Nov 4, 2024

lqiu96 commented Nov 6, 2024

sonarcloud bot commented Nov 11, 2024

sonarcloud bot commented Nov 11, 2024

blakeli0 left a comment

diegomarquezp commented Nov 13, 2024

		// the number of attempts varies. Here we just make sure that all of them except the last are
		// considered as failed

	clientContext =
	ClientContext.newBuilder()
	// we use a custom tracer to confirm whether the retrials are being recorded.
	.setTracerFactory(tracerFactory)
	.setExecutor(executor)
	.setClock(fakeClock)
	.setDefaultCallContext(HttpJsonCallContext.createDefault())
	.build();

fix: httpjson callables to trace attempts (started, failed) #3300

fix: httpjson callables to trace attempts (started, failed) #3300

Conversation

diegomarquezp commented Oct 21, 2024 • edited Loading

Approach

Confirmation that it works

lqiu96 commented Oct 22, 2024

lqiu96 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lqiu96 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lqiu96 commented Nov 6, 2024

sonarcloud bot commented Nov 11, 2024

Quality Gate passed for 'gapic-generator-java-root'

sonarcloud bot commented Nov 11, 2024

Quality Gate passed for 'java_showcase_integration_tests'

blakeli0 left a comment

Choose a reason for hiding this comment

diegomarquezp commented Nov 13, 2024

diegomarquezp commented Oct 21, 2024 •

edited

Loading

lqiu96 left a comment •

edited

Loading