Skip to content

Commit

Permalink
feat: Introducing Tracing with OpenTelemetry API #1537 (#1576)
Browse files Browse the repository at this point in the history
* feat: Adding TraceUtil interface and its implementation to enable Tracing controls via DatastoreOptions (#1431)

* Adding EnabledTraceUtil, DisabledTraceUtil and TraceUtilTest
* Annotating DatastoreOpenTelemetryOptions to be transient as they're not serializable
* Adding google-auth-library-credentials dependency due to https://github.com/googleapis/java-datastore/actions/runs/8944472794/job/24571458116?pr=1431

* feat: Adding Lookup RPC OpenTelemetry Tracing (#1437)

* feat: Adding Lookup RPC OpenTelemetry Tracing
- Removed OpenCensus Tracing
- Added E2E tests with Global and Local OTel SDK
- Moved OTel SDK setup to RemoteDatastoreHelper
- Fixed pom to depend on BOM for all shared dependencies

* feat: Adding Commit RPC Trace Instrumentation (#1440)

- Added end-to-end test for Datastore operationsput, add, update and delete.
- Updated E2E Test to use the namespace correctly for efficient clean-up of test data

* feat: RunQuery trace instrumentation (#1441)

* feat: RunQuery trace instrumentation

* feat: RunAggregationQuery instrumentation (#1447)

* feat: RunQuery trace instrumentation

* Formatting

* Formatting

* Refactor: s/RUNQUERY/RUN_QUERY

* feat: RunAggregationQuery Trace Instrumentation

* Build: retiring test assertions for OpenCensus spans - will be replacing this in hermetic integration tests for OpenTelemetry using in-memory span exports (in addition to ITE2ETraceTest.java).

* Formatting

* Fixing @test annotation missed after merge

* Formatting

* feat: RunQuery trace instrumentation

* Formatting

* Formatting

* Refactor: s/RUNQUERY/RUN_QUERY

* feat: RunAggregationQuery Trace Instrumentation

* Build: retiring test assertions for OpenCensus spans - will be replacing this in hermetic integration tests for OpenTelemetry using in-memory span exports (in addition to ITE2ETraceTest.java).

* Formatting

* Fixing @test annotation missed after merge

* Formatting

* feat: Add Transaction tracing
test: transactionalLookupTest

* test: Transaction test for RunInTransaction - need to fix trace instrumentation for RunIn..

* Adding transaction span names

* TransactionLookupTest

* feat: support for transactional operations
- tested using newTransaction() and runInTransaction()

* Revert "feat: support for transactional operations"

This reverts commit 10341c0.

* feat: support for transactional operations (#1468)

* feat: support for transactional operations
- tested using newTransaction() and runInTransaction()

* feat: Allocateid tracing (#1488)

* feat: Adding tracing for AllocateIds RPC

* formatting

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: Add tracing for ReserveIds operation (#1490)

- added end-to-end test

* fix: Fixed Span nesting for `ReadWriteTransactionCallable` by using parent SpanContext instead of just parent Context (#1495)

* fix: Fixed the TraceUtil.startSpan method to use `SpanContext` for linking with the parent instead of `Context`.
- This fixes the hierarchy of Spans appearing in a transaction under a Run method.
- Tested using existing transaction test

* Fixed commit reordering and typos

* fix: lint errors

* fix: Refactored the ReadWriteTransactioncallable.call method to use startSpan idiomatically
- TraceUtil.startSpan needs more debugging
- return DefaultTracerProvider instance (no-op) when initializing DisabledTraceUtil - this fixes the unit tests in DatastoreTest.testRunInTransactionWithReadWriteOption

* feat: Added tracing for Transaction.RunQuery (#1499)

* feat: Added span for Transactional RunQuery
- tested

* fix: lint

* fix: patch apply issues

* fix: refactor using boolean flag

* fix: s/startSpan/startSpanWithParentContext

* test: Additional Transaction Testing and cleanup OpenCensus usage (#1505)

* test: newTransactionReadWriteTraceTest

* fix: test literal

* feat: Added tests for transaction cases

* fix: Cleanup OpenCensus dead code

* fix: updating version from 2.20.1 -> 2.21.0

* fix: reverting version from 2.21.0 -> 2.20.1

* fix: Adding an exception to the clirr-maven-plugin for an internal API parameter change from com.google.cloud.datastore.TraceUtil -> com.google.cloud.datastore.telemetry.TraceUtil

* fix: Fixing the differenceType in clirr exception

* fix: add an exception for removal of an internal class (com.google.cloud.datastore.TraceUtil)

* fix: fixing incomplete difference details for type 7005

* fix: Fixing `to` of the difference to be the entire signature

* fix: typo

* test: Adding ITTracingTest to verify events and span attributes (whic… (#1514)

* test: Adding ITTracingTest to verify events and span attributes (which are not verified in ITE2ETracingTest) due to TraceClient API limitations.
- This test uses InMemorySpanExporter to read the generated Otel span data by the test process to verify generated span data as it were before exporting to a backend. None of the span data is exported to a durable backend.
- This test is still an E2E test as it requires a project to send RPCs to.

* fix: fixing compilation error due to missing pom dependency.

* test: Test for AllocateId and ReserveId rpcs

* test: Commit/Put/Update/Delete tests

* test: Added fixes and test for RunQuery event

* test: Additional Transaction tests and AggregationQuery test (#1518)

* test: ReadWrite Transaction test

* test: Added test for Transactional RunQuery and Transaction Rollback

* test: runInTransaction API tracing test
- Fixed setting of common span attributes to spans in runInTransaction
- Removed some gRPC related channel attributes that are not present in this Datastore version, yet.

* fix: Undelete gRPC upgrade docs

* fix: Undo merge mistakes

* fix: Updating span event strings (#1539)

* fix: Fixing user-facing span names in line with go/firestore-client-trace-catalog

* fix: updating bom dependency version to fix https://github.com/googleapis/java-datastore/actions/runs/10256441634/job/28375496112?pr=1539

* Fix: typo in test causing integration test failure (#1556)

https://btx.cloud.google.com/invocations/c11a2e8b-4494-4ddc-a77e-cf2bcbcf5254/targets/cloud-devrel%2Fclient-libraries%2Fjava%2Fjava-datastore%2Fpresubmit%2Fintegration;config=default/log

* fix: opentelemetry-sdk should only be used as a Test Dependency

* fix: Update opentelemetry.version - this also fixes the tests failing in https://github.com/googleapis/java-datastore/actions/runs/10891578591/job/30222786908

* fix: Replacing attribute key values w/ constants

* fix: opentelemetry.version to fix RequireUpperBoundDeps check https://github.com/googleapis/java-datastore/actions/runs/10892403348/job/30225154043?pr=1576

* fix: Create Span hierarchy using parent Span (#1580)

* fix: Replace use of TraceUtil.SpanContext w/ TraceUtil.Context

* fix: Fixing how span hierarchy is created across threads - using Span instead of Context

* fix: cleaning up startSpan(spanName, parentContext) variant

* fix: add TracedReadWriteTransactionCallable to bifurcate tracing enabled/disabled paths for the Transaction callback.
- This change implements the idiomatic way to express nested spans as described in https://opentelemetry.io/docs/languages/java/instrumentation/#create-nested-spans

* fix: cleanup

* fix: cleanup

* fix: cleanup

* fix: formatting and import refactoring

* chore: generate libraries at Thu Sep 19 18:35:54 UTC 2024

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: cloud-java-bot <cloud-java-bot@google.com>
  • Loading branch information
3 people authored Sep 19, 2024
1 parent 224b137 commit 5440c22
Show file tree
Hide file tree
Showing 22 changed files with 3,216 additions and 256 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ If you are using Maven without the BOM, add this to your dependencies:
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-datastore</artifactId>
<version>2.21.2</version>
<version>2.21.3</version>
</dependency>

```
Expand Down
12 changes: 12 additions & 0 deletions google-cloud-datastore/clirr-ignored-differences.xml
Original file line number Diff line number Diff line change
Expand Up @@ -55,4 +55,16 @@
<method>java.lang.Object execute(com.google.cloud.datastore.Query, com.google.cloud.datastore.ReadOption[])</method>
<differenceType>7004</differenceType>
</difference>
<difference>
<className>com/google/cloud/datastore/RetryAndTraceDatastoreRpcDecorator</className>
<method>RetryAndTraceDatastoreRpcDecorator(com.google.cloud.datastore.spi.v1.DatastoreRpc, com.google.cloud.datastore.TraceUtil, com.google.api.gax.retrying.RetrySettings, com.google.cloud.datastore.DatastoreOptions)</method>
<to>RetryAndTraceDatastoreRpcDecorator(com.google.cloud.datastore.spi.v1.DatastoreRpc, com.google.cloud.datastore.telemetry.TraceUtil, com.google.api.gax.retrying.RetrySettings, com.google.cloud.datastore.DatastoreOptions)</to>
<differenceType>7005</differenceType>
</difference>

<!-- Class removed -->
<difference>
<className>com/google/cloud/datastore/TraceUtil</className>
<differenceType>8001</differenceType>
</difference>
</differences>
86 changes: 85 additions & 1 deletion google-cloud-datastore/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,20 @@
</parent>
<properties>
<site.installationModule>google-cloud-datastore</site.installationModule>
<opentelemetry.version>1.42.0</opentelemetry.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<!-- Artifacts from google-cloud-java monorepo -->
<groupId>com.google.cloud</groupId>
<artifactId>gapic-libraries-bom</artifactId>
<version>1.37.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.api.grpc</groupId>
Expand All @@ -38,6 +51,10 @@
<groupId>com.google.cloud.datastore</groupId>
<artifactId>datastore-v1-proto-client</artifactId>
</dependency>
<dependency>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-credentials</artifactId>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-api</artifactId>
Expand Down Expand Up @@ -111,14 +128,26 @@
<artifactId>jsr305</artifactId>
</dependency>

<!-- OpenTelemetry -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-api</artifactId>
<version>${opentelemetry.version}</version>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-context</artifactId>
<version>${opentelemetry.version}</version>
</dependency>
<!-- END OpenTelemetry -->

<!-- Test dependencies -->
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>google-cloud-core</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava-testlib</artifactId>
Expand Down Expand Up @@ -160,6 +189,61 @@
<version>1.4.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.testparameterinjector</groupId>
<artifactId>test-parameter-injector</artifactId>
<version>1.16</version>
<scope>test</scope>
</dependency>
<!-- OpenTelemetry -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk</artifactId>
<version>${opentelemetry.version}</version>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-common</artifactId>
<version>${opentelemetry.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-testing</artifactId>
<version>${opentelemetry.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-trace</artifactId>
<version>${opentelemetry.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-semconv</artifactId>
<version>1.1.0-alpha</version>
<scope>test</scope>
</dependency>
<!-- END OpenTelemetry -->
<!-- Cloud Ops -->
<dependency>
<groupId>com.google.cloud.opentelemetry</groupId>
<artifactId>exporter-trace</artifactId>
<version>0.15.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-trace</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.api.grpc</groupId>
<artifactId>proto-google-cloud-trace-v1</artifactId>
<scope>test</scope>
</dependency>
<!-- END Cloud Ops -->
</dependencies>

<build>
Expand Down
Loading

0 comments on commit 5440c22

Please sign in to comment.