Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add OpenTelemetry tracing #1568

Merged
merged 32 commits into from
Jun 27, 2024
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
921f14e
feat: add OpenTelemetry tracing
olavloite Apr 18, 2024
0e2f270
Merge branch 'main' into open-telemetry-tracing
olavloite May 1, 2024
f8a03d6
chore: add copyright headers
olavloite May 1, 2024
b4b191a
chore: add deps + reset tracing for test
olavloite May 1, 2024
7ca00f3
fix: ClassCastException in Spring Data JDBC sample
olavloite May 1, 2024
1572c2c
Merge branch 'fix-class-cast-exception' into open-telemetry-tracing
olavloite May 1, 2024
4c4b9ce
docs: add sample for OpenTelemetry
olavloite May 2, 2024
27f3faa
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 2, 2024
03fb3ed
Merge branch 'main' into open-telemetry-tracing
olavloite May 29, 2024
c1318a6
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 29, 2024
10e1694
docs: update sample for OpenTelemetry
olavloite May 29, 2024
666d97f
Merge branch 'open-telemetry-tracing' of github.com:googleapis/java-s…
olavloite May 29, 2024
268ea3c
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 29, 2024
ef39faf
test: check for credentials before setting up tracing
olavloite May 29, 2024
0a544a8
Merge branch 'open-telemetry-tracing' of github.com:googleapis/java-s…
olavloite May 29, 2024
2f7dbc6
chore: cleanup
olavloite May 30, 2024
a672228
chore: add enable flag for otel to sample app
olavloite May 30, 2024
fc5cc98
chore: add clirr diff
olavloite May 30, 2024
95d331a
Merge branch 'main' into open-telemetry-tracing
olavloite Jun 7, 2024
69be4a7
fix: add tracing prefix + more samples
olavloite Jun 7, 2024
43a3e26
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jun 7, 2024
e1b75b3
chore: remove whitespace
olavloite Jun 20, 2024
97dc64c
Merge branch 'open-telemetry-tracing' of github.com:googleapis/java-s…
olavloite Jun 20, 2024
6fa2066
Merge branch 'main' into open-telemetry-tracing
olavloite Jun 20, 2024
049996d
docs: update sample to use non-snapshot versions
olavloite Jun 20, 2024
341e6ef
docs: update sample + add debugging guide
olavloite Jun 21, 2024
c74e4b7
chore: cleanup
olavloite Jun 21, 2024
29dafa3
chore: actually use properties in example
olavloite Jun 21, 2024
5f03633
docs: add link to tag sample
olavloite Jun 24, 2024
44c60d8
docs: improve documentation based on review comments
olavloite Jun 26, 2024
75aca55
Merge branch 'main' into open-telemetry-tracing
olavloite Jun 27, 2024
a454492
docs: clearify the names of the Spanner generated traces
olavloite Jun 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 103 additions & 1 deletion .readme-partials.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ custom_content: |

### Connection URL Properties

The Cloud Spanner JDBC driver supports the following connection URL properties. Note that all of
The Spanner JDBC driver supports the following connection URL properties. Note that all of
these can also be supplied in a Properties instance that is passed to the
`DriverManager#getConnection(String url, Properties properties)` method.

Expand All @@ -37,6 +37,8 @@ custom_content: |
- autoConfigEmulator (boolean): Automatically configure the connection to try to connect to the Cloud Spanner emulator. You do not need to specify any host or port in the connection string as long as the emulator is running on the default host/port (localhost:9010). The instance and database in the connection string will automatically be created if these do not yet exist on the emulator. This means that you do not need to execute any `gcloud` commands on the emulator to create the instance and database before you can connect to it. Example: `jdbc:cloudspanner:/projects/test-project/instances/test-instance/databases/test-db;autoConfigEmulator=true`
- usePlainText (boolean): Sets whether the JDBC connection should establish an unencrypted connection to a (local) server. This option can only be used when connecting to a local emulator that does not require an encrypted connection, and that does not require authentication. Example: `jdbc:cloudspanner://localhost:9010/projects/test-project/instances/test-instance/databases/test-db;usePlainText=true`
- optimizerVersion (String): Sets the default query optimizer version to use for this connection. See also https://cloud.google.com/spanner/docs/query-optimizer/query-optimizer-versions.
- enableExtendedTracing (boolean): Enables extended OpenTelemetry tracing of queries that are executed by a JDBC connection. When enabled, the SQL string of the query that is executed is added as a property to the trace.
- enableApiTracing (boolean): Enables API OpenTelemetry tracing of all RPCs that are executed by the JDBC driver. When enabled, a trace will be created for each RPC invocation that is executed by the JDBC driver. Enable this to investigate latency problems and/or RPCs that are being retried.

#### Advanced Properties
- minSessions (int): Sets the minimum number of sessions in the backing session pool. Defaults to 100.
Expand All @@ -46,6 +48,106 @@ custom_content: |
- oauthToken (string): A valid pre-existing OAuth token to use for authentication for this connection. Setting this property will take precedence over any value set for a credentials file.
- lenient (boolean): Enable this to force the JDBC driver to ignore unknown properties in the connection URL. Some applications automatically add additional properties to the URL that are not recognized by the JDBC driver. Normally, the JDBC driver will reject this, unless `lenient` mode is enabled.

### OpenTelemetry Tracing
The Spanner JDBC driver supports OpenTelemetry tracing. You can configure the OpenTelemetry instance
that should be used in two ways:
1. Register a global OpenTelemetry instance. This instance will automatically be picked up by the Spanner JDBC driver.
2. Add an OpenTelemetry instance with the key `openTelemetry` to the `java.util.Properties` instance that is used to create the JDBC connection.

By default, the traces that are generated by the Spanner JDBC driver do not include the SQL
statement. You can include the SQL statement with the traces by adding the property `enableExtendedTracing=true`
to the JDBC connection URL.

#### Example Using Global OpenTelemetry
Create and register a global OpenTelemetry object before creating a JDBC connection.
See also the [Spring Data JDBC Sample](samples/spring-data-jdbc) for an example for how to
configure OpenTelemetry in combination with Spring Data.

See [Latency Debugging Guide](documentation/latency-debugging-guide.md) for more information on how to use these traces.

```java
TraceConfiguration traceConfiguration = TraceConfiguration.builder().setProjectId("my-project").build();
SpanExporter traceExporter = TraceExporter.createWithConfiguration(traceConfiguration);
OpenTelemetry openTelemetry =
OpenTelemetrySdk.builder()
.setTracerProvider(
SdkTracerProvider.builder()
.setSampler(Sampler.traceIdRatioBased(0.05))
.setResource(
Resource.builder()
.put("service.name", "my-unique-service-name")
.build())
.addSpanProcessor(BatchSpanProcessor.builder(traceExporter).build())
.build())
.buildAndRegisterGlobal();
String projectId = "my-project";
String instanceId = "my-instance";
String databaseId = "my-database";
// Setting this to true instructs the JDBC driver to include the SQL statement with the traces.
boolean enableExtendedTracing = true;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also enableApiTracing in the code snipper

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added (although we need to wait for googleapis/java-spanner#3166 to be merged before we can use it here)


try (Connection connection =
DriverManager.getConnection(
String.format(
"jdbc:cloudspanner:/projects/%s/instances/%s/databases/%s?enableExtendedTracing=%s",
projectId, instanceId, databaseId, enableExtendedTracing))) {
try (Statement statement = connection.createStatement()) {
try (ResultSet rs = statement.executeQuery("SELECT CURRENT_TIMESTAMP()")) {
while (rs.next()) {
System.out.printf(
"Connected to Cloud Spanner at [%s]%n", rs.getTimestamp(1).toString());
}
}
}
}
```

#### Example Using an OpenTelemetry instance in Properties
Create and register a global OpenTelemetry object before creating a JDBC connection.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not registering global OT object here.

nit: I feel we can simplify this example by not repeating the whole code .

We can create OpenTelemetrySdkBuilder object and then show different usage of it.

for eg.

 OpenTelemetrySdkBuilder openTelemetrySdkBuilder =
        OpenTelemetrySdk.builder()
            .setTracerProvider(
                SdkTracerProvider.builder()
                    .setSampler(Sampler.traceIdRatioBased(0.05))
                    .setResource(
                        Resource.builder()
                            .put("service.name", "my-unique-service-name")
                            .build())
                    .addSpanProcessor(BatchSpanProcessor.builder(traceExporter).build())
                    .build());
 // Example Using Global OpenTelemetry
 openTelemetrySdkBuilder.buildAndRegisterGlobal()
// Example Using an OpenTelemetry instance in Properties 
     OpenTelemetry openTelemetry = openTelemetrySdkBuilder.build()

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for noting. Removed the text regarding registering a global OTEL object.

I also simplified the second example a bit, assuming that readers will understand that they can use the same OpenTelemetry instance as in the example above.

See also the [Spring Data JDBC Sample](samples/spring-data-jdbc) for an example for how to
configure OpenTelemetry in combination with Spring Data.

```java
TraceConfiguration traceConfiguration = TraceConfiguration.builder().setProjectId("my-project").build();
SpanExporter traceExporter = TraceExporter.createWithConfiguration(traceConfiguration);
OpenTelemetry openTelemetry =
OpenTelemetrySdk.builder()
.setTracerProvider(
SdkTracerProvider.builder()
.setSampler(Sampler.traceIdRatioBased(0.05))
.setResource(
Resource.builder()
.put("service.name", "my-unique-service-name")
.build())
.addSpanProcessor(BatchSpanProcessor.builder(traceExporter).build())
.build())
.build();
String projectId = "my-project";
String instanceId = "my-instance";
String databaseId = "my-database";
// Setting this to true instructs the JDBC driver to include the SQL statement with the traces.
boolean enableExtendedTracing = true;

Properties info = new Properties();
info.put(JdbcDriver.OPEN_TELEMETRY_PROPERTY_KEY, openTelemetry);
info.put("enableExtendedTracing", String.valueOf(enableExtendedTracing));

try (Connection connection =
DriverManager.getConnection(
String.format(
"jdbc:cloudspanner:/projects/%s/instances/%s/databases/%s",
projectId, instanceId, databaseId))) {
try (Statement statement = connection.createStatement()) {
try (ResultSet rs = statement.executeQuery("SELECT CURRENT_TIMESTAMP()")) {
while (rs.next()) {
System.out.printf(
"Connected to Cloud Spanner at [%s]%n", rs.getTimestamp(1).toString());
}
}
}
}
```

### Jar with Dependencies
A single jar with all dependencies can be downloaded from https://repo1.maven.org/maven2/com/google/cloud/google-cloud-spanner-jdbc/latest
or be built with the command `mvn package` (select the jar that is named `google-cloud-spanner-jdbc-<version>-single-jar-with-dependencies.jar`).
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added documentation/img/example-api-tracing.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added documentation/img/example-gfe-latency.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added documentation/img/example-search-for-tag.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added documentation/img/example-tracing.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading