Extensions example (open-telemetry#3071)

* Simplify extensions example project

* Add external dependency and fat jar support

* spotless

* Apply suggestions from code review

Co-authored-by: Mateusz Rzeszutek <mrzeszutek@splunk.com>

* Apply suggestions from code review

Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>

Co-authored-by: Mateusz Rzeszutek <mrzeszutek@splunk.com>
Co-authored-by: Trask Stalnaker <trask.stalnaker@gmail.com>

examples/extension/README.md

@@ -0,0 +1,75 @@
+## Introduction
+
+This repository demonstrates how to create an extension archive to use with `otel.javaagent.experimental.extensions`
+configuration option of the OpenTelemetry Java instrumentation agent.
+
+For every extension point provided by OpenTelemetry Java instrumentation, this repository contains an example of
+its usage.
+
+Please carefully read both the source code and Gradle build script file `build.gradle`.
+They contain a lot of documentation and comments explaining the purpose of all major pieces.
+
+## How to use extension archive
+
+When you build this project by running `./gradlew build` you will get a jar file in
+`build/libs/opentelemetry-java-instrumentation-extension-demo-1.0-all.jar`.
+Copy this jar file to a machine running the application that you are monitoring with
+OpenTelemetry Java instrumentation agent.
+
+Assuming that your command line looks similar to this:
+```
+java -javaagent:path/to/opentelemetry-javaagent-all.jar \
+     -jar myapp.jar
+```
+change it to this:
+```
+java -javaagent:path/to/opentelemetry-javaagent-all.jar \
+     -Dotel.javaagent.experimental.extensions=path/to/extension.jar
+     -jar myapp.jar
+```
+specifying the full path and the correct name of your extensions jar.
+
+## Extensions examples
+
+* [DemoIdGenerator](src/main/java/com/example/javaagent/DemoIdGenerator.java) - custom `IdGenerator`
+* [DemoPropagator](src/main/java/com/example/javaagent/DemoPropagator.java) - custom `TextMapPropagator`
+* [DemoPropertySource](src/main/java/com/example/javaagent/DemoPropertySource.java) - default configuration
+* [DemoSampler](src/main/java/com/example/javaagent/DemoSampler.java) - custom `Sampler`
+* [DemoSpanProcessor](src/main/java/com/example/javaagent/DemoSpanProcessor.java) - custom `SpanProcessor`
+* [DemoSpanExporter](src/main/java/com/example/javaagent/DemoSpanExporter.java) - custom `SpanExporter`
+* [DemoServlet3InstrumentationModule](src/main/java/com/example/javaagent/instrumentation/DemoServlet3InstrumentationModule.java) - additional instrumentation
+
+## Instrumentation customisation
+
+There are several options to override or customise instrumentation provided by the upstream agent.
+The following description follows one specific use-case:
+
+> Instrumentation X from Otel distribution creates span that I don't like and I want to change it.
+
+As an example, let us take some database client instrumentation that creates a span for database call
+and extracts data from db connection to provide attributes for that span.
+
+### I don't want this span at all
+The easiest case. You can just pre-configure the agent in your extension and disable given instrumentation.
+
+### I want to add/modify some attributes and their values does NOT depend on a specific db connection instance.
+E.g. you want to add some data from call stack as span attribute. 
+In this case just provide your custom `SpanProcessor`.
+No need for touching instrumentation itself.
+
+### I want to add/modify some attributes and their values depend on a specific db connection instance.
+Write a _new_ instrumentation which injects its own advice into the same method as the original one.
+Use `order` method to ensure it is run after the original instrumentation.
+Now you can augment current span with new information.
+
+See [DemoServlet3InstrumentationModule](instrumentation/servlet-3/src/main/java/com/example/javaagent/instrumentation/DemoServlet3InstrumentationModule.java).
+
+### I want to remove some attributes
+Write custom exporter or use attribute filtering functionality in Collector.
+
+### I don't like Otel span at all. I want to significantly modify it and its lifecycle
+Disable existing instrumentation.
+Write a new one, which injects `Advice` into the same (or better) method as the original instrumentation.
+Write your own `Advice` for this.
+Use existing `Tracer` directly or extend it.
+As you have your own `Advice`, you can control which `Tracer` you use.

examples/extension/build.gradle

@@ -1,52 +1,114 @@
+plugins {
+  id "java"
+
+  /*
+  Instrumentation agent extension mechanism expects a single jar containing everything required
+  for your extension. This also includes any external libraries that your extension uses and
+  cannot access from application classpath (see comment below about `javax.servlet-api` dependency).
+
+  Thus we use Shadow Gradle plugin to package our classes and all required runtime dependencies
+  into a single jar.
+  See https://imperceptiblethoughts.com/shadow/ for more details about Shadow plugin.
+   */
+  id "com.github.johnrengelman.shadow" version "6.1.0"
+}
+
 group 'io.opentelemetry.example'
-version '1.0-SNAPSHOT'
-
-subprojects {
-  version = rootProject.version
-
-  apply plugin: "java"
-
-  ext {
-    versions = [
-      opentelemetry         : "1.2.0",
-      opentelemetryJavaagent: "1.3.0-SNAPSHOT",
-      bytebuddy             : "1.10.18",
-      guava                 : "30.1-jre"
-    ]
-    versions.opentelemetryAlpha = "${versions.opentelemetry}-alpha"
-    versions.opentelemetryJavaagentAlpha = "1.3.0-alpha-SNAPSHOT"
-
-    deps = [
-      bytebuddy           : dependencies.create(group: 'net.bytebuddy', name: 'byte-buddy', version: versions.bytebuddy),
-      bytebuddyagent      : dependencies.create(group: 'net.bytebuddy', name: 'byte-buddy-agent', version: versions.bytebuddy),
-      autoservice         : [
-        dependencies.create(group: 'com.google.auto.service', name: 'auto-service', version: '1.0-rc7'),
-        dependencies.create(group: 'com.google.auto', name: 'auto-common', version: '0.8'),
-        dependencies.create(group: 'com.google.guava', name: 'guava', version: "${versions.guava}"),
-      ],
-      autoValueAnnotations: "com.google.auto.value:auto-value-annotations:${versions.autoValue}",
-    ]
-  }
+version '1.0'
 
-  repositories {
-    // needed because relying on locally built SNAPSHOT versions above for now
-    mavenLocal()
-    mavenCentral()
-  }
+ext {
+  versions = [
+    opentelemetry              : "1.2.0",
+    opentelemetryAlpha         : "1.2.0-alpha",
+    opentelemetryJavaagent     : "1.3.0-SNAPSHOT",
+    opentelemetryJavaagentAlpha: "1.3.0-alpha-SNAPSHOT",
+  ]
 
-  dependencies {
-    testImplementation("org.mockito:mockito-core:3.3.3")
-    testImplementation("org.junit.jupiter:junit-jupiter-api:5.6.2")
-    testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:5.6.2")
+  deps = [
+    autoservice: dependencies.create(group: 'com.google.auto.service', name: 'auto-service', version: '1.0')
+  ]
+}
+
+repositories {
+  mavenCentral()
+  maven {
+    url = uri("https://oss.sonatype.org/content/repositories/snapshots")
   }
+}
 
-  tasks {
-    test {
-      useJUnitPlatform()
-    }
+configurations {
+  /*
+  We create a separate gradle configuration to grab a published Otel instrumentation agent.
+  We don't need the agent during development of this extension module.
+  This agent is used only during integration test.
+  */
+  otel
+}
+
+dependencies {
+  /*
+  Interfaces and SPIs that we implement. We use `compileOnly` dependency because during
+  runtime all neccessary classes are provided by javaagent itself.
+   */
+  compileOnly("io.opentelemetry:opentelemetry-sdk-extension-autoconfigure:${versions.opentelemetryAlpha}")
+  compileOnly("io.opentelemetry.javaagent:opentelemetry-javaagent-api:${versions.opentelemetryJavaagentAlpha}")
+  compileOnly("io.opentelemetry.javaagent:opentelemetry-javaagent-extension-api:${versions.opentelemetryJavaagentAlpha}")
+
+  //Provides @AutoService annotation that makes registration of our SPI implementations much easier
+  compileOnly deps.autoservice
+  annotationProcessor deps.autoservice
 
-    compileJava {
-      options.release.set(11)
+  /*
+   Used by our demo instrumentation module to reference classes of the target instrumented library.
+   We again use `compileOnly` here because during runtime these classes are provided by the
+   actual application that we instrument.
+
+   NB! Only Advice (and "helper") classes of instrumentation modules can access classes from application classpath.
+   See https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/contributing/writing-instrumentation-module.md#advice-classes
+   */
+  compileOnly group: 'javax.servlet', name: 'javax.servlet-api', version: '3.0.1'
+
+  /*
+  This dependency is required for DemoSpanProcessor both during compile and runtime.
+  Only dependencies added to `implementation` configuration will be picked up by Shadow plugin
+  and added to the resulting jar for our extension's distribution.
+   */
+  implementation 'org.apache.commons:commons-lang3:3.11'
+
+  //All dependencies below are only for tests
+  testImplementation("org.testcontainers:testcontainers:1.15.2")
+  testImplementation("com.fasterxml.jackson.core:jackson-databind:2.11.2")
+  testImplementation("com.google.protobuf:protobuf-java-util:3.12.4")
+  testImplementation("com.squareup.okhttp3:okhttp:3.12.12")
+  testImplementation("io.opentelemetry:opentelemetry-api:${versions.opentelemetry}")
+  testImplementation("io.opentelemetry:opentelemetry-proto:${versions.opentelemetryAlpha}")
+
+  testImplementation("org.junit.jupiter:junit-jupiter-api:5.6.2")
+  testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:5.6.2")
+  testRuntimeOnly("ch.qos.logback:logback-classic:1.2.3")
+
+  //Otel Java instrumentation that we use and extend during integration tests
+  otel("io.opentelemetry.javaagent:opentelemetry-javaagent:${versions.opentelemetryJavaagent}:all")
+}
+
+tasks {
+  test {
+    useJUnitPlatform()
+
+    def extensionJar = tasks.shadowJar
+    inputs.files(layout.files(extensionJar))
+
+    doFirst {
+      //To run our tests with the javaagent published by OpenTelemetry Java instrumentation project
+      jvmArgs("-Dio.opentelemetry.smoketest.agentPath=${configurations.getByName("otel").resolve().find().absolutePath}")
+      //Instructs our integration test where to find our extension archive
+      jvmArgs("-Dio.opentelemetry.smoketest.extensionPath=${extensionJar.archiveFile.get()}")
     }
   }
+
+  compileJava {
+    options.release.set(11)
+  }
+
+  assemble.dependsOn(shadowJar)
 }

examples/extension/settings.gradle

@@ -1,5 +1 @@
 rootProject.name = 'opentelemetry-java-instrumentation-extension-demo'
-
-include "custom"
-include "smoke-tests"
-

examples/extension/custom/src/main/java/com/example/javaagent/DemoSpanProcessor.java → examples/extension/src/main/java/com/example/javaagent/DemoSpanProcessor.java

@@ -5,6 +5,7 @@
 import io.opentelemetry.sdk.trace.ReadWriteSpan;
 import io.opentelemetry.sdk.trace.ReadableSpan;
 import io.opentelemetry.sdk.trace.SpanProcessor;
+import org.apache.commons.lang3.RandomStringUtils;
 
 /**
  * See <a href="https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/sdk.md#span-processor">
@@ -13,8 +14,14 @@
  * @see DemoSdkTracerProviderConfigurer
  */
 public class DemoSpanProcessor implements SpanProcessor {
+
   @Override
   public void onStart(Context parentContext, ReadWriteSpan span) {
+    /*
+    The sole purpose of this attribute is to introduce runtime dependency on some external library.
+    We need this to demonstrate how extension can use them.
+     */
+    span.setAttribute("random", RandomStringUtils.random(10));
     span.setAttribute("custom", "demo");
   }