Tracing Work Items

[Harwayne]

Here is the work items I think I see for tracing:

• Tracing through Broker
  • Incoming request already has a trace id (Tracing through the Broker - incoming request has a Trace ID #1756)
  • Incoming request does not have a trace id (Broker doesn't preserve trace id on incoming requests #1708)
• Tracing through Channels
  • InMemoryChannel (Tracing through InMemoryChannel #1757)
  • KafkaChannel (Tracing through KafkaChannel eventing-contrib#563)
  • NatssChannel (Tracing through NatssChannel eventing-contrib#564)
  • PullSubscription (Tracing through Channel google/knative-gcp#226)
• Tracing through Sequences (Tracing through Sequences #1762)
• Traces contain information about why the request is made.
  • Traces from Channel dispatcher to subscriber and reply contain the Subscription's name. (Should be a subset of Tracing through the Broker - incoming request has a Trace ID #1756 and Broker doesn't preserve trace id on incoming requests #1708)
  • Traces from Triggers to subscriber and reply contain the Trigger's name (Traces from Triggers include the Trigger's name #1763)
• Tracing works with Istio (Broker Tracing works with Istio #1764)
  • An importer with the Istio side car sends an event through the Broker to a subscriber with the Istio sidecar. See a single consistent trace.
• Can collect and view traces:
  • In Zipkin
  • In Jaeger
  • In StackDriver
• E2E tests
  • Tracing through Brokers (E2E Test - Trace through Broker #1766)
  • Tracing through Channels (Channel Conformance Test - Tracing #1765)
  • Tracing through Sequences (E2E Test - Tracing through Sequences #1767)
  • Through the core Importers
    • ApiServerSource (E2E Test - Tracing through ApiServerSource #1768)
    • ContainerSource - Specifically not done as we have no control over the data plane.
    • CronJobSource (E2E Test - Tracing through CronJobSource #1769)
• Malformed tracing config is tolerated ([BUG] Broker pods die if tracing configmap is malformed on start #1417)
• Static tracing config for Importers is set via environment variables ([BUG] OnePercentSampling assumes zipkin #1712, Send observability related configmaps to destination namespaces #1758).
• Documentation
  • How to enable tracing, determine where to send it, and set sampling rates
  • How to view traces in:
    • Zipkin
    • Jaeger
    • Stackdriver
  • How to instrument an Importer
  • How to instrument a Subscriber
  • Guide of how to use tracing to debug a lost event

Nice to have:

• Move the tracing setup to sharedmain

[akashrv]

@Harwayne Could you please update this issue and mark things that are done and things that are left. Need it to plan next milestones.

[grantr]

Looks like most of these items have their own issues now. @Harwayne are you able to create issues for any tasks that don't have them and close this one?

[grantr]

@Harwayne friendly ping on this :)

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.