diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 1f260d2a25..2f2674045a 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -8,9 +8,10 @@ For new features contributions please make sure you have completed the following essential items: * [ ] `CHANGELOG.md` updated to document new feature additions -* [ ] Appropriate documentation updates in the -[docs](https://github.com/open-telemetry/opentelemetry-demo/tree/main/docs) folder +* [ ] Appropriate documentation updates in the [docs][] Maintainers will not merge until the above have been completed. If you're unsure which docs need to be changed ping the [@open-telemetry/demo-approvers](https://github.com/orgs/open-telemetry/teams/demo-approvers). + +[docs]: https://opentelemetry.io/docs/demo/ diff --git a/CHANGELOG.md b/CHANGELOG.md index e0071b51e6..5e1381e03b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -212,3 +212,5 @@ significant modifications will be credited to OpenTelemetry Authors. ([#705](https://github.com/open-telemetry/opentelemetry-demo/pull/705)) * Enable exemplar support in the metrics exporter, Prometheus, and Grafana ([#704](https://github.com/open-telemetry/opentelemetry-demo/pull/704)) +* [docs] Drop docs folder as step in migration to OTel website +([#729](https://github.com/open-telemetry/opentelemetry-demo/issues/729)) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a540fe8604..aadf7cc455 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -102,7 +102,7 @@ Once the images are built and containers are started you can access: The Demo team is committed to keeping the demo up to date. That means the documentation as well as the code. When making changes to any service or feature remember to find the related docs and update those as well. Most (but not all) -documentation can be found the [docs](./docs/) folder. +documentation can be found on the OTel website under [Demo docs][docs]. ## Create Your First Pull Request @@ -173,7 +173,7 @@ A PR is considered to be **ready to merge** when: - Major feedbacks are resolved. - It has been open for review for at least one working day. This gives people reasonable time to review. -- The [documentation](./docs/) and [Changelog](./CHANGELOG.md) have been updated +- The [documentation][docs] and [Changelog](./CHANGELOG.md) have been updated to reflect the new changes. - Trivial changes (typo, cosmetic, doc, etc.) don't have to wait for one day. @@ -208,3 +208,5 @@ Maintainers can create a new release when desired by following a few steps. creating a new tag in the format `x.x.x` based on main. Automatically generate release notes. Prepend a summary of the major changes to the release notes. - Click 'Publish Release'. + +[docs]: https://opentelemetry.io/docs/demo/ diff --git a/README.md b/README.md index 3cedc627ef..81ec32a72a 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# ![otel-photo](./docs/img/opentelemetry-logo-nav.png) OpenTelemetry Demo +# OpenTelemetry Demo [![Slack](https://img.shields.io/badge/slack-@cncf/otel/demo-brightgreen.svg?logo=slack)](https://cloud-native.slack.com/archives/C03B4CWV4DA) [![Version](https://img.shields.io/github/v/release/open-telemetry/opentelemetry-demo?color=blueviolet)](https://github.com/open-telemetry/opentelemetry-demo/releases) @@ -29,22 +29,22 @@ OpenTelemetry across its languages in the future. If you'd like to help (**which we would love**), check out our [contributing guidance](./CONTRIBUTING.md). -If you'd like to extend this demo or maintain a fork of it, please read our -[fork guidance](./docs/forking.md). +If you'd like to extend this demo or maintain a fork of it, read our +[fork guidance](https://opentelemetry.io/docs/demo/forking/). ## Quick start You can be up and running with the demo in a few minutes. Check out the docs for your preferred deployment method: -- [Docker](./docs/docker_deployment.md) -- [Kubernetes](./docs/kubernetes_deployment.md) +- [Docker](https://opentelemetry.io/docs/demo/docker_deployment/) +- [Kubernetes](https://opentelemetry.io/docs/demo/kubernetes_deployment/) ## Documentation -We have detailed documentation available in the [docs](./docs/) folder. If you're -curious about a specific feature the docs [README](./docs/README.md) can point -you in the right direction. +For detailed documentation, see [Demo Documentation][docs]. If you're curious +about a specific feature, the [docs landing page][docs] can point you in the +right direction. ## Demos featuring the Astronomy Shop @@ -93,3 +93,5 @@ Emeritus: ### Thanks to all the people who have contributed [![contributors](https://contributors-img.web.app/image?repo=open-telemetry/opentelemetry-demo)](https://github.com/open-telemetry/opentelemetry-demo/graphs/contributors) + +[docs]: https://opentelemetry.io/docs/demo/ diff --git a/docker-compose.yml b/docker-compose.yml index 7a268ea0e9..b192f26485 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -351,13 +351,17 @@ services: # Frontend Proxy (Envoy) frontendproxy: image: ${IMAGE_NAME}:${IMAGE_VERSION}-frontendproxy + container_name: frontend-proxy build: context: ./ dockerfile: src/frontendproxy/Dockerfile args: - INSTANA_AGENT_KEY=${INSTANA_AGENT_KEY} - INSTANA_DOWNLOAD_KEY=${INSTANA_DOWNLOAD_KEY} - container_name: frontend-proxy + deploy: + resources: + limits: + memory: 50M ports: - "${ENVOY_PORT}:${ENVOY_PORT}" - 10000:10000 @@ -428,7 +432,7 @@ services: deploy: resources: limits: - memory: 70M + memory: 120M restart: unless-stopped ports: - "${PAYMENT_SERVICE_PORT}" @@ -582,7 +586,7 @@ services: deploy: resources: limits: - memory: 200M + memory: 120M restart: unless-stopped environment: - POSTGRES_USER=ffs @@ -591,12 +595,12 @@ services: - INSTANA_SERVICE_NAME=ffs_postgres # command: ["-c", "track_activities=on", "-c", "track_counts=on", "-c", "track_io_timing=on"] # The above way of passing paramaeters ins't picked up by Instana. Must use config file :( - logging: *logging healthcheck: test: ["CMD-SHELL", "pg_isready -d ffs -U ffs"] interval: 10s timeout: 5s retries: 5 + logging: *logging # Kafka used by Checkout, Accounting, and Fraud Detection services kafka: @@ -610,7 +614,7 @@ services: deploy: resources: limits: - memory: 800M + memory: 750M restart: unless-stopped environment: - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 @@ -620,13 +624,14 @@ services: - OTEL_RESOURCE_ATTRIBUTES - OTEL_SERVICE_NAME=kafka - INSTANA_SERVICE_NAME=kafka - logging: *logging + - KAFKA_HEAP_OPTS=-Xmx400m -Xms400m healthcheck: test: nc -z kafka 9092 start_period: 10s interval: 5s timeout: 10s retries: 10 + logging: *logging # Redis used by Cart service redis-cart: @@ -662,7 +667,7 @@ services: deploy: resources: limits: - memory: 275M + memory: 300M restart: unless-stopped ports: - "${JAEGER_SERVICE_PORT}" # Jaeger UI @@ -676,6 +681,10 @@ services: grafana: image: grafana/grafana:9.1.0 container_name: grafana + deploy: + resources: + limits: + memory: 75M volumes: - ./src/grafana/grafana.ini:/etc/grafana/grafana.ini - ./src/grafana/provisioning/:/etc/grafana/provisioning/ @@ -690,7 +699,7 @@ services: deploy: resources: limits: - memory: 100M + memory: 125M restart: unless-stopped command: [ "--config=/etc/otelcol-config.yml", "--config=/etc/otelcol-config-extras.yml" ] volumes: @@ -728,6 +737,10 @@ services: - --enable-feature=exemplar-storage volumes: - ./src/prometheus/prometheus-config.yaml:/etc/prometheus/prometheus-config.yaml + deploy: + resources: + limits: + memory: 300M ports: - "${PROMETHEUS_SERVICE_PORT}:${PROMETHEUS_SERVICE_PORT}" logging: *logging diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 9b8200250c..0000000000 --- a/docs/README.md +++ /dev/null @@ -1,84 +0,0 @@ -# OpenTelemetry Demo Documentation - -Welcome to the OpenTelemetry Demo! This folder contains overall documentation -for the project, how to install and run it, and some scenarios you can use to -view OpenTelemetry in action. - -## Table of Contents - -- [Guided Scenarios](#scenarios) -- [Language Instrumentation Examples](#language-feature-reference) -- [Quick Start](#running-the-demo) -- [References](#reference) -- [Service Documentation](#service-documentation) - -### Running the Demo - -Want to deploy the demo and see it in action? Start here. - -- [Docker](./docker_deployment.md) -- [Kubernetes](./kubernetes_deployment.md) - -### Language Feature Reference - -Want to understand how a particular language's instrumentation works? Start -here. - -| Language | Auto Instrumentation | Manual Instrumentation | -|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------| -| .NET | [Cart Service](./services/cartservice.md) | [Cart Service](./services/cartservice.md) | -| C++ | | [Currency Service](./services/currencyservice.md) | -| Erlang/Elixir | [Feature Flag Service](./services/featureflagservice.md) | [Feature Flag Service](./services/featureflagservice.md) | -| Go | [Accounting Service](./services/accountingservice.md), [Checkout Service](./services/checkoutservice.md), [Product Catalog Service]( ./services/productcatalogservice.md ) | [Checkout Service](./services/checkoutservice.md), [Product Catalog Service]( ./services/productcatalogservice.md ) | -| Java | [Ad Service](./services/adservice.md) | [Ad Service](./services/adservice.md) | -| JavaScript | [Frontend]( ./services/frontend.md ) | [Frontend](./services/frontend.md), [Payment Service](./services/paymentservice.md) | -| Kotlin | [Fraud Detection Service]( ./services/frauddetectionservice.md ) | | -| PHP | [Quote Service](./services/quoteservice.md) | [Quote Service](./services/quoteservice.md) | -| Python | [Recommendation Service](./services/recommendationservice.md) | [Recommendation Service](./services/recommendationservice.md) | -| Ruby | [Email Service](./services/emailservice.md) | [Email Service](./services/emailservice.md) | -| Rust | [Shipping Service](./services/shippingservice.md) | [Shipping Service](./services/shippingservice.md) | - -### Service Documentation - -Specific information about how OpenTelemetry is deployed in each service can be -found here: - -- [Ad Service](./services/adservice.md) -- [Cart Service](./services/cartservice.md) -- [Checkout Service](./services/checkoutservice.md) -- [Email Service](./services/emailservice.md) -- [Feature Flag Service](./services/featureflagservice.md) -- [Frontend](./services/frontend.md) -- [Load Generator](./services/loadgenerator.md) -- [Payment Service](./services/paymentservice.md) -- [Product Catalog Service](./services/productcatalogservice.md) -- [Quote Service](./services/quoteservice.md) -- [Recommendation Service](./services/recommendationservice.md) -- [Shipping Service](./services/shippingservice.md) - -### Scenarios - -How can you solve problems with OpenTelemetry? These scenarios walk you through -some pre-configured problems and show you how to interpret OpenTelemetry data to -solve them. - -We'll be adding more scenarios over time. - -- Generate a [Product Catalog error](feature_flags.md) for `GetProduct` requests - with product id: `OLJCESPC7Z` using the Feature Flag service -- Discover a memory leak and diagnose it using metrics and traces. [Read more](./scenarios/recommendation_cache.md) - -### Reference - -Project reference documentation, like requirements and feature matrices. - -- [Architecture](./current_architecture.md) -- [Development](./development.md) -- [Feature Flags Reference](./feature_flags.md) -- [Metric Feature Matrix](./metric_service_features.md) -- [Requirements](./requirements/) -- [Screenshots](./demo_screenshots.md) -- [Service Roles Table](./service_table.md) -- [Span Attributes Reference](./manual_span_attributes.md) -- [Tests](./tests.md) -- [Trace Feature Matrix](./trace_service_features.md) diff --git a/docs/current_architecture.md b/docs/current_architecture.md deleted file mode 100644 index aa9a764207..0000000000 --- a/docs/current_architecture.md +++ /dev/null @@ -1,196 +0,0 @@ -# Demo Architecture - -**OpenTelemetry Demo** is composed of microservices written in different programming -languages that talk to each other over gRPC and HTTP; and a load generator which -uses [Locust](https://locust.io/) to fake user traffic. - -```mermaid -graph TD -subgraph Service Diagram -accountingservice(Accounting Service):::golang -adservice(Ad Service):::java -cache[(Cache
(redis))] -cartservice(Cart Service):::dotnet -checkoutservice(Checkout Service):::golang -currencyservice(Currency Service):::cpp -emailservice(Email Service):::ruby -frauddetectionservice(Fraud Detection Service):::kotlin -frontend(Frontend):::typescript -frontendproxy(Frontend Proxy
(Envoy)):::cpp -loadgenerator([Load Generator]):::python -paymentservice(Payment Service):::javascript -productcatalogservice(Product Catalog Service):::golang -quoteservice(Quote Service):::php -recommendationservice(Recommendation Service):::python -shippingservice(Shipping Service):::rust -featureflagservice(Feature Flag Service):::erlang -featureflagstore[(Feature Flag Store
(PostgreSQL DB))] -queue[(queue
(Kafka))] - -Internet -->|HTTP| frontendproxy -frontendproxy -->|HTTP| frontend -frontendproxy -->|HTTP| featureflagservice -loadgenerator -->|HTTP| frontend - -accountingservice -->|TCP| queue - -checkoutservice --->|gRPC| cartservice --> cache -checkoutservice --->|gRPC| productcatalogservice -checkoutservice --->|gRPC| currencyservice -checkoutservice --->|HTTP| emailservice -checkoutservice --->|gRPC| paymentservice -checkoutservice -->|gRPC| shippingservice -checkoutservice ---->|TCP| queue - -frontend -->|gRPC| adservice -frontend -->|gRPC| cartservice -frontend -->|gRPC| productcatalogservice -frontend -->|gRPC| checkoutservice -frontend -->|gRPC| currencyservice -frontend -->|gRPC| recommendationservice -->|gRPC| productcatalogservice -frontend -->|gRPC| shippingservice -->|HTTP| quoteservice - -frauddetectionservice -->|TCP| queue - -adservice -->|gRPC| featureflagservice - -productcatalogservice -->|gRPC| featureflagservice - -recommendationservice -->|gRPC| featureflagservice - -shippingservice -->|gRPC| featureflagservice - -featureflagservice --> featureflagstore - -end - -classDef dotnet fill:#178600,color:white; -classDef cpp fill:#f34b7d,color:white; -classDef erlang fill:#b83998,color:white; -classDef golang fill:#00add8,color:black; -classDef java fill:#b07219,color:white; -classDef javascript fill:#f1e05a,color:black; -classDef kotlin fill:#560ba1,color:white; -classDef php fill:#4f5d95,color:white; -classDef python fill:#3572A5,color:white; -classDef ruby fill:#701516,color:white; -classDef rust fill:#dea584,color:black; -classDef typescript fill:#e98516,color:black; -``` - -```mermaid -graph TD -subgraph Service Legend - dotnetsvc(.NET):::dotnet - cppsvc(C++):::cpp - erlangsvc(Erlang/Elixir):::erlang - golangsvc(Go):::golang - javasvc(Java):::java - javascriptsvc(JavaScript):::javascript - kotlinsvc(Kotlin):::kotlin - phpsvc(PHP):::php - pythonsvc(Python):::python - rubysvc(Ruby):::ruby - rustsvc(Rust):::rust - typescriptsvc(TypeScript):::typescript -end - -classDef dotnet fill:#178600,color:white; -classDef cpp fill:#f34b7d,color:white; -classDef erlang fill:#b83998,color:white; -classDef golang fill:#00add8,color:black; -classDef java fill:#b07219,color:white; -classDef javascript fill:#f1e05a,color:black; -classDef kotlin fill:#560ba1,color:white; -classDef php fill:#4f5d95,color:white; -classDef python fill:#3572A5,color:white; -classDef ruby fill:#701516,color:white; -classDef rust fill:#dea584,color:black; -classDef typescript fill:#e98516,color:black; -``` - -Follow these links for the current state of - [metric](https://github.com/open-telemetry/opentelemetry-demo/blob/main/docs/metric_service_features.md) - and - [trace](https://github.com/open-telemetry/opentelemetry-demo/blob/main/docs/trace_service_features.md) - instrumentation of the demo applications. - -The collector is configured in - [otelcol-config.yml](https://github.com/open-telemetry/opentelemetry-demo/blob/main/src/otelcollector/otelcol-config.yml), - alternative exporters can be configured here. - -```mermaid -graph TB -subgraph tdf[Telemetry Data Flow] - subgraph subgraph_padding [ ] - style subgraph_padding fill:none,stroke:none; - %% padding to stop the titles clashing - subgraph od[Open Telemetry Demo] - ms(Microservice) - end - - ms -.->|"OTLP
gRPC"| oc-grpc - ms -.->|"OTLP
HTTP POST"| oc-http - - subgraph oc[OTel Collector] - style oc fill:#97aef3,color:black; - oc-grpc[/"OTLP Receiver
listening on
grpc://localhost:4317/"/] - oc-http[/"OTLP Receiver
listening on
http://localhost:4318/
https://localhost:4318/"/] - oc-proc(Processors) - oc-prom[/"Prometheus Exporter
listening on
http://localhost:9464/"/] - oc-jag[/"Jaeger Exporter"/] - - oc-grpc --> oc-proc - oc-http --> oc-proc - - oc-proc --> oc-prom - oc-proc --> oc-jag - end - - oc-prom -->|"http://localhost:9464/metrics"| pr-sc - oc-jag -->|gRPC| ja-col - - subgraph pr[Prometheus] - style pr fill:#e75128,color:black; - pr-sc[/"Prometheus Scraper
polling every 5 seconds"/] - pr-tsdb[(Prometheus TSDB)] - pr-http[/"Prometheus HTTP
listening on
http://localhost:9090"/] - - pr-sc --> pr-tsdb - pr-tsdb --> pr-http - end - - pr-b{{"Browser
Prometheus UI"}} - pr-http ---->|"http://localhost:9090/graph"| pr-b - - subgraph ja[Jaeger] - style ja fill:#60d0e4,color:black; - ja-col[/"Jaeger Collector
listening on
grpc://jaeger:4317/"/] - ja-db[(Jaeger DB)] - ja-http[/"Jaeger HTTP
listening on
http://localhost:16686"/] - - ja-col --> ja-db - ja-db --> ja-http - end - - subgraph gr[Grafana] - style gr fill:#f8b91e,color:black; - gr-srv["Grafana Server"] - gr-http[/"Grafana HTTP
listening on
http://localhost:3000"/] - - gr-srv --> gr-http - end - - pr-http --> |"http://localhost:9090/api"| gr-srv - ja-http --> |"http://localhost:16686/api"| gr-srv - - ja-b{{"Browser
Jaeger UI"}} - ja-http ---->|"http://localhost:16686/search"| ja-b - - gr-b{{"Browser
Grafana UI"}} - gr-http -->|"http://localhost:3000/dashboard"| gr-b - end -end -``` - -Find the **Protocol Buffer Definitions** in the `/pb/` directory. diff --git a/docs/demo_features.md b/docs/demo_features.md deleted file mode 100644 index 4c9632cd2e..0000000000 --- a/docs/demo_features.md +++ /dev/null @@ -1,36 +0,0 @@ -# Features - -- **[Kubernetes](https://kubernetes.io)**: the app is designed to run on - Kubernetes (both locally, as well as on the cloud) using a Helm chart. -- **[Docker](https://docs.docker.com)**: this forked sample can also be executed - only with Docker. -- **[gRPC](https://grpc.io)**: microservices use a high volume of gRPC calls to - communicate to each other. -- **[HTTP](https://www.rfc-editor.org/rfc/rfc9110.html)**: microservices use HTTP - where gRPC is unavailable or not well supported. -- **[OpenTelemetry Traces](https://opentelemetry.io)**: all services are - instrumented using OpenTelemetry available instrumentation libraries. -- **[OpenTelemetry Metrics](https://opentelemetry.io)**: Select services are -instrumented using OpenTelemetry available instrumentation libraries. More will -be added as the relevant SDKs are released. -- **[OpenTelemetry - Collector](https://opentelemetry.io/docs/collector/getting-started)**: all - services are instrumented and sending the generated traces and metrics to the - OpenTelemetry Collector via gRPC. The received traces are then exported to the - logs and to Jaeger; received metrics and exemplars* are exported to logs and Prometheus. -- **[Jaeger](https://www.jaegertracing.io)**: all generated traces are being - sent to Jaeger. -- **Synthetic Load Generation**: the application demo comes with a background - job that creates realistic usage patterns on the website using - [Locust](https://locust.io/) load generator. -- **[Prometheus](https://prometheus.io/)**: all generated metrics and exemplars - are scraped by Prometheus. -- **[Grafana](https://grafana.com/)**: all metric dashboards are stored in - Grafana. -- **[Envoy](https://www.envoyproxy.io/)**: Envoy is used as a reverse proxy for - user-facing web interfaces such as the frontend, load generator, and feature - flag service. - -_\*Only exemplars attached to histograms are currently exported to Prometheus. -See [issue in the collector](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/18201) -for details._ diff --git a/docs/demo_screenshots.md b/docs/demo_screenshots.md deleted file mode 100644 index ef5191716e..0000000000 --- a/docs/demo_screenshots.md +++ /dev/null @@ -1,43 +0,0 @@ -# Demo Screenshots - -- [Demo Screenshots](#demo-screenshots) - - [Webstore](#webstore) - - [Jaeger](#jaeger) - - [Prometheus](#prometheus) - - [Grafana](#grafana) - - [Feature Flag UI](#feature-flag-ui) - - [Load Generator UI](#load-generator-ui) - -## Webstore - -| Home Page | Checkout Screen | -| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -| ![frontend-1](https://user-images.githubusercontent.com/15364991/194416314-d8210de5-814a-45b8-a47c-9d6bace51fa5.png) | ![frontend-2](https://user-images.githubusercontent.com/15364991/194416580-7f7ac8fe-85c0-462d-a896-6e0f6c95afac.png) | - -## Jaeger - -| Jaeger UI | Trace View | -| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -| ![jaeger-ui](https://user-images.githubusercontent.com/15364991/194417029-1e40f276-8785-47fb-b645-c90820c0ff52.png) | ![jaeger-trace-view](https://user-images.githubusercontent.com/15364991/194416769-188acc26-71c9-4611-8150-aa8639f11217.png) | - -| System Architecture| -| --- | -|![jaeger-system-architecture](https://user-images.githubusercontent.com/47896520/196496223-6d6ea729-5bea-4a8c-a2c6-cd51cce386ae.png)| - -## Prometheus - -![prometheus](https://user-images.githubusercontent.com/47896520/196564627-ba9f5e91-a0fa-4bf0-beb1-f3a79ca309b5.png) - -## Grafana - -| Prometheus Data Source |Jaeger Data Source | -| --- | --- | -| ![grafana-prometheus](https://user-images.githubusercontent.com/47896520/196495466-86d3ce33-def4-4808-a6d9-3e32934bbf03.png) | ![gragana-jaeger](https://user-images.githubusercontent.com/47896520/196495509-c233d0d9-5e4c-4607-b609-dbf7869298d9.png) | - -## Feature Flag UI - -![feature-flag-ui](https://user-images.githubusercontent.com/47896520/196496050-22fbbb56-9e62-46dd-a23c-c34c5566f7d2.png) - -## Load Generator UI - -![load-generator-ui](https://user-images.githubusercontent.com/47896520/196496589-bda802fc-aada-4e72-a184-93bb4711abca.png) diff --git a/docs/development.md b/docs/development.md deleted file mode 100644 index cb4edae1e0..0000000000 --- a/docs/development.md +++ /dev/null @@ -1,71 +0,0 @@ -# Development - -Development for this demo requires tooling in several programming languages. -Minimum required versions will be noted where possible, but it is recommended -to update to the latest version for all tooling. The OpenTelemetry demo team -will attempt to keep the services in this repository up to date with the latest -version for dependencies and tooling when possible. - -## Generate Protobuf files - -The `make generate-protobuf` command is provided to generate protobuf files for -all services. This can be used to compile code locally (without Docker) and -receive hints from IDEs such as IntelliJ or VS Code. It may be necessary to run -`npm install` within the frontend source folder before generating the files. - -## Development tooling requirements - -### .NET - -- .NET 6.0+ - -### C++ - -- build-essential -- cmake -- libcurl4-openssl-dev -- libprotobuf-dev -- nlohmann-json3-dev -- pkg-config -- protobuf-compiler - -### Elixir - -- Erlang/OTP 23+ -- Elixir 1.13+ -- Rebar3 3.20+ - -### Go - -- Go 1.19+ -- protoc-gen-go -- protoc-gen-go-grpc - -### Java - -- JDK 17+ -- Gradle 7+ - -### JavaScript - -- Node.js 16+ - -### PHP - -- PHP 8.1+ -- Composer 2.4+ - -### Python - -- Python 3.10 -- grpcio-tools 1.48+ - -### Ruby - -- Ruby 3.1+ - -### Rust - -- Rust 1.61+ -- protoc 3.21+ -- protobuf-dev diff --git a/docs/docker_deployment.md b/docs/docker_deployment.md deleted file mode 100644 index de06ea59eb..0000000000 --- a/docs/docker_deployment.md +++ /dev/null @@ -1,95 +0,0 @@ -# Docker - -## Prerequisites - -- Docker -- [Docker Compose](https://docs.docker.com/compose/install/#install-compose) v2.0.0+ -- 5 GB of RAM - -## Clone Repo - -- Clone the Webstore Demo repository: - -```shell -git clone https://github.com/open-telemetry/opentelemetry-demo.git -``` - -## Open Folder - -- Navigate to the cloned folder: - -```shell -cd opentelemetry-demo/ -``` - -## Run Docker Compose - -- Start the demo: - -```shell -docker compose up --no-build -``` - -> **Note:** If you're running on Apple Silicon, please run `docker compose -> build` in order to create local images vs. pulling them from the repository. - -**Note:** The `--no-build` flag is used to fetch released docker images from -[ghcr](http://ghcr.io/open-telemetry/demo) instead of building from source. -Removing the `--no-build` command line option will rebuild all images from -source. It may take more than 20 minutes to build if the flag is omitted. - -## Verify the Webstore & the Telemetry - -Once the images are built and containers are started you can access: - -- Webstore: -- Grafana: -- Feature Flags UI: -- Load Generator UI: -- Jaeger UI: - -## Bring your own backend - -Likely you want to use the Webstore as a demo application for an observability -backend you already have (e.g. an existing instance of Jaeger, Zipkin, or one -of the [vendor of your choice](https://opentelemetry.io/vendors/). - -OpenTelemetry Collector can be used to export telemetry data to multiple -backends. By default, the collector in the demo application will merge the -configuration from two files: - -- otelcol-config.yml -- otelcol-config-extras.yml - -To add your backend, open the file -[src/otelcollector/otelcol-config-extras.yml](../src/otelcollector/otelcol-config-extras.yml) -with an editor. - -- Start by adding a new exporter. For example, if your backend supports - OTLP over HTTP, add the following: - -```yaml -exporters: - otlphttp/example: - endpoint: -``` - -- Then add a new pipeline with your new exporter: - -```yaml -service: - pipelines: - traces: - receivers: [otlp] - processors: [batch] - exporters: [otlphttp/example] -``` - -Vendor backends might require you to add additional parameters for -authentication, please check their documentation. Some backends require -different exporters, you may find them and their documentation available at -[opentelemetry-collector-contrib/exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter). - -After updating the `otelcol-config-extras.yml`, start the demo by running -`docker compose up`. After a while, you should see the traces flowing into -your backend as well. diff --git a/docs/feature_flags.md b/docs/feature_flags.md deleted file mode 100644 index e250f9b63c..0000000000 --- a/docs/feature_flags.md +++ /dev/null @@ -1,12 +0,0 @@ -# Feature Flags - -This demo comes with several feature flags which can control failure conditions -in specific services. By default the flags are disabled. Using the Feature Flags -UI you will be able to control the status of these -feature flags. - -| Feature Flag | Service(s) | Description | -|-------------------------|-----------------|----------------------------------------------------------------------------------------------------------| -| `adServiceFailure` | Ad Service | Generate an error for `GetAds` 1/10th of the time | -| `productCatalogFailure` | Product Catalog | Generate an error for `GetProduct` requests with product id: `OLJCESPC7Z` | -| `recommendationCache` | Recommendation | Create a memory leak due to an exponentially growing cache. 1.4x growth, 50% of requests trigger growth. | diff --git a/docs/forking.md b/docs/forking.md deleted file mode 100644 index 72a58e3223..0000000000 --- a/docs/forking.md +++ /dev/null @@ -1,28 +0,0 @@ -# Forking this Repository - -This repository is designed to be forked and used as a tool to show off what you -are doing with OpenTelemetry. - -Setting up a fork or a demo usually only requires overriding some environment -variables and possibly replacing some container images. - -Live demos can be added to the -[README](https://github.com/open-telemetry/opentelemetry-demo/blob/main/README.md?plain=1#L186). - -## Suggestions for Fork Maintainers - -- If you'd like to enhance the telemetry data emitted or collected by the demo, - then we strongly encourage you to backport your changes to this repository. - For vendor or implementation specific changes, a strategy of modifying - telemetry in the pipeline via config is preferable to underlying code changes. -- Extend rather than replace. Adding net-new services that interface with the - existing API is a great way to add vendor-specific or tool-specific features - that can't be accomplished through telemetry modification. -- To support extensibility, please use repository or facade patterns around - resources like queues, databases, caches, etc. This will allow for different - implementations of these services to be shimmed in for different platforms. -- Please do not attempt to backport vendor or tool-specific enhancements to this - repository. - -If you have any questions or would like to suggest ways that we can make your -life easier as a fork maintainer, please open an issue. diff --git a/docs/img/opentelemetry-logo-nav.png b/docs/img/opentelemetry-logo-nav.png deleted file mode 100644 index 1606a20448..0000000000 Binary files a/docs/img/opentelemetry-logo-nav.png and /dev/null differ diff --git a/docs/kubernetes_deployment.md b/docs/kubernetes_deployment.md deleted file mode 100644 index 8874bf0c42..0000000000 --- a/docs/kubernetes_deployment.md +++ /dev/null @@ -1,167 +0,0 @@ -# Kubernetes - -We provide a [OpenTelemetry Demo Helm -chart](https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-demo) -to help deploy the demo to an existing Kubernetes cluster. - -[Helm](https://helm.sh) must be installed to use the charts. -Please refer to Helm's [documentation](https://helm.sh/docs/) to get started. - -## Prerequisites - -- Pre-existing Kubernetes Cluster -- Helm 3.0+ - -## Install the Chart - -Add OpenTelemetry Helm repository: - -```shell -helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts -``` - -To install the chart with the release name my-otel-demo, run the following command: - -```shell -helm install my-otel-demo open-telemetry/opentelemetry-demo -``` - -> **Note** -> The OpenTelemetry Demo Helm chart version 0.11.0 or greater is required to -> perform all usage methods mentioned below. - -## Use the Demo - -The demo application will need the services exposed outside of the Kubernetes -cluster in order to use them. You can expose the services to your local system -using the `kubectl port-forward` command or by configuring service types -(ie: LoadBalancer) with optionally deployed ingress resources. - -### Expose services using kubectl port-forward - -To expose the frontendproxy service use the following command (replace -`my-otel-demo` with your Helm chart release name accordingly): - -```shell -kubectl port-forward svc/my-otel-demo-frontendproxy 8080:8080 -``` - -In order for spans from the browser to be properly collected, you will also -need to expose the OpenTelemetry Collector's OTLP/HTTP port (replace -`my-otel-demo` with your Helm chart release name accordingly): - -```shell -kubectl port-forward svc/my-otel-demo-otelcol 4318:4318 -``` - -> **Note** -> `kubectl port-forward` will proxy the port until the process terminates. You -> may need to create separate terminal sessions for each use of -> `kubectl port-forward`, and use CTRL-C to terminate the process when done. - -With the frontendproxy and Collector port-forward set up, you can access: - -- Webstore: -- Grafana: -- Feature Flags UI: -- Load Generator UI: -- Jaeger UI: - -### Expose services using service type configurations - -> **Note** -> Kubernetes clusters may not have the proper infrastructure components to -> enable LoadBalancer service types or ingress resources. Verify your cluster -> has the proper support before using these configuration options. - -Each demo service (ie: frontendproxy) offers a way to have its Kubernetes -service type configured. By default these will be `ClusterIP` but you can change -each one using the `serviceType` property for each service. - -To configure the frontendproxy service to use a LoadBalancer service type you -would specify the following in your values file: - -```yaml -components: - frontendProxy: - serviceType: LoadBalancer -``` - -> **Note** -> It is recommended to use a values file when installing the Helm chart in order -> to specify additional configuration options. - -The Helm chart does not provide facilities to create ingress resources. If -required these would need to be created manually after installing the Helm chart. -Some Kubernetes providers require specific service types in order to be used by -ingress resources (ie: EKS ALB ingress, requires a NodePort service type). - -In order for spans from the browser to be properly collected, you will also -need to expose the OpenTelemetry Collector's OTLP/HTTP port to be accessible to -user web browsers. The location where the OpenTelemetry Collector is exposed -must also be passed into the frontend service using the -`PUBLIC_OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` environment variable. You can do -this using the following in your values file: - -```yaml -components: - frontend: - env: - - name: PUBLIC_OTEL_EXPORTER_OTLP_TRACES_ENDPOINT - value: "http://otel-demo-collector.mydomain.com:4318/v1/traces" -``` - -To install the Helm chart with a custom `my-values-file.yaml` values file use: - -```shell -helm install my-otel-demo open-telemetry/opentelemetry-demo --values my-values-file.yaml -``` - -With the frontendproxy and Collector exposed, you can access the demo UI at the -base path for the frontendproxy. Other demo components can be accessed at the -following sub-paths: - -- Webstore: `/` (base) -- Grafana: `/grafana` -- Feature Flags UI: `/feature` -- Load Generator UI: `/loadgen/` (must include trailing slash) -- Jaeger UI: `/jaeger/ui` - -## Bring your own backend - -Likely you want to use the Webstore as a demo application for an observability -backend you already have (e.g. an existing instance of Jaeger, Zipkin, or one -of the [vendor of your choice](https://opentelemetry.io/vendors/). - -The OpenTelemetry Collector's configuration is exposed in the Helm chart. Any -additions you do will be merged into the default configuration. You can use -this to add your own exporters, and add them to the desired pipeline(s) - -```yaml -opentelemetry-collector: - config: - exporters: - otlphttp/example: - endpoint: - - service: - pipelines: - traces: - receivers: [otlp] - processors: [batch] - exporters: [otlphttp/example] -``` - -> **Note** -> When merging YAML values with Helm, objects are merged and arrays are replaced. - -Vendor backends might require you to add additional parameters for -authentication, please check their documentation. Some backends require -different exporters, you may find them and their documentation available at -[opentelemetry-collector-contrib/exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter). - -To install the Helm chart with a custom `my-values-file.yaml` values file use: - -```shell -helm install my-otel-demo open-telemetry/opentelemetry-demo --values my-values-file.yaml -``` diff --git a/docs/manual_span_attributes.md b/docs/manual_span_attributes.md deleted file mode 100644 index 2dfcda9ce4..0000000000 --- a/docs/manual_span_attributes.md +++ /dev/null @@ -1,126 +0,0 @@ -# Manual Span Attributes - -This document contains the list of manual Span Attributes used throughout the demo: - -## AdService - -| Name | Type | Description | -|-----------------------------|--------|---------------------------------------| -| `app.ads.category` | string | Category for returned ad | -| `app.ads.contextKeys` | string | Context keys used to find related ads | -| `app.ads.contextKeys.count` | number | Count of unique context keys used | -| `app.ads.count` | number | Count of ads returned to user | -| `app.ads.ad_request_type` | string | Either `targeted` or `not_targeted` | -| `app.ads.ad_response_type` | string | Either `targeted` or `random` | - -## CartService - -| Name | Type | Description | -|------------------------|--------|--------------------------------| -| `app.cart.items.count` | number | Number of unique items in cart | -| `app.product.id` | string | Product Id for cart item | -| `app.product.quantity` | string | Quantity for cart item | -| `app.user.id` | string | User Id | - -## CheckoutService - -| Name | Type | Description | -|------------------------------|--------|---------------------------------| -| `app.cart.items.count` | number | Total number of items in cart | -| `app.order.amount` | number | Order amount | -| `app.order.id` | string | Order Id | -| `app.order.items.count` | number | Number of unique items in order | -| `app.payment.transaction.id` | string | Payment transaction Id | -| `app.shipping.amount` | number | Shipping amount | -| `app.shipping.tracking.id` | string | Shipping tracking Id | -| `app.user.currency` | string | User currency | -| `app.user.id` | string | User Id | - -## CurrencyService - -| Name | Type | Description | -|--------------------------------|--------|-------------------------------| -| `app.currency.conversion.from` | string | Currency code to convert from | -| `app.currency.conversion.to` | string | Currency code to convert to | - -## EmailService - -| Name | Type | Description | -|-----------------------|--------|-----------------------------------| -| `app.email.recipient` | string | Email used for order confirmation | -| `app.order.id` | string | Order Id | - -## FeatureFlagService - -| Name | Type | Description | -|-------------------------------|--------|--------------------------| -| `app.featureflag.name` | string | Name of the feature flag | -| `app.featureflag.description` | string | Admin description | -| `app.featureflag.enabled` | boolean | The feature flag status | - -## Frontend - -| Name | Type | Description | -|--------------------------|--------|-------------------------------| -| `app.cart.size` | number | Total number of items in cart | -| `app.cart.items.count` | number | Count of unique items in cart | -| `app.cart.shipping.cost` | number | Cart shipping cost | -| `app.cart.total.price` | number | Cart total price | -| `app.currency` | string | User currency | -| `app.currency.new` | string | New currency to set | -| `app.order.total` | number | Order total cost | -| `app.product.id` | string | Product Id | -| `app.product.quantity` | number | Product quantity | -| `app.products.count` | number | Total products displayed | -| `app.request.id` | string | Request Id | -| `app.session.id` | string | Session Id | -| `app.user.id` | string | User Id | - -## LoadGenerator - -| Name | Type | Description | -|----------|------|-------------| -| None yet | | | - -## PaymentService - -| Name | Type | Description | -|--------------------------|---------|------------------------------------------------------| -| `app.payment.amount` | number | Total payment amount | -| `app.payment.card_type` | string | Type of card used for payment | -| `app.payment.card_valid` | boolean | Was the card used valid | -| `app.payment.charged` | boolean | Was the charge successful (false with loadgenerator) | - -## ProductCatalogService - -| Name | Type | Description | -|-----------------------------|--------|---------------------------------------| -| `app.product.id` | string | Product Id | -| `app.product.name` | string | Product name | -| `app.products.count` | number | Number of products in catalog | -| `app.products_search.count` | number | Number of products returned in search | - -## QuoteService - -| Name | Type | Description | -|-------------------------|--------|----------------------| -| `app.quote.items.count` | number | Total items to ship | -| `app.quote.cost.total` | number | Total shipping quote | - -## RecommendationService - -| Name | Type | Description | -|----------------------------------|---------|-----------------------------------------| -| `app.filtered_products.count` | number | Number of filtered products returned | -| `app.products.count` | number | Number of products in catalog | -| `app.products_recommended.count` | number | Number of recommended products returned | -| `app.cache_hit` | boolean | If cache was accessed or not | - -## ShippingService - -| Name | Type | Description | -|----------------------------|--------|-------------------------------| -| `app.shipping.cost.total` | number | Total shipping cost | -| `app.shipping.items.count` | number | Total items to ship | -| `app.shipping.tracking.id` | string | Shipping tracking Id | -| `app.shipping.zip_code` | string | Zip code used to ship item(s) | diff --git a/docs/metric_service_features.md b/docs/metric_service_features.md deleted file mode 100644 index 691fb7a463..0000000000 --- a/docs/metric_service_features.md +++ /dev/null @@ -1,24 +0,0 @@ -# Metric Feature Coverage by Service - -Emoji Legend - -- Completed: :100: -- Not Applicable: :no_bell: -- Not Present (Yet): :construction: - -| Service | Language | Auto-instrumentation | Manual Instrumentation | Multiple Instruments | Views | Custom Attributes | Resource Detection | Trace Exemplars | -|-----------------|-----------------|----------------------|------------------------|----------------------|----------------|-------------------|--------------------|-----------------| -| Accounting | Go | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Ad | Java | :100: | :100: | :construction: | :construction: | :100: | :100: | :100: | -| Cart | .NET | :100: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Checkout | Go | :100: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Currency | C++ | :no_bell: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Email | Ruby | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Feature Flag | Erlang / Elixir | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Fraud Detection | Kotlin | :100: | :construction: | :construction: | :construction: | :construction: | :100: | :construction: | -| Frontend | TypeScript | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Payment | JavaScript | :construction: | :100: | :construction: | :construction: | :construction: | :100: | :construction: | -| Product Catalog | Go | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Quote | PHP | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Recommendation | Python | :100: | :100: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Shipping | Rust | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | diff --git a/docs/requirements/README.md b/docs/requirements/README.md deleted file mode 100644 index 3fea3f6890..0000000000 --- a/docs/requirements/README.md +++ /dev/null @@ -1,27 +0,0 @@ -# OpenTelemetry Community Demo Requirements - -The following documents capture the Application, OpenTelemetry (OTel), and System -requirements for our shared demo application. These were decided upon in the -ongoing SIG meeting. - -1. [Application Requirements](./application_requirements.md) - -2. [OpenTelemetry Requirements](./opentelemetry_requirements.md) - -3. [System Requirements](./system_requirements.md) - -## Target Personas - -We're building the demo application with several different target personas in mind: - -1. **Enthusiasts** at a company that can use the demo app as an individual to - advocate for OTel within their organization. - -2. **Developers** with specific language skills who want to see a larger picture - view. - -3. **APM Vendors** who can evaluate OTel in general or need to produce a demo of - their OTel capabilities for customers. - -4. **Enterprises** considering adopting OTel and interested in understanding - what a production-lite experience would be. diff --git a/docs/requirements/application_requirements.md b/docs/requirements/application_requirements.md deleted file mode 100644 index 4093a0e612..0000000000 --- a/docs/requirements/application_requirements.md +++ /dev/null @@ -1,28 +0,0 @@ -# Application Requirements - -The following requirements were decided upon to define what OpenTelemetry (OTel) -signals the application will produce & when support for future SDKs should be -added: - -1. Every supported language that has a GA Traces or Metrics SDK must have at - least 1 service example. - - * Mobile support (Swift) is not an initial priority and not included in the - above requirement. - -2. Application processes must be language independent. - - * gRPC is preferred where available and HTTP is to be used where it is not. - -3. Services should be architected to be modular components that can be switched out. - - * Individual services can and should be encouraged to have multiple language - options available. - -4. The architecture must allow for the possible integration of platform generic - components like a database, queue, or blob storage. - - * There is no requirement for a particular component type - at least 1 generic - component should be present in general. - -5. A load generator must be provided to simulate user load against the demo. diff --git a/docs/requirements/architecture_requirements.md b/docs/requirements/architecture_requirements.md deleted file mode 100644 index 72a6e6940e..0000000000 --- a/docs/requirements/architecture_requirements.md +++ /dev/null @@ -1,92 +0,0 @@ -# Demo Architecture - -## Summary - -The OpenTelemetry Community Demo application is intended to be a 'showcase' for -OpenTelemetry API, SDK, and tools in a production-lite cloud native -application. The overall goal of this application is not only to provide a -canonical 'demo' of OpenTelemetry components, but also to act as a framework for -further customization by end-users, vendors, and other stakeholders. - -### Requirements - -- [Application Requirements](./application_requirements.md) -- [OpenTelemetry Requirements](./opentelemetry_requirements.md) -- [System Requirements](./system_requirements.md) - -### Application Goals - -- Provide developers with a robust sample application they can use in learning -OpenTelemetry instrumentation. -- Provide observability vendors with a single, -well-supported, demo platform that they can further customize (or simply use -OOB). -- Provide the OpenTelemetry community with a living artifact that -demonstrates the features and capabilities of OTel APIs, SDKs, and tools. -- Provide OpenTelemetry maintainers and WGs a platform to demonstrate new -features/concepts 'in the wild'. - -The following is a general description of the logical components of the demo -application. - -## Main Application - -The bulk of the demo app is a self-contained -microservice-based application that does some useful 'real-world' work, such as -an eCommerce site. This application is composed of multiple services that -communicate with each other over gRPC and HTTP and runs on Kubernetes (or -Docker, locally). - -Each service shall be instrumented with OpenTelemetry for traces, metrics, and -logs (as applicable/available). - -Each service should be 'swappable' with a service that performs the same -business logic, implementing the same gRPC endpoints, but written in a different -language/implementation. For the initial implementation of the demo, we should -focus on adding as many missing languages as possible by swapping out existing -services with implementations in un-represented languages. For future versions -we will look to add more distinct language options per service. - -Each service should communicate with a feature flag service in order to -enable/disable 'faults' that can be used to illustrate how telemetry helps solve -problems in distributed applications. - -A PHP service should be added to the main application as an 'admin service'. A -Database should be added to enable CRUD functionality on the Product Catalog. - -The 'shippingservice' should be reimplemented in Rust. - -The 'currencyservice' should be reimplemented in C++. - -The 'emailservice' should be reimplemented in Ruby. - -For future iterations, the 'frontend' service can be extended with a mobile -application written in Swift. - -## Feature Flag Component - -This component should consist of one (or more) services -that provides a simple feature flag configuration utility for the main -application. It is made up of a browser-based client/admin interface and a -backend service or services. The role of the client is to allow an operator to -visualize the available feature flags and toggle their state. The server should -provide a catalog of feature flags that main application services can register -with and interrogate for their current status and targeting rules. - -The feature flag component should be implemented as an Erlang+Elixir/Phoenix -service. The catalog of feature flags should be stored in a Database. - -## Orchestration and Deployment - -All services should run on Kubernetes. The -OpenTelemetry Collector should be deployed via the OpenTelemetry Operator, and -run in a sidecar + gateway mode. Telemetry from each pod should be routed from -agents to a gateway, and the gateway should export telemetry by default to an -open-source trace + metrics visualizer. - -For local/non-kubernetes deployment, the Collector should be deployed via -compose file and monitor not only traces/metrics from applications, but also the -docker containers via dockerstatsreceiver. - -A design goal of this project is to include a CI/CD pipeline for self-deployment -into cloud environments. This could be skipped for local development. diff --git a/docs/requirements/opentelemetry_requirements.md b/docs/requirements/opentelemetry_requirements.md deleted file mode 100644 index 0c4802e2f0..0000000000 --- a/docs/requirements/opentelemetry_requirements.md +++ /dev/null @@ -1,24 +0,0 @@ -# OpenTelemetry Requirements - -The following requirements were decided upon to define what OpenTelemetry (OTel) -signals the application will produce & when support for future SDKs should be -added: - -1. The demo must produce OTel logs, traces, & metrics out of the box for - languages that have a GA SDK. - -2. Languages that have a Beta SDK available may be included but are not required - like GA SDKs. - -3. Native OTel metrics should be produced where possible. - -4. Both manual instrumentation and instrumentation libraries - (auto-instrumentation) should be demonstrated in each language. - -5. All data should be exported to the Collector first. - -6. The Collector must be configurable to allow for a variety of consumption - experiences but default tools must be selected for each signal. - -7. The demo application architecture using the Collector should be designed to - be a best practices reference architecture. diff --git a/docs/requirements/system_requirements.md b/docs/requirements/system_requirements.md deleted file mode 100644 index bf1fb79795..0000000000 --- a/docs/requirements/system_requirements.md +++ /dev/null @@ -1,10 +0,0 @@ -# System Requirements - -To ensure the demo runs correctly please ensure your environment meets the -following system requirements: - -1. Your system must meet [Docker Desktop](https://docs.docker.com/desktop) - system requirements or you should use your preferred Cloud Service. - -2. The demo must be able to work on the following Operating Systems (OS): - Linux, macOS and Windows with documentation provided for each OS. diff --git a/docs/scenarios/recommendation_cache.md b/docs/scenarios/recommendation_cache.md deleted file mode 100644 index b6cf442e3e..0000000000 --- a/docs/scenarios/recommendation_cache.md +++ /dev/null @@ -1,79 +0,0 @@ -# Using Metrics and Traces to diagnose a memory leak - -Application telemetry, such as the kind that OpenTelemetry can provide, is very -useful for diagnosing issues in a distributed system. In this scenario, we will -walk through a scenario demonstrating how to move from high-level metrics and -traces to determine the cause of a memory leak. - -## Setup - -To run this scenario, you will need to deploy the demo application and enable -the `recommendationCache` feature flag. Let the application run for about 10 -minutes or so after enabling the feature flag to allow for data to populate. - -## Diagnosis - -The first step in diagnosing a problem is to determine that a problem exists. -Often the first stop will be a metrics dashboard provided by a tool such as -Grafana. - -A [demo dashboard folder](http://localhost:8080/grafana/dashboards) should exist -after launching the demo with two dashboards; One is to monitor your -OpenTelemetry Collector, and the other contains several queries and charts to -analyze latency and request rate from each service. - -![Grafana -Dashboard](https://user-images.githubusercontent.com/4140740/197369586-57b49c94-fef2-49d8-bcd0-9424b8312acc.png) - -This dashboard will contain a number of charts, but a few should appear -interesting: - -- Recommendation Service (CPU% and Memory) -- Service Latency (from SpanMetrics) -- Error Rate - -Recommendation Service charts are generated from OpenTelemetry Metrics exported -to Prometheus, while the Service Latency and Error Rate charts are generated -through the OpenTelemetry Collector Span Metrics processor. - -From our dashboard, we can see that there seems to be anomalous behavior in the -recommendation service -- spiky CPU utilization, as well as long tail latency in -our p95, 99, and 99.9 histograms. We can also see that there are intermittent -spikes in the memory utilization of this service. - -We know that we're emitting trace data from our application as well, so let's -think about another way that we'd be able to determine that a problem exist. - -![Jaeger](https://user-images.githubusercontent.com/4140740/197369715-6475902a-9601-4dda-b21a-db6160d6922a.png) - -Jaeger allows us to search for traces and display the end-to-end latency of an -entire request with visibility into each individual part of the overall request. -Perhaps we noticed an increase in tail latency on our frontend requests. Jaeger -allows us to then search and filter our traces to include only those that -include requests to recommendation service. - -By sorting by latency, we're able to quickly find specific traces that took a -long time. Clicking on a trace in the right panel, we're able to view the -waterfall view. - -![Jaeger -Waterfall](https://user-images.githubusercontent.com/4140740/197369866-4b27c02d-035d-4f3f-b369-6138b09a8d68.png) - -We can see that the recommendation service is taking a long time to complete its -work, and viewing the details allows us to get a better idea of what's going on. - -## Confirming the Diagnosis - -We can see in our waterfall view that the `app.cache_hit` attribute is set to -`false`, and that the `app.products.count` value is extremely high. - -Returning to the search UI, filter to `recommendationservice` in the Service -dropdown, and search for `app.cache_hit=true` in the Tags box. Notice that -requests tend to be faster when the cache is hit. Now search for -`app.cache_hit=false` and compare the latency. You should notice some changes in -the visualization at the top of the trace list. - -Now, since this is a contrived scenario, we know where to find the underlying -bug in our code. However, in a real-world scenario, we may need to perform -further searching to find out what's going on in our code, or the interactions -between services that cause it. diff --git a/docs/service_table.md b/docs/service_table.md deleted file mode 100644 index 69f3f5b4fb..0000000000 --- a/docs/service_table.md +++ /dev/null @@ -1,21 +0,0 @@ -# Service Roles - -View [Service Graph](./current_architecture.md) to visualize request flows. - -| Service | Language | Description | -|--------------------------------------------------------------|---------------|----------------------------------------------------------------------------------------------------------------------------------------------| -| [accountingservice](./services/accountingservice.md) | Go | Processes incoming orders and count the sum of all orders (mock). | -| [adservice](./services/adservice.md) | Java | Provides text ads based on given context words. | -| [cartservice](./services/cartservice.md) | DotNet | Stores the items in the user's shopping cart in Redis and retrieves it. | -| [checkoutservice](./services/checkoutservice.md) | Go | Retrieves user cart, prepares order and orchestrates the payment, shipping and the email notification. | -| [currencyservice](./services/currencyservice.md) | C++ | Converts one money amount to another currency. Uses real values fetched from European Central Bank. It's the highest QPS service. | -| [emailservice](./services/emailservice.md) | Ruby | Sends users an order confirmation email (mock). | -| [frauddetectionservice](./services/frauddetectionservice.md) | Kotlin | Analyzes incoming orders and detects fraud attempts (mock). | -| [featureflagservice](./services/featureflagservice.md) | Erlang/Elixir | CRUD feature flag service to demonstrate various scenarios like fault injection & how to emit telemetry from a feature flag reliant service. | -| [frontend](./services/frontend.md) | JavaScript | Exposes an HTTP server to serve the website. Does not require signup/login and generates session IDs for all users automatically. | -| [loadgenerator](./services/loadgenerator.md) | Python/Locust | Continuously sends requests imitating realistic user shopping flows to the frontend. | -| [paymentservice](./services/paymentservice.md) | JavaScript | Charges the given credit card info (mock) with the given amount and returns a transaction ID. | -| [productcatalogservice](./services/productcatalogservice.md) | Go | Provides the list of products from a JSON file and ability to search products and get individual products. | -| [quoteservice](./services/quoteservice.md) | PHP | Calculates the shipping costs, based on the number of items to be shipped. | -| [recommendationservice](./services/recommendationservice.md) | Python | Recommends other products based on what's given in the cart. | -| [shippingservice](./services/shippingservice.md) | Rust | Gives shipping cost estimates based on the shopping cart. Ships items to the given address (mock). | diff --git a/docs/services/accountingservice.md b/docs/services/accountingservice.md deleted file mode 100644 index 47180f5b4c..0000000000 --- a/docs/services/accountingservice.md +++ /dev/null @@ -1,59 +0,0 @@ -# Accounting Service - -This service calculates the total amount of sold products. -This is only mocked and received orders are printed out. - -[Accounting Service](../../src/accountingservice/) - -## Traces - -### Initializing Tracing - -The OpenTelemetry SDK is initialized from `main` using the `initTracerProvider` -function. - -```go -func initTracerProvider() (*sdktrace.TracerProvider, error) { - ctx := context.Background() - - exporter, err := otlptracegrpc.New(ctx) - if err != nil { - return nil, err - } - tp := sdktrace.NewTracerProvider( - sdktrace.WithBatcher(exporter), - sdktrace.WithResource(initResource()), - ) - otel.SetTracerProvider(tp) - otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{})) - return tp, nil -} -``` - -You should call `TracerProvider.Shutdown()` when your service is shutdown to -ensure all spans are exported. This service makes that call as part of a -deferred function in main - -```go - tp, err := initTracerProvider() - if err != nil { - log.Fatal(err) - } - defer func() { - if err := tp.Shutdown(context.Background()); err != nil { - log.Printf("Error shutting down tracer provider: %v", err) - } - }() -``` - -### Adding Kafka ( Sarama ) auto-instrumentation - -This service will receive the processed results of the Checkout Service via a -Kafka topic. -To instrument the Kafka client the ConsumerHandler implemented by the developer -has to be wrapped. - -```go - handler := groupHandler{} // implements sarama.ConsumerGroupHandler - wrappedHandler := otelsarama.WrapConsumerGroupHandler(&handler) -``` diff --git a/docs/services/adservice.md b/docs/services/adservice.md deleted file mode 100644 index 10b10911ee..0000000000 --- a/docs/services/adservice.md +++ /dev/null @@ -1,124 +0,0 @@ -# Ad Service - -This service determines appropriate ads to serve to users based on context -keys. The ads will be for products available in the store. - -[Ad service source](../../src/adservice/) - -## Auto-instrumentation - -This service relies on the OpenTelemetry Java Agent to automatically instrument -libraries such as gRPC, and to configure the OpenTelemetry SDK. The agent is -passed into the process using the `-javaagent` command line argument. Command -line arguments are added through the `JAVA_TOOL_OPTIONS` in the `Dockerfile`, -and leveraged during the automatically generated Gradle startup script. - -```dockerfile -ENV JAVA_TOOL_OPTIONS=-javaagent:/app/opentelemetry-javaagent.jar -``` - -## Traces - -### Add attributes to auto-instrumented spans - -Within the execution of auto-instrumented code you can get current span from -context. - -```java - Span span = Span.current(); -``` - -Adding attributes to a span is accomplished using `setAttribute` on the span -object. In the `getAds` function multiples attribute are added to the span. - -```java - span.setAttribute("app.ads.contextKeys", req.getContextKeysList().toString()); - span.setAttribute("app.ads.contextKeys.count", req.getContextKeysCount()); -``` - -### Add span events - -Adding an event to a span is accomplished using `addEvent` on the span object. -In the `getAds` function an event with an attribute is added when an exception -is caught. - -```java - span.addEvent("Error", Attributes.of(AttributeKey.stringKey("exception.message"), e.getMessage())); -``` - -### Setting span status - -If the result of the operation is an error, the span status should be set -accordingly using `setStatus` on the span object. In the `getAds` function the -span status is set when an exception is caught. - -```java - span.setStatus(StatusCode.ERROR); -``` - -### Create new spans - -New spans can be created and started using -`Tracer.spanBuilder("spanName").startSpan()`. Newly created spans should be set -into context using `Span.makeCurrent()`. The `getRandomAds` function will -create a new span, set it into context, perform an operation, and finally end -the span. - -```java - // create and start a new span manually - Tracer tracer = GlobalOpenTelemetry.getTracer("adservice"); - Span span = tracer.spanBuilder("getRandomAds").startSpan(); - - // put the span into context, so if any child span is started the parent will be set properly - try (Scope ignored = span.makeCurrent()) { - - Collection allAds = adsMap.values(); - for (int i = 0; i < MAX_ADS_TO_SERVE; i++) { - ads.add(Iterables.get(allAds, random.nextInt(allAds.size()))); - } - span.setAttribute("app.ads.count", ads.size()); - - } finally { - span.end(); - } -``` - -## Metrics - -### Initializing Metrics - -Similar to creating spans, the first step in creating metrics is initializing a -`Meter` instance, e.g. `GlobalOpenTelemetry.getMeter("adservice")`. From -there, use the various builder methods available on the `Meter` instance to -create the desired metric instrument, e.g.: - -```java -meter - .counterBuilder("app.ads.ad_requests") - .setDescription("Counts ad requests by request and response type") - .build(); -``` - -### Current Metrics Produced - -Note that all the metric names below appear in Prometheus/Grafana with `.` -characters transformed to `_`. - -#### Custom metrics - -The following custom metrics are currently available: - -* `app.ads.ad_requests`: A counter of ad requests with dimensions describing -whether the request was targeted with context keys or not, and whether the -response was targeted or random ads. - -#### Auto-instrumented metrics - -The following auto-instrumented metrics are available for the application: - -* [Runtime metrics for the JVM](https://opentelemetry.io/docs/reference/specification/metrics/semantic_conventions/runtime-environment-metrics/#jvm-metrics). -* [Latency metrics for RPCs](https://opentelemetry.io/docs/reference/specification/metrics/semantic_conventions/rpc-metrics/#rpc-server) - -## Logs - -TBD diff --git a/docs/services/cartservice.md b/docs/services/cartservice.md deleted file mode 100644 index bb54caac50..0000000000 --- a/docs/services/cartservice.md +++ /dev/null @@ -1,89 +0,0 @@ -# Cart Service - -This service maintains items placed in the shopping cart by users. It interacts -with a Redis caching service for fast access to shopping cart data. - -[Cart service source](../../src/cartservice/) - -> **Note** -> OpenTelemetry for .NET uses the `System.Diagnostic` library as its API -> in lieu of the standard OpenTelemetry API. - -## Traces - -### Initializing Tracing - -OpenTelemetry is configured in the .NET DI container. The -`AddOpenTelemetryTracing()` builder method is used to configure desired -instrumentation libraries, add exporters, and set other options. Configuration -of the exporter and resource attributes is performed through environment variables. - -```cs -services.AddOpenTelemetryTracing((builder) => builder - .ConfigureResource(r => r - .AddTelemetrySdk() - .AddEnvironmentVariableDetector() - .AddDetector(new DockerResourceDetector()) - ) - .AddRedisInstrumentation( - cartStore.GetConnection(), - options => options.SetVerboseDatabaseStatements = true) - .AddAspNetCoreInstrumentation() - .AddGrpcClientInstrumentation() - .AddHttpClientInstrumentation() - .AddOtlpExporter()); -``` - -### Add attributes to auto-instrumented spans - -Within the execution of auto-instrumented code you can get current span -(activity) from context. - -```cs - var activity = Activity.Current; -``` - -Adding attributes (tags in .NET) to a span (activity) is accomplished using -`SetTag` on the activity object. In the `AddItem` function from -`services/CartService.cs` several attributes are added to the auto-instrumented -span. - -```cs - activity?.SetTag("app.user.id", request.UserId); - activity?.SetTag("app.product.quantity", request.Item.Quantity); - activity?.SetTag("app.product.id", request.Item.ProductId); -``` - -### Add span events - -Adding span (activity) events is accomplished using `AddEvent` on the activity -object. In the `GetCart` function from `services/CartService.cs` a span event is -added. - -```cs - activity?.AddEvent(new("Fetch cart")); -``` - -## Metrics - -### Initializing Metrics - -Similar to configuring OpenTelemetry Traces, the .NET DI container requires a -call to `AddOpenTelemetryMetrics()`. This builder configures desired -instrumentation libraries, exporters, etc. - -```cs -services.AddOpenTelemetryMetrics(builder => builder - .ConfigureResource(r => r - .AddTelemetrySdk() - .AddEnvironmentVariableDetector() - .AddDetector(new DockerResourceDetector()) - ) - .AddRuntimeInstrumentation() - .AddAspNetCoreInstrumentation() - .AddOtlpExporter()); -``` - -## Logs - -TBD diff --git a/docs/services/checkoutservice.md b/docs/services/checkoutservice.md deleted file mode 100644 index 9b51c41000..0000000000 --- a/docs/services/checkoutservice.md +++ /dev/null @@ -1,174 +0,0 @@ -# Checkout Service - -This service is responsible to process a checkout order from the user. The -checkout service will call many other services in order to process an order. - -[Checkout service source](../../src/checkoutservice/) - -## Traces - -### Initializing Tracing - -The OpenTelemetry SDK is initialized from `main` using the `initTracerProvider` -function. - -```go -func initTracerProvider() *sdktrace.TracerProvider { - ctx := context.Background() - - exporter, err := otlptracegrpc.New(ctx) - if err != nil { - log.Fatal(err) - } - tp := sdktrace.NewTracerProvider( - sdktrace.WithBatcher(exporter), - sdktrace.WithResource(initResource()), - ) - otel.SetTracerProvider(tp) - otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{})) - return tp -} -``` - -You should call `TracerProvider.Shutdown()` when your service is shutdown to -ensure all spans are exported. This service makes that call as part of a -deferred function in main - -```go - tp := initTracerProvider() - defer func() { - if err := tp.Shutdown(context.Background()); err != nil { - log.Printf("Error shutting down tracer provider: %v", err) - } - }() -``` - -### Adding gRPC auto-instrumentation - -This service receives gRPC requests, which are instrumented in the main function -as part of the gRPC server creation. - -```go - var srv = grpc.NewServer( - grpc.UnaryInterceptor(otelgrpc.UnaryServerInterceptor()), - grpc.StreamInterceptor(otelgrpc.StreamServerInterceptor()), - ) -``` - -This service will issue several outgoing gRPC calls, which are all instrumented -by wrapping the gRPC client with instrumentation - -```go -func createClient(ctx context.Context, svcAddr string) (*grpc.ClientConn, error) { - return grpc.DialContext(ctx, svcAddr, - grpc.WithTransportCredentials(insecure.NewCredentials()), - grpc.WithUnaryInterceptor(otelgrpc.UnaryClientInterceptor()), - grpc.WithStreamInterceptor(otelgrpc.StreamClientInterceptor()), - ) -} -``` - -### Adding Kafka ( Sarama ) auto-instrumentation - -This service will write the processed results onto a Kafka topic which will then -be in turn be processed by other microservices. -To instrument the Kafka client the Producer has to be wrapped after it has been created. - -```go - saramaConfig := sarama.NewConfig() - producer, err := sarama.NewAsyncProducer(brokers, saramaConfig) - if err != nil { - return nil, err - } - producer = otelsarama.WrapAsyncProducer(saramaConfig, producer) -``` - -### Add attributes to auto-instrumented spans - -Within the execution of auto-instrumented code you can get current span from -context. - -```go - span := trace.SpanFromContext(ctx) -``` - -Adding attributes to a span is accomplished using `SetAttributes` on the span -object. In the `PlaceOrder` function several attributes are added to the span. - -```go - span.SetAttributes( - attribute.String("app.order.id", orderID.String()), shippingTrackingAttribute, - attribute.Float64("app.shipping.amount", shippingCostFloat), - attribute.Float64("app.order.amount", totalPriceFloat), - attribute.Int("app.order.items.count", len(prep.orderItems)), - ) -``` - -### Add span events - -Adding span events is accomplished using `AddEvent` on the span object. In the -`PlaceOrder` function several span events are added. Some events have -additional attributes, others do not. - -Adding a span event without attributes: - -```go - span.AddEvent("prepared") -``` - -Adding a span event with additional attributes: - -```go - span.AddEvent("charged", - trace.WithAttributes(attribute.String("app.payment.transaction.id", txID))) -``` - -## Metrics - -### Initializing Metrics - -The OpenTelemetry SDK is initialized from `main` using the `initMeterProvider` -function. - -```go -func initMeterProvider() *sdkmetric.MeterProvider { - ctx := context.Background() - - exporter, err := otlpmetricgrpc.New(ctx) - if err != nil { - log.Fatalf("new otlp metric grpc exporter failed: %v", err) - } - - mp := sdkmetric.NewMeterProvider(sdkmetric.WithReader(sdkmetric.NewPeriodicReader(exporter))) - global.SetMeterProvider(mp) - return mp -} -``` - -You should call `MeterProvider.Shutdown()` when your service is shutdown to -ensure all records are exported. This service makes that call as part of a -deferred function in main - -```go - mp := initMeterProvider() - defer func() { - if err := mp.Shutdown(context.Background()); err != nil { - log.Printf("Error shutting down meter provider: %v", err) - } - }() -``` - -### Adding golang runtime auto-instrumentation - -Golang runtime are instrumented in the main function - -```go - err := runtime.Start(runtime.WithMinimumReadMemStatsInterval(time.Second)) - if err != nil { - log.Fatal(err) - } -``` - -## Logs - -TBD diff --git a/docs/services/currencyservice.md b/docs/services/currencyservice.md deleted file mode 100644 index 091c85f6ed..0000000000 --- a/docs/services/currencyservice.md +++ /dev/null @@ -1,140 +0,0 @@ -# Currency Service - -This service provides functionality to convert amounts between different -currencies. - -[Currency service source](../../src/currencyservice/) - -## Traces - -### Initializing Tracing - -The OpenTelemetry SDK is initialized from `main` using the `initTracer` -function defined in `tacer_common.h` - -```cpp -void initTracer() -{ - auto exporter = opentelemetry::exporter::otlp::OtlpGrpcExporterFactory::Create(); - auto processor = - opentelemetry::sdk::trace::SimpleSpanProcessorFactory::Create(std::move(exporter)); - std::vector> processors; - processors.push_back(std::move(processor)); - std::shared_ptr context = - opentelemetry::sdk::trace::TracerContextFactory::Create(std::move(processors)); - std::shared_ptr provider = - opentelemetry::sdk::trace::TracerProviderFactory::Create(context); - // Set the global trace provider - opentelemetry::trace::Provider::SetTracerProvider(provider); - - // set global propagator - opentelemetry::context::propagation::GlobalTextMapPropagator::SetGlobalPropagator( - opentelemetry::nostd::shared_ptr( - new opentelemetry::trace::propagation::HttpTraceContext())); -} -``` - -### Create new spans - -New spans can be created and started using -`Tracer->StartSpan("spanName", attributes, options)`. After a span is created -you need to start and put it into active context using -`Tracer->WithActiveSpan(span)`. You can find an example of this in the `Convert` -function. - -```cpp - std::string span_name = "CurrencyService/Convert"; - auto span = - get_tracer("currencyservice")->StartSpan(span_name, - {{SemanticConventions::kRpcSystem, "grpc"}, - {SemanticConventions::kRpcService, "CurrencyService"}, - {SemanticConventions::kRpcMethod, "Convert"}, - {SemanticConventions::kRpcGrpcStatusCode, 0}}, - options); - auto scope = get_tracer("currencyservice")->WithActiveSpan(span); -``` - -### Adding attributes to spans - -You can add an attribute to a span using `Span->SetAttribute(key, value)`. - -```cpp - span->SetAttribute("app.currency.conversion.from", from_code); - span->SetAttribute("app.currency.conversion.to", to_code); -``` - -### Add span events - -Adding span events is accomplished using `Span->AddEvent(name)`. - -```cpp - span->AddEvent("Conversion successful, response sent back"); -``` - -### Set span status - -Make sure to set your span status to Ok, or Error accordingly. You can do this -using `Span->SetStatus(status)` - -```cpp - span->SetStatus(StatusCode::kOk); -``` - -### Tracing context propagation - -In C++ propagation is not automatically handled. You need to extract it from the -caller and inject the propagation context into subsequent spans. The -`GrpcServerCarrier` class defines a method to extract context from inbound gRPC -requests which is leveraged in the service call implementations. - -The `GrpcServerCarrier` class is defined in `tracer_common.h` as follows: - -```cpp -class GrpcServerCarrier : public opentelemetry::context::propagation::TextMapCarrier -{ -public: - GrpcServerCarrier(ServerContext *context) : context_(context) {} - GrpcServerCarrier() = default; - virtual opentelemetry::nostd::string_view Get( - opentelemetry::nostd::string_view key) const noexcept override - { - auto it = context_->client_metadata().find(key.data()); - if (it != context_->client_metadata().end()) - { - return it->second.data(); - } - return ""; - } - - virtual void Set(opentelemetry::nostd::string_view key, - opentelemetry::nostd::string_view value) noexcept override - { - // Not required for server - } - - ServerContext *context_; -}; -``` - -This class is leveraged in the `Convert` method to extract context and create a -`StartSpanOptions` object to contain the right context which is used when -creating new spans. - -```cpp - StartSpanOptions options; - options.kind = SpanKind::kServer; - GrpcServerCarrier carrier(context); - - auto prop = context::propagation::GlobalTextMapPropagator::GetGlobalPropagator(); - auto current_ctx = context::RuntimeContext::GetCurrent(); - auto new_context = prop->Extract(carrier, current_ctx); - options.parent = GetSpan(new_context)->GetContext(); -``` - -## Metrics - -TBD - -## Logs - -TBD diff --git a/docs/services/emailservice.md b/docs/services/emailservice.md deleted file mode 100644 index 075b19a037..0000000000 --- a/docs/services/emailservice.md +++ /dev/null @@ -1,76 +0,0 @@ -# Email Service - -This service will send a confirmation email to the user when an order is placed. - -[Email service source](../../src/emailservice/) - -## Initializing Tracing - -You will need to require the core OpenTelemetry SDK and exporter Ruby gems, as -well as any gem that will be needed for auto-instrumentation libraries -(ie: Sinatra) - -```ruby -require "opentelemetry/sdk" -require "opentelemetry/exporter/otlp" -require "opentelemetry/instrumentation/sinatra" -``` - -The Ruby SDK uses OpenTelemetry standard environment variables to configure -OTLP export, resource attributes, and service name automatically. When -initializing the OpenTelemetry SDK, you will also specify which -auto-instrumentation libraries to leverage (ie: Sinatra) - -```ruby -OpenTelemetry::SDK.configure do |c| - c.use "OpenTelemetry::Instrumentation::Sinatra" -end -``` - -## Traces - -### Add attributes to auto-instrumented spans - -Within the execution of auto-instrumented code you can get current span from -context. - -```ruby - current_span = OpenTelemetry::Trace.current_span -``` - -Adding multiple attributes to a span is accomplished using `add_attributes` on -the span object. - -```ruby - current_span.add_attributes({ - "app.order.id" => data.order.order_id, - }) -``` - -Adding only a single attribute can be accomplished using `set_attribute` on the -span object. - -```ruby - span.set_attribute("app.email.recipient", data.email) -``` - -### Create new spans - -New spans can be created and placed into active context using `in_span` from an -OpenTelemetry Tracer object. When used in conjunction with a `do..end` block, -the span will automatically be ended when the block ends execution. - -```ruby - tracer = OpenTelemetry.tracer_provider.tracer('emailservice') - tracer.in_span("send_email") do |span| - # logic in context of span here - end -``` - -## Metrics - -TBD - -## Logs - -TBD diff --git a/docs/services/featureflagservice.md b/docs/services/featureflagservice.md deleted file mode 100644 index 583fe3a34d..0000000000 --- a/docs/services/featureflagservice.md +++ /dev/null @@ -1,108 +0,0 @@ -# Feature Flag Service - -This service is written in Erlang/Elixir and it is responsible for creating, -reading, updating and deleting feature flags in a PostgreSQL DB. -It is called by Product Catalog and Shipping services. - -[Feature Flag Service Source](../../src/featureflagservice/) - -## Traces - -### Initializing Tracing - -In order to set up OpenTelemetry instrumentation for -[Phoenix](https://github.com/open-telemetry/opentelemetry-erlang-contrib/tree/main/instrumentation/opentelemetry_phoenix/), -and -[Ecto](https://github.com/open-telemetry/opentelemetry-erlang-contrib/tree/main/instrumentation/opentelemetry_ecto/), -, we need to call the setup methods of their instrumentation packages before -starting the Supervisor. - -This is done in the `application.ex` as follows: - -```elixir -@impl true - def start(_type, _args) do - OpentelemetryEcto.setup([:featureflagservice, :repo]) - OpentelemetryPhoenix.setup() - - children = [ - # Start the Ecto repository - Featureflagservice.Repo, - # Start the PubSub system - {Phoenix.PubSub, name: Featureflagservice.PubSub}, - # Start the Endpoint (http/https) - FeatureflagserviceWeb.Endpoint - # Start a worker by calling: Featureflagservice.Worker.start_link(arg) - # {Featureflagservice.Worker, arg} - ] - - # See https://hexdocs.pm/elixir/Supervisor.html - # for other strategies and supported options - opts = [strategy: :one_for_one, name: Featureflagservice.Supervisor] - Supervisor.start_link(children, opts) - end -``` - -To add tracing to [grpcbox](https://github.com/tsloughter/grpcbox), we need to -add the appropriate -[interceptor](https://github.com/open-telemetry/opentelemetry-erlang-contrib/tree/main/instrumentation/opentelemetry_grpcbox). - -This is configured in the `runtime.exs` file, as follows: - -```elixir -config :grpcbox, - servers: [ - %{ - :grpc_opts => %{ - :service_protos => [:ffs_demo_pb], - :unary_interceptor => {:otel_grpcbox_interceptor, :unary}, - :services => %{:"hipstershop.FeatureFlagService" => :ffs_service} - }, - :listen_opts => %{:port => grpc_port} - } - ] -``` - -### Add attributes to auto-instrumented spans - -Adding attributes to a span is accomplished by using `?set_attribute` on the span -object. In the `get_flag` function two attributes are added to the span. - -```elixir --include_lib("grpcbox/include/grpcbox.hrl"). - --include_lib("opentelemetry_api/include/otel_tracer.hrl"). - --spec get_flag(ctx:t(), ffs_demo_pb:get_flag_request()) -> - {ok, ffs_demo_pb:get_flag_response(), ctx:t()} | grpcbox_stream:grpc_error_response(). -get_flag(Ctx, #{name := Name}) -> - case 'Elixir.Featureflagservice.FeatureFlags':get_feature_flag_by_name(Name) of - nil -> - {grpc_error, {?GRPC_STATUS_NOT_FOUND, <<"the requested feature flag does not exist">>}}; - #{'__struct__' := 'Elixir.Featureflagservice.FeatureFlags.FeatureFlag', - description := Description, - enabled := Enabled, - inserted_at := CreatedAt, - updated_at := UpdatedAt - } -> - ?set_attribute('app.featureflag.name', Name), - ?set_attribute('app.featureflag.enabled', Enabled), - {ok, Epoch} = 'Elixir.NaiveDateTime':from_erl({{1970, 1, 1}, {0, 0, 0}}), - CreatedAtSeconds = 'Elixir.NaiveDateTime':diff(CreatedAt, Epoch), - UpdatedAtSeconds = 'Elixir.NaiveDateTime':diff(UpdatedAt, Epoch), - Flag = #{name => Name, - description => Description, - enabled => Enabled, - created_at => #{seconds => CreatedAtSeconds, nanos => 0}, - updated_at => #{seconds => UpdatedAtSeconds, nanos => 0}}, - {ok, #{flag => Flag}, Ctx} - end. -``` - -## Metrics - -TBD - -## Logs - -TBD diff --git a/docs/services/frauddetectionservice.md b/docs/services/frauddetectionservice.md deleted file mode 100644 index 5d2f34d79d..0000000000 --- a/docs/services/frauddetectionservice.md +++ /dev/null @@ -1,16 +0,0 @@ -# Fraud Detection Service - -This service analyses incoming orders and detects malicious customers. -This is only mocked and received orders are printed out. - -## Auto-instrumentation - -This service relies on the OpenTelemetry Java Agent to automatically instrument -libraries such as Kafka, and to configure the OpenTelemetry SDK. The agent is -passed into the process using the `-javaagent` command line argument. Command -line arguments are added through the `JAVA_TOOL_OPTIONS` in the `Dockerfile`, -and leveraged during the automatically generated Gradle startup script. - -```dockerfile -ENV JAVA_TOOL_OPTIONS=-javaagent:/app/opentelemetry-javaagent.jar -``` diff --git a/docs/services/frontend.md b/docs/services/frontend.md deleted file mode 100644 index 9727a37368..0000000000 --- a/docs/services/frontend.md +++ /dev/null @@ -1,197 +0,0 @@ -# Frontend - -The frontend is responsible to provide a UI for users, as well -as an API leveraged by the UI or other clients. The application is based on -[Next.JS](https://nextjs.org/) to provide a React web-based UI and API routes. - -[Frontend source](../../src/frontend/) - -## Server Instrumentation - -It is recommended to use a Node required module when starting your NodeJS -application to initialize the SDK and auto-instrumentation. When initializing -the OpenTelemetry NodeJS SDK, you optionally specify which auto-instrumentation -libraries to leverage, or make use of the `getNodeAutoInstrumentations()` -function which includes most popular frameworks. The `utils/telemetry/Instrumentation.js` -file contains all code required to initialize the SDK and auto-instrumentation -based on standard [OpenTelemetry environment variables](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md) -for OTLP export, resource attributes, and service name. - -```javascript -const opentelemetry = require("@opentelemetry/sdk-node") -const { getNodeAutoInstrumentations } = require("@opentelemetry/auto-instrumentations-node") -const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc') -const { alibabaCloudEcsDetector } = require('@opentelemetry/resource-detector-alibaba-cloud') -const { awsEc2Detector, awsEksDetector } = require('@opentelemetry/resource-detector-aws') -const { containerDetector } = require('@opentelemetry/resource-detector-container') -const { gcpDetector } = require('@opentelemetry/resource-detector-gcp') -const { envDetector, hostDetector, osDetector, processDetector } = require('@opentelemetry/resources') - -const sdk = new opentelemetry.NodeSDK({ - traceExporter: new OTLPTraceExporter(), - instrumentations: [ getNodeAutoInstrumentations() ], - resourceDetectors: [ - containerDetector, - envDetector, - hostDetector, - osDetector, - processDetector, - alibabaCloudEcsDetector, - awsEksDetector, - awsEc2Detector, - gcpDetector - ] -}) - -sdk.start() -``` - -Node required modules are loaded using the `--require` command line argument. -This can be done in the `scripts.start` section of `package.json` and starting -the application using `npm start`. - -```json - "scripts": { - "start": "node --require ./Instrumentation.js server.js", - }, -``` - -## Traces - -### Span Exceptions and status - -You can use the span object's `recordException` function to create a span event -with the full stack trace of a handled error. When recording an exception also -be sure to set the span's status accordingly. You can see this in the catch -block of the `NextApiHandler` function in the `utils/telemetry/InstrumentationMiddleware.ts` -file. - -```typescript - span.recordException(error as Exception); - span.setStatus({code: SpanStatusCode.ERROR}); -``` - -### Create new spans - -New spans can be created and started using `Tracer.startSpan("spanName", options)`. -Several options can be used to specify how the span can be created. - -- `root: true` will create a new trace, setting this span as the root. -- `links` are used to specify links to other spans (even within another trace) -that should be referenced. -- `attributes` are key/value pairs added to a span, typically used for -application context. - -```typescript - span = tracer.startSpan(`HTTP ${method}`, { - root: true, - kind: SpanKind.SERVER, - links: [{context: syntheticSpan.spanContext()}], - attributes: { - "app.synthetic_request": true, - [SemanticAttributes.HTTP_TARGET]: target, - [SemanticAttributes.HTTP_STATUS_CODE]: response.statusCode, - [SemanticAttributes.HTTP_METHOD]: method, - [SemanticAttributes.HTTP_USER_AGENT]: headers['user-agent'] || '', - [SemanticAttributes.HTTP_URL]: `${headers.host}${url}`, - [SemanticAttributes.HTTP_FLAVOR]: httpVersion, - } - }); -``` - -## Browser Instrumentation - -The web-based UI that the frontend provides is also instrumented for web -browsers. OpenTelemetry instrumentation is included as part of the Next.js App -component in `pages/_app.tsx`. Here instrumentation is imported and -initialized. - -```typescript -import FrontendTracer from '../utils/telemetry/FrontendTracer'; - -if (typeof window !== 'undefined') FrontendTracer(); -``` - -The `utils/telemetry/FrontendTracer.ts` file contains code to intialize a -TracerProvider, establish an OTLP export, register trace context propagators, -and register web specific auto-instrumentation libraries. Since the browser -will send data to an OpenTelemetry collector that will likely be on a separate -domain, CORS headers are also setup accordingly. - -As part of the changes to carry over the `synthetic_request` attribute flag for -the backend services, the `applyCustomAttributesOnSpan` configuration function -has been added to the `instrumentation-fetch` library custom span attributes logic -that way every browser-side span will include it. - -```typescript -import { CompositePropagator, W3CBaggagePropagator, W3CTraceContextPropagator } from '@opentelemetry/core'; -import { WebTracerProvider } from '@opentelemetry/sdk-trace-web'; -import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base'; -import { registerInstrumentations } from '@opentelemetry/instrumentation'; -import { getWebAutoInstrumentations } from '@opentelemetry/auto-instrumentations-web'; -import { Resource } from '@opentelemetry/resources'; -import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions'; -import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http'; - -const FrontendTracer = async () => { - const { ZoneContextManager } = await import('@opentelemetry/context-zone'); - - const provider = new WebTracerProvider({ - resource: new Resource({ - [SemanticResourceAttributes.SERVICE_NAME]: process.env.NEXT_PUBLIC_OTEL_SERVICE_NAME, - }), - }); - - provider.addSpanProcessor(new SimpleSpanProcessor(new OTLPTraceExporter())); - - const contextManager = new ZoneContextManager(); - - provider.register({ - contextManager, - propagator: new CompositePropagator({ - propagators: [new W3CBaggagePropagator(), new W3CTraceContextPropagator()], - }), - }); - - registerInstrumentations({ - tracerProvider: provider, - instrumentations: [ - getWebAutoInstrumentations({ - '@opentelemetry/instrumentation-fetch': { - propagateTraceHeaderCorsUrls: /.*/, - clearTimingResources: true, - applyCustomAttributesOnSpan(span) { - span.setAttribute('app.synthetic_request', 'false'); - }, - }, - }), - ], - }); -}; - -export default FrontendTracer; -``` - -## Metrics - -TBD - -## Logs - -TBD - -## Baggage - -OpenTelemetry Baggage is leveraged in the frontend to check if the request is -synthetic (from the load generator). Synthetic requests will force the creation -of a new trace. The root span from the new trace will contain many of the same -attributes as an HTTP request instrumented span. - -To determine if a Baggage item is set, you can leverage the `propagation` API -to parse the Baggage header, and leverage the `baggage` API to get or -set entries. - -```typescript - const baggage = propagation.getBaggage(context.active()); - if (baggage?.getEntry("synthetic_request")?.value == "true") {...} -``` diff --git a/docs/services/frontendproxy.md b/docs/services/frontendproxy.md deleted file mode 100644 index 9eca9fe3d2..0000000000 --- a/docs/services/frontendproxy.md +++ /dev/null @@ -1,60 +0,0 @@ -# Frontend Proxy(Envoy) - -The frontend proxy is used as a reverse proxy for user-facing web -interfaces such as the frontend, Jaeger, Grafana, load generator, -and feature flag service. - -## Enabling OpenTelemetry - -**NOTE: Only non-synthetic requests will trigger the envoy tracing.** - -In order to enable Envoy to produce spans whenever receiving a request, -the following configuration is required: - -```yaml -static_resources: - listeners: - - address: - socket_address: - address: 0.0.0.0 - port_value: ${ENVOY_PORT} - filter_chains: - - filters: - - name: envoy.filters.network.http_connection_manager - typed_config: - "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager - codec_type: AUTO - stat_prefix: ingress_http - tracing: - provider: - name: envoy.tracers.opentelemetry - typed_config: - "@type": type.googleapis.com/envoy.config.trace.v3.OpenTelemetryConfig - grpc_service: - envoy_grpc: - cluster_name: opentelemetry_collector - timeout: 0.250s - service_name: frontend-proxy - - clusters: - - name: opentelemetry_collector - type: STRICT_DNS - lb_policy: ROUND_ROBIN - typed_extension_protocol_options: - envoy.extensions.upstreams.http.v3.HttpProtocolOptions: - "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions - explicit_http_config: - http2_protocol_options: {} - load_assignment: - cluster_name: opentelemetry_collector - endpoints: - - lb_endpoints: - - endpoint: - address: - socket_address: - address: ${OTEL_COLLECTOR_HOST} - port_value: ${OTEL_COLLECTOR_PORT} -``` - -Where `OTEL_COLLECTOR_HOST` and `OTEL_COLLECTOR_PORT` are passed via -environment variables. diff --git a/docs/services/kafka.md b/docs/services/kafka.md deleted file mode 100644 index 023ffe30b8..0000000000 --- a/docs/services/kafka.md +++ /dev/null @@ -1,21 +0,0 @@ -# Kafka - -This is used as a message queue service to connect the checkout service with -the accounting and fraud detection services. - -[Kafka service source](../../src/kafka/) - -## Auto-instrumentation - -This service relies on the OpenTelemetry Java Agent and the built in -[JMX Metric Insight Module](https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/jmx-metrics/javaagent) -to capture [kafka broker metrics](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/instrumentation/jmx-metrics/javaagent/kafka-broker.md) -and send them off to the collector via OTLP. - -The agent is passed into the process using the `-javaagent` command line -argument. Command line arguments are added through the `KAFKA_OPTS` -in the `Dockerfile`. - -```dockerfile -ENV KAFKA_OPTS="-javaagent:/tmp/opentelemetry-javaagent.jar -Dotel.jmx.target.system=kafka-broker" -``` diff --git a/docs/services/loadgenerator.md b/docs/services/loadgenerator.md deleted file mode 100644 index ee2ea45dbc..0000000000 --- a/docs/services/loadgenerator.md +++ /dev/null @@ -1,65 +0,0 @@ -# Load Generator - -The load generator is based on the Python load testing framework [Locust](https://locust.io). -By default it will simulate users requesting several different routes from the -frontend. - -[Load generator source](../../src/loadgenerator/) - -## Traces - -### Initializing Tracing - -Since this service is a [locustfile](https://docs.locust.io/en/stable/writing-a-locustfile.html), -the The OpenTelemetry SDK is initialized after the import statements. This code -will create a tracer provider, and establish a Span Processor to use. Export -endpoints, resource attributes, and service name are automatically set using -[OpenTelemetry environment variables](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md). - -```python -tracer_provider = TracerProvider() -trace.set_tracer_provider(tracer_provider) -tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter())) -``` - -### Adding instrumentation libraries - -To add instrumentation libraries you need to import the Instrumentors for each -library in your Python code. Locust uses the `Requests` and`URLLib3` libraries, -so we will import their Instrumentors. - -```python -from opentelemetry.instrumentation.requests import RequestsInstrumentor -from opentelemetry.instrumentation.urllib3 import URLLib3Instrumentor -``` - -In your code before the library is leveraged, the Instrumentor needs to be -initialized by calling `instrument()`. - -```python -RequestsInstrumentor().instrument() -URLLib3Instrumentor().instrument() -``` - -Once initialized, every Locust requests for this load generator will have their -own trace with a span for each of the `Requests` and `URLLib3` libraries. - -## Metrics - -TBD - -## Logs - -TBD - -## Baggage - -OpenTelemetry Baggage is used by the load generator to indicate that the traces -are synthetically generated. This is done in the `on_start` function by creating -a context object containing the baggage item, and associating that context for -all tasks by the load generator. - -```python - ctx = baggage.set_baggage("synthetic_request", "true") - context.attach(ctx) -``` diff --git a/docs/services/paymentservice.md b/docs/services/paymentservice.md deleted file mode 100644 index 1702464fd5..0000000000 --- a/docs/services/paymentservice.md +++ /dev/null @@ -1,133 +0,0 @@ -# Payment Service - -This service is responsible to process credit card payments for orders. It will -return an error if the credit card is invalid or the payment can not be -processed. - -[Payment service source](../../src/paymentservice/) - -## Initializing OpenTelemetry - -It is recommended to `require` Node.js app using an initializer file that -initializes the SDK and auto-instrumentation. When initializing the -OpenTelemetry NodeJS SDK in that module, you optionally specify which -auto-instrumentation libraries to leverage, or make use of the -`getNodeAutoInstrumentations()` function which includes most popular frameworks. -The below example of an intiailizer file (`opentelemetry.js`) contains all code -required to initialize the SDK and auto-instrumentation based on standard -OpenTelemetry environment variables for OTLP export, resource attributes, and -service name. It then `require`s your app at `./index.js` to start it up once -the SDK is initialized. - -```javascript -const opentelemetry = require("@opentelemetry/sdk-node") -const { getNodeAutoInstrumentations } = require("@opentelemetry/auto-instrumentations-node") -const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc') -const { OTLPMetricExporter } = require('@opentelemetry/exporter-metrics-otlp-grpc') -const { PeriodicExportingMetricReader } = require('@opentelemetry/sdk-metrics') -const { alibabaCloudEcsDetector } = require('@opentelemetry/resource-detector-alibaba-cloud') -const { awsEc2Detector, awsEksDetector } = require('@opentelemetry/resource-detector-aws') -const { containerDetector } = require('@opentelemetry/resource-detector-container') -const { gcpDetector } = require('@opentelemetry/resource-detector-gcp') -const { envDetector, hostDetector, osDetector, processDetector } = require('@opentelemetry/resources') - -const sdk = new opentelemetry.NodeSDK({ - traceExporter: new OTLPTraceExporter(), - instrumentations: [ getNodeAutoInstrumentations() ], - metricReader: new PeriodicExportingMetricReader({ - exporter: new OTLPMetricExporter() - }), - resourceDetectors: [ - containerDetector, - envDetector, - hostDetector, - osDetector, - processDetector, - alibabaCloudEcsDetector, - awsEksDetector, - awsEc2Detector, - gcpDetector - ], -}) - -sdk.start().then(() => require("./index")); -``` - -You can then use `opentelemetry.js` to start your app. -This can be done in the `ENTRYPOINT` command for the service's `Dockerfile`. - -```dockerfile -ENTRYPOINT [ "node", "./opentelemetry.js" ] -``` - -## Traces - -### Add attributes to auto-instrumented spans - -Within the execution of auto-instrumented code you can get current span from -context. - -```javascript - const span = opentelemetry.trace.getActiveSpan(); -``` - -Adding attributes to a span is accomplished using `setAttributes` on the span -object. In the `chargeServiceHandler` function an attributes is added to -the span as an anonymous object (map) for the attribute key/values pair. - -```javascript - span.setAttributes({ - 'app.payment.amount': parseFloat(`${amount.units}.${amount.nanos}`) - }) -``` - -### Span Exceptions and status - -You can use the span object's `recordException` function to create a span event -with the full stack trace of a handled error. When recording an exception also -be sure to set the span's status accordingly. You can see this in the -`chargeServiceHandler` function - -```javascript - span.recordException(err) - span.setStatus({ code: opentelemetry.SpanStatusCode.ERROR }) -``` - -## Metrics - -### Creating Meters and Instruments - -Meters can be created using the `@opentelemetry/api-metrics` package. You can -create meters as seen below, and then use the created meter to create -instruments. - -```javascript -const { metrics } = require('@opentelemetry/api-metrics'); - -const meter = metrics.getMeter('paymentservice'); -const transactionsCounter = meter.createCounter('app.payment.transactions') -``` - -Meters and Instruments are supposed to stick around. This means you should -get a Meter or an Instrument once , and then re-use it as needed, if possible. - -## Logs - -TBD - -## Baggage - -OpenTelemetry Baggage is leveraged in this service to check if the request is -synthetic (from the load generator). Synthetic requests will not be charged, -which is indicated with a span attribute. The `charge.js` file which does the -actual payment processing, has logic to check the baggage. - -```javascript - // check baggage for synthetic_request=true, and add charged attribute accordingly - const baggage = propagation.getBaggage(context.active()); - if (baggage && baggage.getEntry("synthetic_request") && baggage.getEntry("synthetic_request").value == "true") { - span.setAttribute('app.payment.charged', false); - } else { - span.setAttribute('app.payment.charged', true); - } -``` diff --git a/docs/services/productcatalogservice.md b/docs/services/productcatalogservice.md deleted file mode 100644 index 24e3548f1e..0000000000 --- a/docs/services/productcatalogservice.md +++ /dev/null @@ -1,160 +0,0 @@ -# Product Catalog Service - -This service is responsible to return information about products. The service -can be used to get all products, search for specific products, or return details -about any single product. - -[Product Catalog service source](../../src/productcatalogservice/) - -## Traces - -### Initializing Tracing - -The OpenTelemetry SDK is initialized from `main` using the `initTracerProvider` -function. - -```go -func initTracerProvider() *sdktrace.TracerProvider { - ctx := context.Background() - - exporter, err := otlptracegrpc.New(ctx) - if err != nil { - log.Fatalf("OTLP Trace gRPC Creation: %v", err) - } - tp := sdktrace.NewTracerProvider( - sdktrace.WithBatcher(exporter), - sdktrace.WithResource(initResource()), - ) - otel.SetTracerProvider(tp) - otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{})) - return tp -} -``` - -You should call `TracerProvider.Shutdown()` when your service is shutdown to -ensure all spans are exported. This service makes that call as part of a -deferred function in main - -```go - tp := InitTracerProvider() - defer func() { - if err := tp.Shutdown(context.Background()); err != nil { - log.Fatalf("Tracer Provider Shutdown: %v", err) - } - }() -``` - -### Adding gRPC auto-instrumentation - -This service receives gRPC requests, which are instrumented in the main function -as part of the gRPC server creation. - -```go - srv := grpc.NewServer( - grpc.UnaryInterceptor(otelgrpc.UnaryServerInterceptor()), - grpc.StreamInterceptor(otelgrpc.StreamServerInterceptor()), - ) -``` - -This service will issue outgoing gRPC calls, which are all instrumented by -wrapping the gRPC client with instrumentation. - -```go -func createClient(ctx context.Context, svcAddr string) (*grpc.ClientConn, error) { - return grpc.DialContext(ctx, svcAddr, - grpc.WithTransportCredentials(insecure.NewCredentials()), - grpc.WithUnaryInterceptor(otelgrpc.UnaryClientInterceptor()), - grpc.WithStreamInterceptor(otelgrpc.StreamClientInterceptor()), - ) -} -``` - -### Add attributes to auto-instrumented spans - -Within the execution of auto-instrumented code you can get current span from -context. - -```go - span := trace.SpanFromContext(ctx) -``` - -Adding attributes to a span is accomplished using `SetAttributes` on the span -object. In the `GetProduct` function an attribute for the product id is added -to the span. - -```go - span.SetAttributes( - attribute.String("app.product.id", req.Id), - ) -``` - -### Setting span status - -This service can catch and handle an error condition based on a feature flag. -In an error condition, the span status is set accordingly using `SetStatus` on -the span object. You can see this in the `GetProduct` function. - -```go - msg := fmt.Sprintf("Error: ProductCatalogService Fail Feature Flag Enabled") - span.SetStatus(otelcodes.Error, msg) -``` - -### Add span events - -Adding span events is accomplished using `AddEvent` on the span object. In the -`GetProduct` function a span event is added when an error condition is handled, -or when a product is successfully found. - -```go - span.AddEvent(msg) -``` - -## Metrics - -### Initializing Metrics - -The OpenTelemetry SDK is initialized from `main` using the `initMeterProvider` -function. - -```go -func initMeterProvider() *sdkmetric.MeterProvider { - ctx := context.Background() - - exporter, err := otlpmetricgrpc.New(ctx) - if err != nil { - log.Fatalf("new otlp metric grpc exporter failed: %v", err) - } - - mp := sdkmetric.NewMeterProvider(sdkmetric.WithReader(sdkmetric.NewPeriodicReader(exporter))) - global.SetMeterProvider(mp) - return mp -} -``` - -You should call `initMeterProvider.Shutdown()` when your service is shutdown to -ensure all records are exported. This service makes that call as part of a -deferred function in main. - -```go - mp := initMeterProvider() - defer func() { - if err := mp.Shutdown(context.Background()); err != nil { - log.Fatalf("Error shutting down meter provider: %v", err) - } - }() -``` - -### Adding golang runtime auto-instrumentation - -Golang runtime is instrumented in the main function - -```go - err := runtime.Start(runtime.WithMinimumReadMemStatsInterval(time.Second)) - if err != nil { - log.Fatal(err) - } -``` - -## Logs - -TBD diff --git a/docs/services/quoteservice.md b/docs/services/quoteservice.md deleted file mode 100644 index d4b32fa5d6..0000000000 --- a/docs/services/quoteservice.md +++ /dev/null @@ -1,92 +0,0 @@ -# Quote Service - -This service is responsible for calculating shipping costs, based on -the number of items to be shipped. The quote service is called from -Shipping Service via HTTP. - -The Quote Service is implemented using the Slim framework and -php-di for managing the Dependency Injection. - -The PHP instrumentation may vary when using a different framework. - -[Quote service source](../../src/quoteservice/) - -## Traces - -### Initializing Tracing - -In this demo, the OpenTelemetry SDK has been automatically created as part -of SDK autoloading, which happens as part of composer autoloading. - -This is enabled by setting the environment variable `OTEL_PHP_AUTOLOAD_ENABLED=true`. - -```php - require __DIR__ . '/../vendor/autoload.php'; -``` - -There are multiple ways to create or obtain a `Tracer`, in this example we -obtain one from the global tracer provider which was initialized above, as -part of SDK autoloading: - -```php - $tracer = Globals::tracerProvider()->getTracer('manual-instrumentation'); -``` - -### Manually creating spans - -Creating a span manually can be done via a `Tracer`. The span will be default -be a child of the active span in the current execution context: - -```php - $span = Globals::tracerProvider() - ->getTracer('manual-instrumentation') - ->spanBuilder('calculate-quote') - ->setSpanKind(SpanKind::KIND_INTERNAL) - ->startSpan(); - /* calculate quote */ - $span->end(); -``` - -### Add span attributes - -You can obtain the current span using `OpenTelemetry\API\Trace\Span`. - -```php - $span = Span::getCurrent(); -``` - -Adding attributes to a span is accomplished using `setAttribute` on the span -object. In the `calculateQuote` function 2 attributes are added to the `childSpan`. - -```php - $childSpan->setAttribute('app.quote.items.count', $numberOfItems); - $childSpan->setAttribute('app.quote.cost.total', $quote); -``` - -### Add span events - -Adding span events is accomplished using `addEvent` on the span object. In the -`getquote` route span events are added. Some events have -additional attributes, others do not. - -Adding a span event without attributes: - -```php - $span->addEvent('Received get quote request, processing it'); -``` - -Adding a span event with additional attributes: - -```php - $span->addEvent('Quote processed, response sent back', [ - 'app.quote.cost.total' => $payload - ]); -``` - -## Metrics - -TBD - -## Logs - -TBD diff --git a/docs/services/recommendationservice.md b/docs/services/recommendationservice.md deleted file mode 100644 index 66d3225939..0000000000 --- a/docs/services/recommendationservice.md +++ /dev/null @@ -1,91 +0,0 @@ -# Recommendation Service - -This service is responsible to get a list of recommended products for the user -based on existing product ids the user is browsing. - -[Recommendation service source](../../src/recommendationservice/) - -## Auto-instrumentation - -This Python based service, makes use of the OpenTelemetry auto-instrumentor -for Python, accomplished by leveraging the `opentelemetry-instrument` Python -wrapper to run the scripts. This can be done in the `ENTRYPOINT` command for the -service's `Dockerfile`. - -```dockerfile -ENTRYPOINT [ "opentelemetry-instrument", "python", "recommendation_server.py" ] -``` - -## Traces - -### Initializing Tracing - -The OpenTelemetry SDK is initialized in the `__main__` code block. This code -will create a tracer provider, and establish a Span Processor to use. Export -endpoints, resource attributes, and service name are automatically set by the -OpenTelemetry auto instrumentor based on environment variables. - -```python - tracer = trace.get_tracer_provider().get_tracer("recommendationservice") -``` - -### Add attributes to auto-instrumented spans - -Within the execution of auto-instrumented code you can get current span from -context. - -```python - span = trace.get_current_span() -``` - -Adding attributes to a span is accomplished using `set_attribute` on the span -object. In the `ListRecommendations` function an attribute is added to the span. - -```python - span.set_attribute("app.products_recommended.count", len(prod_list)) -``` - -### Create new spans - -New spans can be created and placed into active context using -`start_as_current_span` from an OpenTelemetry Tracer object. When used in -conjunction with a `with` block, the span will automatically be ended when the -block ends execution. This is done in the `get_product_list` function. - -```python - with tracer.start_as_current_span("get_product_list") as span: -``` - -## Metrics - -### Initializing Metrics - -The OpenTelemetry SDK is initialized in the `__main__` code block. This code -will create a meter provider. Export -endpoints, resource attributes, and service name are automatically set by the -OpenTelemetry auto instrumentor based on environment variables. - -```python - meter = metrics.get_meter_provider().get_meter("recommendationservice") -``` - -### Custom metrics - -The following custom metrics are currently available: - -* `app_recommendations_counter`: Cumulative count of # recommended - products per service call - -### Auto-instrumented metrics - -The following metrics are available through auto-instrumentation, courtesy of -the `opentelemetry-instrumentation-system-metrics`, which is installed as part -of `opentelemetry-bootstrap` on building the recommendationservice Docker image: - -* `runtime.cpython.cpu_time` -* `runtime.cpython.memory` -* `runtime.cpython.gc_count` - -## Logs - -TBD diff --git a/docs/services/shippingservice.md b/docs/services/shippingservice.md deleted file mode 100644 index d64a3b75ab..0000000000 --- a/docs/services/shippingservice.md +++ /dev/null @@ -1,176 +0,0 @@ -# Shipping Service - -This service is responsible for providing shipping information including pricing -and tracking information, when requested from Checkout Service. - -Shipping service is built primarily with Tonic, Reqwest, and OpenTelemetry -Libraries/Components. Other sub-dependencies are included in `Cargo.toml`. - -Depending on your framework and runtime, you may consider consulting -[rust docs](https://opentelemetry.io/docs/instrumentation/rust/) to supplement. -You'll find examples of async and sync spans in quote requests and tracking ID's -respectively. - -The `build.rs` supports development outside docker, given a rust installation. -Otherwise, consider building with `docker compose` to edit / assess changes as needed. - -[Shipping service source](../../src/shippingservice/) - -## Traces - -### Initializing Tracing - -The OpenTelemetry SDK is initialized from `main`. - -```rust -fn init_tracer() -> Result { - global::set_text_map_propagator(TraceContextPropagator::new()); - let os_resource = OsResourceDetector.detect(Duration::from_secs(0)); - let process_resource = ProcessResourceDetector.detect(Duration::from_secs(0)); - let sdk_resource = SdkProvidedResourceDetector.detect(Duration::from_secs(0)); - let env_resource = EnvResourceDetector::new().detect(Duration::from_secs(0)); - opentelemetry_otlp::new_pipeline() - .tracing() - .with_exporter( - opentelemetry_otlp::new_exporter() - .tonic() - .with_endpoint(format!( - "{}{}", - env::var("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT") - .unwrap_or_else(|_| "http://otelcol:4317".to_string()), - "/v1/traces" - )), // TODO: assume this ^ is true from config when opentelemetry crate > v0.17.0 - // https://github.com/open-telemetry/opentelemetry-rust/pull/806 includes the environment variable. - ) - .with_trace_config( - sdktrace::config() - .with_resource(os_resource.merge(&process_resource).merge(&sdk_resource).merge(&env_resource)), - ) - .install_batch(opentelemetry::runtime::Tokio) -} -``` - -Spans and other metrics are created in this example throughout `tokio` async -runtimes found within [`tonic` server -functions](https://github.com/hyperium/tonic/blob/master/examples/helloworld-tutorial.md#writing-our-server). -Be mindful of async runtime, [context -guards](https://docs.rs/opentelemetry/latest/opentelemetry/struct.ContextGuard.html), -and inability to move and clone `spans` when replicating from these samples. - -### Adding gRPC instrumentation - -This service receives gRPC requests, which are instrumented in the middleware. - -The root span is started and passed down as reference in the same thread -to another closure where we call `quoteservice`. - -```rust - let tracer = global::tracer("shippingservice"); - let mut span = tracer.span_builder("hipstershop.ShippingService/GetQuote").with_kind(SpanKind::Server).start_with_context(&tracer, &parent_cx); - span.set_attribute(semcov::trace::RPC_SYSTEM.string(RPC_SYSTEM_GRPC)); - - span.add_event("Processing get quote request".to_string(), vec![]); - - let cx = Context::current_with_span(span); - let q = match create_quote_from_count(itemct) - .with_context(cx.clone()) - .await -//-> create_quote_from_count()... - let f = match request_quote(count).await { - Ok(float) => float, - Err(err) => { - let msg = format!("{}", err); - return Err(tonic::Status::unknown(msg)); - } - }; - - Ok(get_active_span(|span| { - let q = create_quote_from_float(f); - span.add_event( - "Received Quote".to_string(), - vec![KeyValue::new("app.shipping.cost.total", format!("{}", q))], - ); - span.set_attribute(KeyValue::new("app.shipping.items.count", count as i64)); - span.set_attribute(KeyValue::new("app.shipping.cost.total", format!("{}", q))); - q - })) -//<- create_quote_from_count()... - cx.span().set_attribute(semcov::trace::RPC_GRPC_STATUS_CODE.i64(RPC_GRPC_STATUS_CODE_OK)); -``` - -Note that we create a context around the root span and send a clone to the -async function create_quote_from_count(). After create_quote_from_count() -completes, we can add additional attributes to the root span as appropriate. - -You may also notice the `attributes` set on the span in this example, and -`events` propogated similarly. With any valid `span` pointer (attached to -context) the [OpenTelemetry API](https://docs.rs/opentelemetry/0.17.0/opentelemetry/trace/struct.SpanRef.html) -will work. - -### Adding HTTP instrumentation - -A child *client* span is also produced for the outoing HTTP call to -`quoteservice` via the `reqwest` client. This span pairs up with the -corresponding `quoteservice` *server* span. The tracing instrumentation is -implemented in the client middleware making use of the available -`reqwest-middleware`, `reqwest-tracing` and `tracing-opentelementry` libraries: - -```rust - let reqwest_client = reqwest::Client::new(); - let client = ClientBuilder::new(reqwest_client) - .with(TracingMiddleware::::new()) - .build(); -``` - -### Add span attributes - -Provided you are on the same thread, or in a context passed from a -span-owning thread, or a `ContextGuard` is in scope, you can get -an active span with `get_active_span`. You can find examples of all of these -in the demo, with context available in `shipping_service` for sync/async runtime. -You should consult `quote.rs` and/or the example above to see -context-passed-to-async runtime. - -See below for a snippet from `shiporder` that holds context and a span in scope. -This is appropriate in our case of a sync runtime. - -```rust - let parent_cx = - global::get_text_map_propagator(|prop| prop.extract(&MetadataMap(request.metadata()))); - // in this case, generating a tracking ID is trivial - // we'll create a span and associated events all in this function. - let tracer = global::tracer("shippingservice"); - let mut span = tracer - .span_builder("hipstershop.ShippingService/ShipOrder").with_kind(SpanKind::Server).start_with_context(&tracer, &parent_cx); -``` - -You must add attributes to a span in context with `set_attribute`, followed by a -`KeyValue` object, containing a key, and value. - -```rust - let tid = create_tracking_id(); - span.set_attribute(KeyValue::new("app.shipping.tracking.id", tid.clone())); - info!("Tracking ID Created: {}", tid); -``` - -### Add span events - -Adding span events is accomplished using `add_event` on the span object. Both -server routes, for `ShipOrderRequest` (sync) and `GetQuoteRequest` (async), -have events on spans. Attributes are not included here, but are [simple to include](https://docs.rs/opentelemetry/latest/opentelemetry/trace/trait.Span.html#method.add_event). - -Adding a span event: - -```rust - let tid = create_tracking_id(); - span.set_attribute(KeyValue::new("app.shipping.tracking.id", tid.clone())); - info!("Tracking ID Created: {}", tid); -``` - -## Metrics - -TBD - -## Logs - -TBD diff --git a/docs/tests.md b/docs/tests.md deleted file mode 100644 index 42604d2682..0000000000 --- a/docs/tests.md +++ /dev/null @@ -1,12 +0,0 @@ -# Tests - -Currently, the repository includes E2E tests for both the frontend and backend services. -For the Frontend we are using [Cypress](https://www.cypress.io/) execute the -different flows in the webstore. -While the backend services use [AVA](https://avajs.dev) as the main testing framework. - -To run the test you can simply run `make run-tests` at the root directory. - -In case you need to run a specific suite of tests you can execute -`docker compose run frontendTests` for the frontend tests or -`docker compose run integrationTests` for the backend tests. diff --git a/docs/trace_service_features.md b/docs/trace_service_features.md deleted file mode 100644 index 9a4adf216a..0000000000 --- a/docs/trace_service_features.md +++ /dev/null @@ -1,24 +0,0 @@ -# Trace Feature Coverage by Service - -Emoji Legend - -- Completed: :100: -- Not Applicable: :no_bell: -- Not Present (Yet): :construction: - -| Service | Language | Instrumentation Libraries | Manual Span Creation | Span Data Enrichment | RPC Context Propagation | Span Links | Baggage | Resource Detection | -|--------------------|-----------------|---------------------------|----------------------|----------------------|-------------------------|----------------|----------------|--------------------| -| Accounting Service | Go | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | :100: | -| Ad | Java | :100: | :100: | :100: | :no_bell: | :no_bell: | :no_bell: | :construction: | -| Cart | .NET | :100: | :100: | :100: | :no_bell: | :no_bell: | :no_bell: | :100: | -| Checkout | Go | :100: | :100: | :100: | :no_bell: | :no_bell: | :no_bell: | :100: | -| Currency | C++ | :no_bell: | :100: | :100: | :100: | :no_bell: | :no_bell: | :construction: | -| Email | Ruby | :100: | :100: | :100: | :no_bell: | :no_bell: | :no_bell: | :construction: | -| Feature Flag | Erlang / Elixir | :100: | :100: | :100: | :no_bell: | :no_bell: | :no_bell: | :construction: | -| Fraud Detection | Kotlin | :100: | :construction: | :construction: | :construction: | :construction: | :construction: | :construction: | -| Frontend | JavaScript | :100: | :100: | :100: | :no_bell: | :100: | :100: | :100: | -| Payment | JavaScript | :100: | :100: | :100: | :no_bell: | :no_bell: | :100: | :100: | -| Product Catalog | Go | :100: | :no_bell: | :100: | :no_bell: | :no_bell: | :no_bell: | :construction: | -| Quote Service | PHP | :100: | :100: | :100: | :no_bell: | :no_bell: | :no_bell: | :construction: | -| Recommendation | Python | :100: | :100: | :100: | :no_bell: | :no_bell: | :no_bell: | :construction: | -| Shipping | Rust | :no_bell: | :100: | :100: | :100: | :no_bell: | :no_bell: | :construction: |