diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 62541218442..ee59fe0158b 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -217,6 +217,24 @@ updates: schedule: interval: weekly day: sunday + - package-ecosystem: gomod + directory: /log + labels: + - dependencies + - go + - Skip Changelog + schedule: + interval: weekly + day: sunday + - package-ecosystem: gomod + directory: /log/internal + labels: + - dependencies + - go + - Skip Changelog + schedule: + interval: weekly + day: sunday - package-ecosystem: gomod directory: /metric labels: diff --git a/CHANGELOG.md b/CHANGELOG.md index 6147c177966..26385388d88 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ### Added +- Add `go.opentelemetry.io/otel/log` OpenTelemetry Bridge API module. (#4798) - The `go.opentelemetry.io/otel/semconv/v1.22.0` package. The package contains semantic conventions from the `v1.22.0` version of the OpenTelemetry Semantic Conventions. (#4735) - The `go.opentelemetry.io/otel/semconv/v1.23.0` package. diff --git a/Makefile b/Makefile index 35fc189961b..2cedd200d3c 100644 --- a/Makefile +++ b/Makefile @@ -315,4 +315,4 @@ add-tags: | $(MULTIMOD) .PHONY: lint-markdown lint-markdown: - docker run -v "$(CURDIR):$(WORKDIR)" docker://avtodev/markdown-lint:v1 -c $(WORKDIR)/.markdownlint.yaml $(WORKDIR)/**/*.md + docker run -v "$(CURDIR):$(WORKDIR)" avtodev/markdown-lint:v1 -c $(WORKDIR)/.markdownlint.yaml $(WORKDIR)/**/*.md diff --git a/log/DESIGN.md b/log/DESIGN.md new file mode 100644 index 00000000000..1e83fe8e32f --- /dev/null +++ b/log/DESIGN.md @@ -0,0 +1,301 @@ +# Logs Bridge API + +## Abstract + +`go.opentelemetry.io/otel/log` provides +[Logs Bridge API](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/). + +The initial version of the design and the prototype +was created in [#4725](https://github.com/open-telemetry/opentelemetry-go/pull/4725). + +## Background + +The key challenge is to create a well-performant API compliant with the specification. +Performance is seen as one of the most important characteristics of logging libraries in Go. + +## Design + +This proposed design aims to: + +- be specification compliant, +- be similar to Trace and Metrics API, +- take advantage of both OpenTelemetry and `slog` experience to achieve acceptable performance. + +### Module structure + +The Go module consits of the following packages: + +- `go.opentelemetry.io/otel/log` +- `go.opentelemetry.io/otel/log/embedded` +- `go.opentelemetry.io/otel/log/noop` + +The module name is compliant with +[Artifact Naming](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/bridge-api.md#artifact-naming) +and the package structure is the same as for Trace API and Metrics API. + +### LoggerProvider + +The [`LoggerProvider` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#loggerprovider) +is defined as an interface [provider.go](provider.go). + +### Logger + +The [`Logger` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#logger) +is defined as an interface in [logger.go](logger.go). + +Canceling the context pass to `Emit` should not affect record processing. +Among other things, log messages may be necessary to debug a +cancellation-related problem. +The context is used to pass request-scoped values. +The API implementation should handle the trace context passed +in `ctx` to the `Emit` method. + +### Record + +The [`LogRecord` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#logger) +is defined as a struct in [record.go](record.go). + +## Usage examples + +### Log Bridge implementation + +The log bridges can use [`sync.Pool`](https://pkg.go.dev/sync#Pool) +for reducing the number of allocations when passing attributes. + +The bridge implementation should do its best to pass +the `ctx` containing the trace context from the caller +so it can later passed via `Emit`. +Re-constructing a `context.Context` with [`trace.ContextWithSpanContext`](https://pkg.go.dev/go.opentelemetry.io/otel/trace#ContextWithSpanContext) +and [`trace.NewSpanContext`](https://pkg.go.dev/go.opentelemetry.io/otel/trace#NewSpanContext) +would usually involve more memory allocations. + +The logging libraries which have recording methods that accepts `context.Context`, +such us [`slog`](https://pkg.go.dev/log/slog), +[`logrus`](https://pkg.go.dev/github.com/sirupsen/logrus) +[`zerolog`](https://pkg.go.dev/github.com/rs/zerolog), +makes passing the trace context trivial. + +However, some libraries do not accept a `context.Context` in their recording methods. +Structured logging libraries, +such as [`logr`](https://pkg.go.dev/github.com/go-logr/logr) +and [`zap`](https://pkg.go.dev/go.uber.org/zap), +offer passing `any` type as a log attribute/field. +Therefore, their bridge implementations can define a "special" log attributes/field +that will be used to capture the trace context. + +[The prototype](https://github.com/open-telemetry/opentelemetry-go/pull/4725) +has a naive implementation of +[slog.Handler](https://pkg.go.dev/log/slog#Handler) in `log/internal/slog.go` +and [logr.LogSink](https://pkg.go.dev/github.com/go-logr/logr#LogSink) in `log/internal/logr.go`. + +### Direct API usage + +The users may also chose to use the API directly. + +```go +package app + +var logger = otel.Logger("my-service") + +// In some function: +logger.Emit(ctx, Record{Severity: log.SeverityInfo, Body: "Application started."}) +``` + +### API implementation + +If the implementation processes the record asynchronously, +then it has to copy record attributes, +in order to avoid use after free bugs and race condition. + +Excerpt of how SDK can implement the `Logger` interface. + +```go +type Logger struct { + scope instrumentation.Scope + processor Processor +} + +func (l *Logger) Emit(ctx context.Context, r log.Record) { + // Create log record model. + record, err := toModel(r) + if err != nil { + otel.Handle(err) + return + } + l.processor.Process(ctx, record) // Note: A batch processor copies the attributes. +} +``` + +A test implementation of the the `Logger` interface +used for benchmarking is in [internal/writer_logger.go](internal/writer_logger.go). + +## Compatibility + +The backwards compatibility is achieved using the `embedded` design pattern +that is already used in Trace API and Metrics API. + +Additionally, the `Logger.Emit` functionality can be extended by +adding new exported fields to the `Record` struct. + +## Benchmarking + +The benchmarks take inspiration from [`slog`](https://pkg.go.dev/log/slog), +because for the Go team it was also critical to create API that would be fast +and interoperable with existing logging packages.[^1][^2] + +## Rejected Alternatives + +### Reuse slog + +The API must not be coupled to [`slog`](https://pkg.go.dev/log/slog), +nor any other logging library. + +The API needs to evolve orthogonally to `slog`. + +`slog` is not compliant with the [Logs Bridge API](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/). +and we cannot expect the Go team to make `slog` compliant with it. + +The interoperabilty can be achieved using [a log bridge](https://opentelemetry.io/docs/specs/otel/glossary/#log-appender--bridge). + +You can read more about OpenTelemetry Logs design on [opentelemetry.io](https://opentelemetry.io/docs/concepts/signals/logs/). + +### Record as interface + +`Record` is defined as a `struct` because of the following reasons. + +Log record is a value object without any behavior. +It is used as data input for Logger methods. + +The log record resembles the instrument config structs like [metric.Float64CounterConfig](https://pkg.go.dev/go.opentelemetry.io/otel/metric#Float64CounterConfig). + +Using `struct` instead of `interface` should have better the performance as e.g. +indirect calls are less optimized, +usage of interfaces tend to increase heap allocations.[^2] + +The `Record` design is inspired by [`slog.Record`](https://pkg.go.dev/log/slog#Record). + +### Options as parameter to Logger.Emit + +One of the initial ideas was to have: + +```go +type Logger interface{ + embedded.Logger + Emit(ctx context.Context, options ...RecordOption) +} +``` + +The main reason was that design would be similar +to the [Meter API](https://pkg.go.dev/go.opentelemetry.io/otel/metric#Meter) +for creating instruments. + +However, passing `Record` directly, instead of using options, +is more performant as it reduces heap allocations.[^3] + +Another advantage of passing `Record` is that API would not have functions like `NewRecord(options...)`, +which would be used by the SDK and not by the users. + +At last, the definition would be similar to [`slog.Handler.Handle`](https://pkg.go.dev/log/slog#Handler) +that was designed to provide optimization opportunities.[^1] + +### Passing record as pointer to Logger.Emit + +So far the benchmarks do not show differences that would +favor passing the record via pointer (and vice versa). + +Passing via value feels safer because of the following reasons. + +It follows the design of [`slog.Handler`](https://pkg.go.dev/log/slog#Handler). + +It should reduce the possibility of a heap allocation. + +The user would not be able to pass `nil`. +Therefore, it reduces the possiblity to have a nil pointer dereference. + +### Passing struct as parameter to LoggerProvider.Logger + +Similarly to `Logger.Emit`, we could have something like: + +```go +type Logger interface{ + embedded.Logger + Logger(name context.Context, config LoggerConfig) +} +``` + +The drawback of this idea would be that this would be +a different design from Trace and Metrics API. + +The performance of acquiring a logger is not as critical +as the performance of emitting a log record. While a single +HTTP/RPC handler could write hundreds of logs, it should not +create a new logger for each log entry. +The application should reuse loggers whenever possible. + +### Logger.WithAttributes + +We could add `WithAttributes` to the `Logger` interface. +Then `Record` could be a simple struct with only exported fields. +The idea was that the SDK would implement the performance improvements +instead of doing it in the API. +This would allow having different optimisation strategies. + +During the analysis[^4], it occurred that the main problem of this proposal +is that the variadic slice passed to an interface method is always heap allocated. + +Moreover, the logger returned by `WithAttribute` was allocated on the heap. + +At last, the proposal was not specification compliant. + +### Record attributes like in slog.Record + +To reduce the number of allocations of the attributes, +the `Record` could be modeled similarly to [`slog.Record`](https://pkg.go.dev/log/slog#Record). +`Record` could have `WalkAttributes` and `AddAttributes` methods, +like [`slog.Record.Attrs`](https://pkg.go.dev/log/slog#Record.Attrs) +and [`slog.Record.AddAttrs`](https://pkg.go.dev/log/slog#Record.AddAttrs), +in order to achieve high-performance when accessing and setting attributes efficiently. +`Record` would have a `AttributesLen` method that returns +the number of attributes to allow slice preallocation +when converting records to a different representation. + +However, during the analysis[^5] we decided that having +a simple slice in `Record` is more flexible. + +It is possible to achieve better performance, by using [`sync.Pool`](https://pkg.go.dev/sync#Pool). + +Having a simple `Record` without any logic makes it possible +that the optimisations can be done in API implementation +and bridge implementations. +For instance, in order to reduce the heap allocations of attributes, +the bridge implementation can use a `sync.Pool`. +In such case, the API implementation (SDK) would need to copy the attributes +when the records are processed asynchrounsly, +in order to avoid use after free bugs and race conditions. + +For reference, here is the reason why `slog` does not use `sync.Pool`[^2]: + +> We can use a sync pool for records though we decided not to. +You can but it's a bad idea for us. Why? +Because users have control of Records. +Handler writers can get their hands on a record +and we'd have to ask them to free it +or try to free it magically at some some point. +But either way, they could get themselves in trouble by freeing it twice +or holding on to one after they free it. +That's a use after free bug and that's why `zerolog` was problematic for us. +`zerolog` as as part of its speed exposes a pool allocated value to users +if you use `zerolog` the normal way, that you'll see in all the examples, +you will never encounter a problem. +But if you do something a little out of the ordinary you can get +use after free bugs and we just didn't want to put that in the standard library. + +We took a different decision, because the key difference is that `slog` +is a logging library and Logs Bridge API is only a logging abstraction. +We want to provide more flexibility and offer better speed. + +[^1]: Jonathan Amsterdam, [The Go Blog: Structured Logging with slog](https://go.dev/blog/slog) +[^2]: Jonathan Amsterdam, [GopherCon Europe 2023: A Fast Structured Logging Package](https://www.youtube.com/watch?v=tC4Jt3i62ns) +[^3]: [Emit definition discussion with benchmarks](https://github.com/open-telemetry/opentelemetry-go/pull/4725#discussion_r1400869566) +[^4]: [Logger.WithAttributes analysis](https://github.com/pellared/opentelemetry-go/pull/3) +[^5]: [Record attributes as field and use sync.Pool for reducing allocations analysis](https://github.com/pellared/opentelemetry-go/pull/4) diff --git a/log/doc.go b/log/doc.go new file mode 100644 index 00000000000..f9a9ec96a36 --- /dev/null +++ b/log/doc.go @@ -0,0 +1,85 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +/* +Package log defines the OpenTelemetry Bridge API. +It is supposed to be used by a log bridge implementation +that is an adapter between an existing logging library and OpenTelemetry. +Application code should not call this API directly. + +Existing logging libraries generally provide a much richer set of features +than in OpenTelemetry. It is not a goal of OpenTelemetry, to ship +a feature-rich logging library. + +# Bridge Implementations + +The bridge implementation should allow passing a [context.Context] containing +a trace context from the caller to [Logger]'s Emit method. + +A bridge can use [sync.Pool] of attributes for reducing the number of +heap allocations. + +# API Implementations + +This package does not conform to the standard Go versioning policy, all of its +interfaces may have methods added to them without a package major version bump. +This non-standard API evolution could surprise an uninformed implementation +author. They could unknowingly build their implementation in a way that would +result in a runtime panic for their users that update to the new API. + +The API is designed to help inform an instrumentation author about this +non-standard API evolution. It requires them to choose a default behavior for +unimplemented interface methods. There are three behavior choices they can +make: + + - Compilation failure + - Panic + - Default to another implementation + +All interfaces in this API embed a corresponding interface from +[go.opentelemetry.io/otel/log/embedded]. If an author wants the default +behavior of their implementations to be a compilation failure, signaling to +their users they need to update to the latest version of that implementation, +they need to embed the corresponding interface from +[go.opentelemetry.io/otel/log/embedded] in their implementation. For +example, + + import "go.opentelemetry.io/otel/log/embedded" + + type LoggerProvider struct { + embedded.LoggerProvider + // ... + } + +If an author wants the default behavior of their implementations to a panic, +they need to embed the API interface directly. + + import "go.opentelemetry.io/otel/log" + + type LoggerProvider struct { + log.LoggerProvider + // ... + } + +This is not a recommended behavior as it could lead to publishing packages that +contain runtime panics when users update other package that use newer versions +of [go.opentelemetry.io/otel/log]. + +Finally, an author can embed another implementation in theirs. The embedded +implementation will be used for methods not defined by the author. For example, +an author who wants to default to silently dropping the call can use +[go.opentelemetry.io/otel/log/noop]: + + import "go.opentelemetry.io/otel/log/noop" + + type LoggerProvider struct { + noop.LoggerProvider + // ... + } + +It is strongly recommended that authors only embed +[go.opentelemetry.io/otel/log/noop] if they choose this default behavior. +That implementation is the only one OpenTelemetry authors can guarantee will +fully implement all the API interfaces when a user updates their API. +*/ +package log // import "go.opentelemetry.io/otel/log" diff --git a/log/embedded/embedded.go b/log/embedded/embedded.go new file mode 100644 index 00000000000..30bda5c4f7b --- /dev/null +++ b/log/embedded/embedded.go @@ -0,0 +1,35 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +// Package embedded provides interfaces embedded within +// the OpenTelemetry Logs Bridge API. +// +// Implementers of the [OpenTelemetry Logs Bridge API] can embed the relevant type +// from this package into their implementation directly. Doing so will result +// in a compilation error for users when the [OpenTelemetry Logs Bridge API] is +// extended (which is something that can happen without a major version bump of +// the API package). +// +// [OpenTelemetry Logs Bridge API]: https://pkg.go.dev/go.opentelemetry.io/otel/log +package embedded // import "go.opentelemetry.io/otel/log/embedded" + +// LoggerProvider is embedded in +// [go.opentelemetry.io/otel/log.LoggerProvider]. +// +// Embed this interface in your implementation of the +// [go.opentelemetry.io/otel/log.LoggerProvider] if you want users to +// experience a compilation error, signaling they need to update to your latest +// implementation, when the [go.opentelemetry.io/otel/log.LoggerProvider] +// interface is extended (which is something that can happen without a major +// version bump of the API package). +type LoggerProvider interface{ loggerProvider() } + +// Logger is embedded in [go.opentelemetry.io/otel/log.Logger]. +// +// Embed this interface in your implementation of the +// [go.opentelemetry.io/otel/log.Logger] if you want users to experience a +// compilation error, signaling they need to update to your latest +// implementation, when the [go.opentelemetry.io/otel/log.Logger] interface +// is extended (which is something that can happen without a major version bump +// of the API package). +type Logger interface{ logger() } diff --git a/log/go.mod b/log/go.mod new file mode 100644 index 00000000000..121ff5f10f1 --- /dev/null +++ b/log/go.mod @@ -0,0 +1,20 @@ +module go.opentelemetry.io/otel/log + +go 1.20 + +require ( + github.com/stretchr/testify v1.8.4 + go.opentelemetry.io/otel v1.21.0 +) + +require ( + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect +) + +replace go.opentelemetry.io/otel => ../ + +replace go.opentelemetry.io/otel/trace => ../trace + +replace go.opentelemetry.io/otel/metric => ../metric diff --git a/log/go.sum b/log/go.sum new file mode 100644 index 00000000000..a6bcd03a15e --- /dev/null +++ b/log/go.sum @@ -0,0 +1,11 @@ +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/log/internal/bench_test.go b/log/internal/bench_test.go new file mode 100644 index 00000000000..60449a5aeef --- /dev/null +++ b/log/internal/bench_test.go @@ -0,0 +1,227 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +// Copyright 2022 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// These benchmarks are based on slog/internal/benchmarks. +// +// They test a complete log record, from the user's call to its return. + +package internal + +import ( + "context" + "io" + "sync" + "testing" + "time" + + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/log" + "go.opentelemetry.io/otel/log/noop" + "go.opentelemetry.io/otel/trace" +) + +var ( + ctx = trace.ContextWithSpanContext(context.Background(), trace.NewSpanContext(trace.SpanContextConfig{TraceID: [16]byte{1}, SpanID: [8]byte{42}})) + testTimestamp = time.Date(1988, time.November, 17, 0, 0, 0, 0, time.UTC) + testBody = "log message" + testSeverity = log.SeverityInfo + testFloat = 1.2345 + testString = "7e3b3b2aaeff56a7108fe11e154200dd/7819479873059528190" + testInt = 32768 + testBool = true +) + +// WriterLogger is an optimistic version of a real logger, doing real-world +// tasks as fast as possible. This gives us an upper bound on handler +// performance, so we can evaluate the (logger-independent) core activity of +// the package in an end-to-end context without concern that a slow logger +// implementation is skewing the results. The writerLogger allocates memory +// only when using strconv. +// The tests are using a sync.Pool for attributes as the bridge implementation +// can do the same do the same in order to reduce the number of heap allocations. +func BenchmarkEmit(b *testing.B) { + attrPool := sync.Pool{ + New: func() interface{} { + attr := make([]attribute.KeyValue, 0, 5) + return &attr + }, + } + + for _, tc := range []struct { + name string + logger log.Logger + }{ + {"noop", noop.Logger{}}, // There should be no allocations. + {"writer", &writerLogger{w: io.Discard}}, // If there are allocations, they should only be from strconv used in writerLogger. + } { + b.Run(tc.name, func(b *testing.B) { + for _, call := range []struct { + name string + f func() + }{ + { + "no attrs", + func() { + r := log.Record{ + Timestamp: testTimestamp, + Severity: testSeverity, + Body: testBody, + } + tc.logger.Emit(ctx, r) + }, + }, + { + "3 attrs", + func() { + ptr := attrPool.Get().(*[]attribute.KeyValue) + attrs := *ptr + defer func() { + *ptr = attrs[:0] + attrPool.Put(ptr) + }() + attrs = append(attrs, + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + ) + r := log.Record{ + Timestamp: testTimestamp, + Severity: testSeverity, + Body: testBody, + Attributes: attrs, + } + tc.logger.Emit(ctx, r) + }, + }, + { + "5 attrs", + func() { + ptr := attrPool.Get().(*[]attribute.KeyValue) + attrs := *ptr + defer func() { + *ptr = attrs[:0] + attrPool.Put(ptr) + }() + attrs = append(attrs, + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + ) + r := log.Record{ + Timestamp: testTimestamp, + Severity: testSeverity, + Body: testBody, + Attributes: attrs, + } + tc.logger.Emit(ctx, r) + }, + }, + { + "10 attrs", + func() { + ptr := attrPool.Get().(*[]attribute.KeyValue) + attrs := *ptr + defer func() { + *ptr = attrs[:0] + attrPool.Put(ptr) + }() + attrs = append(attrs, + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + ) + r := log.Record{ + Timestamp: testTimestamp, + Severity: testSeverity, + Body: testBody, + Attributes: attrs, + } + tc.logger.Emit(ctx, r) + }, + }, + { + "40 attrs", + func() { + ptr := attrPool.Get().(*[]attribute.KeyValue) + attrs := *ptr + defer func() { + *ptr = attrs[:0] + attrPool.Put(ptr) + }() + attrs = append(attrs, + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + attribute.String("string", testString), + ) + r := log.Record{ + Timestamp: testTimestamp, + Severity: testSeverity, + Body: testBody, + Attributes: attrs, + } + tc.logger.Emit(ctx, r) + }, + }, + } { + b.Run(call.name, func(b *testing.B) { + b.ReportAllocs() + b.RunParallel(func(pb *testing.PB) { + for pb.Next() { + call.f() + } + }) + }) + } + }) + } +} diff --git a/log/internal/go.mod b/log/internal/go.mod new file mode 100644 index 00000000000..4bec966037d --- /dev/null +++ b/log/internal/go.mod @@ -0,0 +1,24 @@ +module go.opentelemetry.io/otel/log/internal + +go 1.20 + +require ( + github.com/stretchr/testify v1.8.4 + go.opentelemetry.io/otel v1.21.0 + go.opentelemetry.io/otel/log v0.0.0-00010101000000-000000000000 + go.opentelemetry.io/otel/trace v1.21.0 +) + +require ( + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/pmezard/go-difflib v1.0.0 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect +) + +replace go.opentelemetry.io/otel/log => ../ + +replace go.opentelemetry.io/otel => ../.. + +replace go.opentelemetry.io/otel/trace => ../../trace + +replace go.opentelemetry.io/otel/metric => ../../metric diff --git a/log/internal/go.sum b/log/internal/go.sum new file mode 100644 index 00000000000..a6bcd03a15e --- /dev/null +++ b/log/internal/go.sum @@ -0,0 +1,11 @@ +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/log/internal/writer_logger.go b/log/internal/writer_logger.go new file mode 100644 index 00000000000..7e21f3035b4 --- /dev/null +++ b/log/internal/writer_logger.go @@ -0,0 +1,69 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package internal // import "go.opentelemetry.io/otel/log/internal" + +import ( + "context" + "fmt" + "io" + "strconv" + + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/log" + "go.opentelemetry.io/otel/log/embedded" + "go.opentelemetry.io/otel/trace" +) + +// writerLogger is a logger that writes to a provided io.Writer without any locking. +// It is intended to represent a high-performance logger that synchronously +// writes text. +type writerLogger struct { + embedded.Logger + w io.Writer +} + +func (l *writerLogger) Emit(ctx context.Context, r log.Record) { + if !r.Timestamp.IsZero() { + l.write("timestamp=") + l.write(strconv.FormatInt(r.Timestamp.Unix(), 10)) + l.write(" ") + } + l.write("severity=") + l.write(strconv.FormatInt(int64(r.Severity), 10)) + l.write(" ") + l.write("body=") + l.write(r.Body) + for _, kv := range r.Attributes { + l.write(" ") + l.write(string(kv.Key)) + l.write("=") + l.appendValue(kv.Value) + } + + span := trace.SpanContextFromContext(ctx) + if span.IsValid() { + l.write(" traced=true") + } + + l.write("\n") +} + +func (l *writerLogger) appendValue(v attribute.Value) { + switch v.Type() { + case attribute.STRING: + l.write(v.AsString()) + case attribute.INT64: + l.write(strconv.FormatInt(v.AsInt64(), 10)) // strconv.FormatInt allocates memory. + case attribute.FLOAT64: + l.write(strconv.FormatFloat(v.AsFloat64(), 'g', -1, 64)) // strconv.FormatFloat allocates memory. + case attribute.BOOL: + l.write(strconv.FormatBool(v.AsBool())) + default: + panic(fmt.Sprintf("unhandled attribute type: %s", v.Type())) + } +} + +func (l *writerLogger) write(s string) { + _, _ = io.WriteString(l.w, s) +} diff --git a/log/internal/writer_logger_test.go b/log/internal/writer_logger_test.go new file mode 100644 index 00000000000..ab0dae929f7 --- /dev/null +++ b/log/internal/writer_logger_test.go @@ -0,0 +1,35 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package internal + +import ( + "strings" + "testing" + + "github.com/stretchr/testify/assert" + + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/log" +) + +func TestWriterLogger(t *testing.T) { + sb := &strings.Builder{} + l := &writerLogger{w: sb} + + r := log.Record{ + Timestamp: testTimestamp, + Severity: testSeverity, + Body: testBody, + Attributes: []attribute.KeyValue{ + attribute.String("string", testString), + attribute.Float64("float", testFloat), + attribute.Int("int", testInt), + attribute.Bool("bool", testBool), + }, + } + l.Emit(ctx, r) + + want := "timestamp=595728000 severity=9 body=log message string=7e3b3b2aaeff56a7108fe11e154200dd/7819479873059528190 float=1.2345 int=32768 bool=true traced=true\n" + assert.Equal(t, want, sb.String()) +} diff --git a/log/logger.go b/log/logger.go new file mode 100644 index 00000000000..9845b2754ff --- /dev/null +++ b/log/logger.go @@ -0,0 +1,32 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package log // import "go.opentelemetry.io/otel/log" + +import ( + "context" + + "go.opentelemetry.io/otel/log/embedded" +) + +// Logger emits log records. +// +// Warning: Methods may be added to this interface in minor releases. See +// package documentation on API implementation for information on how to set +// default behavior for unimplemented methods. +type Logger interface { + // Users of the interface can ignore this. This embedded type is only used + // by implementations of this interface. See the "API Implementations" + // section of the package documentation for more information. + embedded.Logger + + // Emit emits a log record. + // + // This method should: + // - be safe to call concurrently, + // - handle the trace context passed via ctx argument, + // - not modify the record's attributes, + // - copy the record's attributes in case of asynchronous processing, + // - use the current time as observed timestamp if the passed is empty. + Emit(ctx context.Context, record Record) +} diff --git a/log/noop/noop.go b/log/noop/noop.go new file mode 100644 index 00000000000..266c9bc22fe --- /dev/null +++ b/log/noop/noop.go @@ -0,0 +1,47 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +// Package noop provides an implementation of the OpenTelemetry Logs Bridge API that +// produces no telemetry and minimizes used computation resources. +// +// Using this package to implement the [OpenTelemetry Logs Bridge API] will effectively +// disable OpenTelemetry. +// +// This implementation can be embedded in other implementations of the +// [OpenTelemetry Logs Bridge API]. Doing so will mean the implementation defaults to +// no operation for methods it does not implement. +// +// [OpenTelemetry Logs Bridge API]: https://pkg.go.dev/go.opentelemetry.io/otel/log +package noop // import "go.opentelemetry.io/otel/log/noop" + +import ( + "context" + + "go.opentelemetry.io/otel/log" + "go.opentelemetry.io/otel/log/embedded" +) + +var ( + // Compile-time check this implements the OpenTelemetry API. + _ log.LoggerProvider = LoggerProvider{} + _ log.Logger = Logger{} +) + +// LoggerProvider is an OpenTelemetry No-Op LoggerProvider. +type LoggerProvider struct{ embedded.LoggerProvider } + +// NewLoggerProvider returns a LoggerProvider that does not record any telemetry. +func NewLoggerProvider() LoggerProvider { + return LoggerProvider{} +} + +// Logger returns an OpenTelemetry Logger that does not record any telemetry. +func (LoggerProvider) Logger(string, ...log.LoggerOption) log.Logger { + return Logger{} +} + +// Logger is an OpenTelemetry No-Op Logger. +type Logger struct{ embedded.Logger } + +// Emit does nothing. +func (Logger) Emit(context.Context, log.Record) {} diff --git a/log/provider.go b/log/provider.go new file mode 100644 index 00000000000..ebb43e46ec2 --- /dev/null +++ b/log/provider.go @@ -0,0 +1,103 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package log // import "go.opentelemetry.io/otel/log" + +import ( + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/log/embedded" +) + +// LoggerProvider provides access to named [Logger] instances. +// +// Warning: Methods may be added to this interface in minor releases. See +// package documentation on API implementation for information on how to set +// default behavior for unimplemented methods. +type LoggerProvider interface { + // Users of the interface can ignore this. This embedded type is only used + // by implementations of this interface. See the "API Implementations" + // section of the package documentation for more information. + embedded.LoggerProvider + + // Logger returns a new [Logger] with the provided name and configuration. + // + // This method should: + // - be safe to call concurrently, + // - use some default name if the passed name is empty. + Logger(name string, options ...LoggerOption) Logger +} + +// LoggerConfig contains options for Logger. +type LoggerConfig struct { + instrumentationVersion string + schemaURL string + attrs attribute.Set + + // Ensure forward compatibility by explicitly making this not comparable. + noCmp [0]func() //nolint: unused // This is indeed used. +} + +// InstrumentationVersion returns the version of the library providing +// instrumentation. +func (cfg LoggerConfig) InstrumentationVersion() string { + return cfg.instrumentationVersion +} + +// InstrumentationAttributes returns the attributes associated with the library +// providing instrumentation. +func (cfg LoggerConfig) InstrumentationAttributes() attribute.Set { + return cfg.attrs +} + +// SchemaURL is the schema_url of the library providing instrumentation. +func (cfg LoggerConfig) SchemaURL() string { + return cfg.schemaURL +} + +// LoggerOption is an interface for applying Logger options. +type LoggerOption interface { + // applyLogger is used to set a LoggerOption value of a LoggerConfig. + applyLogger(LoggerConfig) LoggerConfig +} + +// NewLoggerConfig creates a new LoggerConfig and applies +// all the given options. +func NewLoggerConfig(opts ...LoggerOption) LoggerConfig { + var config LoggerConfig + for _, o := range opts { + config = o.applyLogger(config) + } + return config +} + +type loggerOptionFunc func(LoggerConfig) LoggerConfig + +func (fn loggerOptionFunc) applyLogger(cfg LoggerConfig) LoggerConfig { + return fn(cfg) +} + +// WithInstrumentationVersion sets the instrumentation version. +func WithInstrumentationVersion(version string) LoggerOption { + return loggerOptionFunc(func(config LoggerConfig) LoggerConfig { + config.instrumentationVersion = version + return config + }) +} + +// WithInstrumentationAttributes sets the instrumentation attributes. +// +// The passed attributes will be de-duplicated. +func WithInstrumentationAttributes(attr ...attribute.KeyValue) LoggerOption { + return loggerOptionFunc(func(config LoggerConfig) LoggerConfig { + config.attrs = attribute.NewSet(attr...) + return config + }) +} + +// WithSchemaURL sets the schema URL. +func WithSchemaURL(schemaURL string) LoggerOption { + return loggerOptionFunc(func(config LoggerConfig) LoggerConfig { + config.schemaURL = schemaURL + return config + }) +} diff --git a/log/provider_test.go b/log/provider_test.go new file mode 100644 index 00000000000..23f2ab64be0 --- /dev/null +++ b/log/provider_test.go @@ -0,0 +1,32 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +package log_test + +import ( + "testing" + + "github.com/stretchr/testify/assert" + + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/log" +) + +func TestNewLoggerConfig(t *testing.T) { + version := "v1.1.1" + schemaURL := "https://opentelemetry.io/schemas/1.0.0" + attr := attribute.NewSet( + attribute.String("user", "alice"), + attribute.Bool("admin", true), + ) + + c := log.NewLoggerConfig( + log.WithInstrumentationVersion(version), + log.WithSchemaURL(schemaURL), + log.WithInstrumentationAttributes(attr.ToSlice()...), + ) + + assert.Equal(t, version, c.InstrumentationVersion(), "instrumentation version") + assert.Equal(t, schemaURL, c.SchemaURL(), "schema URL") + assert.Equal(t, attr, c.InstrumentationAttributes(), "instrumentation attributes") +} diff --git a/log/record.go b/log/record.go new file mode 100644 index 00000000000..f4dccb37417 --- /dev/null +++ b/log/record.go @@ -0,0 +1,68 @@ +// Copyright The OpenTelemetry Authors +// SPDX-License-Identifier: Apache-2.0 + +// Copyright 2022 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +package log // import "go.opentelemetry.io/otel/log" + +import ( + "time" + + "go.opentelemetry.io/otel/attribute" +) + +// Record represents a log record. +type Record struct { + Timestamp time.Time + ObservedTimestamp time.Time + Severity Severity + SeverityText string + Body string + Attributes []attribute.KeyValue +} + +// Severity represents a log record severity. +// Smaller numerical values correspond to less severe log records (such as debug events), +// larger numerical values correspond to more severe log records (such as errors and critical events). +type Severity int + +// Severity values defined by OpenTelemetry. +const ( + // A fine-grained debugging log record. Typically disabled in default configurations. + SeverityTrace Severity = iota + 1 + SeverityTrace2 + SeverityTrace3 + SeverityTrace4 + + // A debugging log record. + SeverityDebug + SeverityDebug2 + SeverityDebug3 + SeverityDebug4 + + // An informational log record. Indicates that an event happened. + SeverityInfo + SeverityInfo2 + SeverityInfo3 + SeverityInfo4 + + // A warning log record. Not an error but is likely more important than an informational event. + SeverityWarn + SeverityWarn2 + SeverityWarn3 + SeverityWarn4 + + // An error log record. Something went wrong. + SeverityError + SeverityError2 + SeverityError3 + SeverityError4 + + // A fatal log record such as application or system crash. + SeverityFatal + SeverityFatal2 + SeverityFatal3 + SeverityFatal4 +) diff --git a/versions.yaml b/versions.yaml index 3c153c9d6fc..ce59812628b 100644 --- a/versions.yaml +++ b/versions.yaml @@ -48,5 +48,10 @@ module-sets: version: v0.0.7 modules: - go.opentelemetry.io/otel/schema + experimental-logs: + version: v0.0.1 + modules: + - go.opentelemetry.io/otel/log + - go.opentelemetry.io/otel/log/internal excluded-modules: - go.opentelemetry.io/otel/internal/tools