Skip to content

Commit

Permalink
docs: add how-it-works.md (#8)
Browse files Browse the repository at this point in the history
  • Loading branch information
y1yang0 committed Jul 30, 2024
1 parent 1af9e32 commit b993e26
Show file tree
Hide file tree
Showing 4 changed files with 128 additions and 12 deletions.
136 changes: 125 additions & 11 deletions docs/how-it-works.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,133 @@
# How it works

The automatic instrumentation solution is implemented around the concept of `Rule`.
The workflow could roughly be divided into two main phases:

Three types of rules defined in [ruledef.go](../api/ruledef.go) are supported now:
![](workflow.png)

- `InstFuncRule`: describes how to instrument a specific go function.
- `InstStructRule`: describes how to instrument a specific go struct.
- `InstFileRule`: describes how to instrument a specific go file.
- `Preprocess`: Analyze dependencies and select rules that should be used later.
- `Instrument`: Generate code based on rules and inject new code into source code.

There are no mandatory restrictions on the scope of target code, which can be the project's own code, code from
dependent libraries, or even the go runtime.
## Preprocess
Analyze the third-party library dependencies of the user's project code and match
them with existing instrumentation rules to find suitable rules. Additionally,
configure the extra dependencies required by these rules in advance. When all
preprocessing tasks are ready, the `go build -toolexec otel-go-auto-instrumentation cmd/app`
command is invoked for compilation.

The workflow could roughly be divided into two main phases:
Instrumentation rules precisely define which code needs to be injected into which
version of which framework or standard library. Different types of instrumentation
rules serve different purposes. The currently available types of instrumentation
rules include:

- `Preprocess`: Analyze dependencies and select rules that should be used later.
- `Instrument`: Generate code based on rules and inject new code into source code.
- InstFuncRule: Inject code at the entry and exit points of a method.
- InstStructRule: Modify a struct by adding a new field.
- InstFileRule: Add a new file to participate in the original compilation process.

The `-toolexec` parameter is the core of automatic instrumentation. It is used to
intercept the regular build process and replace it with user-defined tools,
allowing developers more flexibility in customizing the build process.
The **otel-go-auto-instrumentation** invoked here is the automatic instrumentation
tool, which leads to the second stage: code injection, i.e. Instrument.

## Instrument
Based on the rules, trampoline code is inserted into the target functions,
compilation parameters are modified, and then the `go build cmd/app` command is
invoked for compilation. Trampoline code (Trampoline Jump) is essentially a complex
If-statement. Through it, instrumentation code can be inserted at the entry and
exit points of the target function, enabling the collection of monitoring data.

# `net/http` example
First, we classify the following three types of functions: *RawFunc*, *TrampolineFunc*, *HookFunc*. RawFunc is the original function that needs to be injected. TrampolineFunc is the trampoline function. HookFunc is onEnter/onExit functions that need to be inserted at the entry and exit points of the original function as probe code. RawFunc jumps to TrampolineFunc via the inserted trampoline code, then TrampolineFunc constructs the context, prepares the error recovery handling, and finally jumps to HookFunc to execute the probe code.

![](tjump.png)

Next, we use `net/http` as an example to demonstrate how compile-time automatic instrumentation can insert monitoring code into the target function `(*Transport).RoundTrip()`. The framework will generate trampoline code at the entry of this function, which is an if statement (actually one line, written in multiple lines for demonstration) that jumps to TrampolineFunc:

```go
func (t *Transport) RoundTrip(req *Request) (retVal0 *Response, retVal1 error) {
if callContext37639, _ := OtelOnEnterTrampoline_RoundTrip37639(&t, &req); false {
} else {
defer OtelOnExitTrampoline_RoundTrip37639(callContext37639, &retVal0, &retVal1)
}
return t.roundTrip(req)
}
```

Here, `OtelOnEnterTrampoline_RoundTrip37639` is the TrampolineFunc. It prepares error handling and the call context, then jumps to `ClientOnEnterImpl`:

```go
func OtelOnEnterTrampoline_RoundTrip37639(t **Transport, req **Request) (*CallContext, bool) {
defer func() {
if err := recover(); err != nil {
println("failed to exec onEnter hook", "clientOnEnter")
if e, ok := err.(error); ok {
println(e.Error())
}
fetchStack, printStack := OtelGetStackImpl, OtelPrintStackImpl
if fetchStack != nil && printStack != nil {
printStack(fetchStack())
}
}
}()
callContext := &CallContext{
Params: nil,
ReturnVals: nil,
SkipCall: false,
}
callContext.Params = []interface{}{t, req}
ClientOnEnterImpl(callContext, *t, *req)
return callContext, callContext.SkipCall
}
```

The `ClientOnEnterImpl` is the HookFunc, which is our probe code where traces, metrics reporting, etc., are performed. `ClientOnEnterImpl` is a function pointer, pre-configured in the automatically generated *otel_setup_inst.go* during the preprocessing stage, and it actually points to `clientOnEnter`:

```go
// == otel_setup_inst.go
package otel_rules

import http328 "net/http"
...

func init() {
http328.ClientOnEnterImpl = clientOnEnter
...
}
```

The `clientOnEnter` function performs the actual monitoring tasks:

```go
// == otel_rule_http59729.go
func clientOnEnter(call *http.CallContext, t *http.Transport, req *http.Request) {
...
var tracer trace.Tracer
if span := trace.SpanFromContext(req.Context()); span.SpanContext().IsValid() {
tracer = span.TracerProvider().Tracer("")
} else {
tracer = otel.GetTracerProvider().Tracer("")
}
opts := append([]trace.SpanStartOption{}, trace.WithSpanKind(trace.SpanKindClient))
ctx, span := tracer.Start(req.Context(), req.URL.Path, opts...)
var attrs []attribute.KeyValue
attrs = append(attrs, semconv.HTTPMethodKey.String(req.Method))
attrs = append(attrs, attributes.MakeSpanAttrs(req.URL.Path, req.URL.Host, attributes.Http)...)
span.SetAttributes(attrs...)
bag := baggage.FromContext(ctx)
if mem, err := baggage.NewMemberRaw(constants.BAGGAGE_PARENT_PID, attributes.Pid); err == nil {
bag, _ = bag.SetMember(mem)
}
if mem, err := baggage.NewMemberRaw(constants.BAGGAGE_PARENT_RPC, sdktrace.GetRpc()); err == nil {
bag, _ = bag.SetMember(mem)
}
sdktrace.SetGLocalData(constants.TRACE_ID, span.SpanContext().TraceID().String())
sdktrace.SetGLocalData(constants.SPAN_ID, span.SpanContext().SpanID().String())
ctx = baggage.ContextWithBaggage(ctx, bag)
otel.GetTextMapPropagator().Inject(ctx, propagation.HeaderCarrier(req.Header))
req = req.WithContext(ctx)
*(call.Params[1].(**http.Request)) = req
return
}
```

TODO
Through the above steps, we not only inserted monitoring code into the `(*Transport).RoundTrip()` function but also ensured the accuracy and propagation of monitoring data and context. During compile-time automatic instrumentation, these operations are all done automatically, saving developers a significant amount of time and reducing the error rate of manual probes.
Binary file added docs/tjump.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion tool/preprocess/dependency.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,9 @@ func getCompileCommands() ([]string, error) {
}
err = runDryBuild()
if err != nil {
return nil, fmt.Errorf("failed to run dry build: %w", err)
// Tell us more about what happened in the dry run
errLog, _ := util.ReadFile(shared.GetLogPath(DryRunLog))
return nil, fmt.Errorf("failed to run dry build: %w\n%v", err, errLog)
}
dryRunLog, err := os.Open(shared.GetLogPath(DryRunLog))
if err != nil {
Expand Down

0 comments on commit b993e26

Please sign in to comment.