runtime/trace: expose API to parse traces

[cdvr1993]

Go pprof allows a way to see profiles in a text based format. This is very useful when you have hundreds of profiles to analyze and you want to find the specific instance that has the most code in a specific function.

On the other hand, trace only allows to do UI analysis. Then if I have hundreds of traces I need to go one by one and traces can take several seconds if not minutes to load. To put a "real" example, I recently had to find why we were having long GC pauses, but these GC pauses were only happening in 1-3 instances once in a while (3-5 minutes). I was able to find the pattern to run a complete horizontal tracing of it, but after that I needed to analyze ~64 90 seconds traces.

What I ended up doing was to copy the source code from internal/trace and manually create a function that only returned the data that I wanted. Once I got the instance with the highest GC pause I fallback to the normal way of loading a trace in the UI and I was able to find the source of the problem.

The proposal/ask is if the community has considered exposing some of the APIs under internal/trace? Or provide a text based command line approach for the trace command?

[rhysh]

CC @mknyszek (and @golang/runtime ?)

The design doc in #60773 includes plans for a parsing API, sometime in the future. You might also be interested in following the discussion on #57175.

Today, you might be able to use the parser in https://pkg.go.dev/honnef.co/go/gotraceui@v0.3.0/trace#Parse

Or, if the information you're trying to extract is something relatively easy to calculate like "the time difference between STW begin and end events", you might be able to use go tool trace -d to dump the text format of the execution trace for processing with a tool like awk.

[mknyszek]

FWIW, I'm already working on this as part of #57175. I plan to land a trace format reader in x/exp for the Go 1.22 release, which will get vendored or copied into std as an internal package. Just to set expectations, to begin with, it may only support Go 1.22+ traces depending on how much time I have. But long-term I expect it to support older traces as well (with worse decoding performance, since the old trace format requires decoding the entire trace up-front).

[ianlancetaylor]

Taking this issue out of the proposal process.

[mknyszek]

See #62627, which I think this issue is now a duplicate of. That issue has a link to an experimental public API for this, which is now available at golang.org/x/exp/trace. It only works for Go 1.22 traces, but it'll work for older traces soon, sometime early next year. If you're inclined, please give it a try! We're collecting feedback at this point.