Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/cgo/internal/testsanitizers,x/build: upstream clang does not build against zlib, so checking for ASAN symbolization is disabled during the ASAN tests #65606

Open
mknyszek opened this issue Feb 8, 2024 · 5 comments
Open

cmd/cgo/internal/testsanitizers,x/build: upstream clang does not build against zlib, so checking for ASAN symbolization is disabled during the ASAN tests #65606

mknyszek opened this issue Feb 8, 2024 · 5 comments
Assignees
@mknyszek
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Backlog

Comments

@mknyszek
Copy link
Contributor

mknyszek commented Feb 8, 2024

See #65469 (comment) for complete context.

For our LUCI infrastructure, we store an official clang toolchain release (https://releases.llvm.org/) in CIPD, and download it into our build environment on-demand. Previously, we just baked the toolchain into the image using an officially-maintained package for Debian (https://apt.llvm.org), but this made upgrading onerous enough that we were stayed on clang 7 (2018) for a long time.

Unfortunately, the official clang toolchain releases aren't built against zlib, and the Go toolchain compresses DWARF sections. When the ASAN runtime tries to symbolize the stack trace it takes, we get failures like:

        error: failed to decompress '.debug_aranges', zlib is not available
        error: failed to decompress '.debug_info', zlib is not available
        error: failed to decompress '.debug_abbrev', zlib is not available
        error: failed to decompress '.debug_line', zlib is not available
        error: failed to decompress '.debug_frame', zlib is not available
        error: failed to decompress '.debug_str', zlib is not available
        error: failed to decompress '.debug_loc', zlib is not available
        error: failed to decompress '.debug_ranges', zlib is not available
        error: failed to decompress '.debug_addr', zlib is not available
        error: failed to decompress '.debug_rnglists', zlib is not available
        error: failed to decompress '.debug_str_offsets', zlib is not available
        error: failed to decompress '.debug_line_str', zlib is not available

The alternatives here are not ideal. They require either maintaining more images or complicating our clang update process significantly. For now, we've decided to just not check for symbolized output for our ASAN tests when building with clang. We bake GCC into our images, and GCC can symbolize the output, so we still have coverage on our debug information.

This issue tracks either:

  • Waiting for clang to build against zlib in one of their releases, or
  • Deciding to bake clang into the image again.
@mknyszek mknyszek self-assigned this Feb 8, 2024
@mknyszek mknyszek added this to the Backlog milestone Feb 8, 2024
@mknyszek mknyszek added NeedsFix The path to resolution is known, but the work has not been done. compiler/runtime Issues related to the Go compiler and/or runtime. labels Feb 8, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/562675 mentions this issue: cmd/cgo/internal/testsanitizers: disable location checking for clang

@mknyszek
Copy link
Contributor Author

mknyszek commented Feb 8, 2024 •
edited
Loading

Another benefit of keeping clang in CIPD is that it's easier to test multiple versions of clang, if we ever need to. (Though, given #65302 is still open at this time, it's not clear to me if that's necessary at this point.)

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/562538 mentions this issue: env/linux-x86-bullseye: add clang to image

gopherbot pushed a commit that referenced this issue Feb 9, 2024
@mknyszek
cmd/cgo/internal/testsanitizers: disable location checking for clang
d94ab59 
Pending a resolution to #65606, this CL marks clang's ASAN runtime as
unable to symbolize stack traces to unblock the LUCI clang builder.

For #65606.
Fixes #65469.

Change-Id: I649773085aff30e5703e7f7ac2c72a0430a015c2
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/562675
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/563015 mentions this issue: [release-branch.go1.22] cmd/cgo/internal/testsanitizers: disable location checking for clang

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/562999 mentions this issue: [release-branch.go1.21] cmd/cgo/internal/testsanitizers: disable location checking for clang

@mknyszek mknyszek mentioned this issue Feb 14, 2024
Open
@mknyszek mknyszek moved this to In Progress in Go Compiler / Runtime Feb 14, 2024
gopherbot pushed a commit that referenced this issue Feb 16, 2024
@mknyszek @thanm
[release-branch.go1.21] cmd/cgo/internal/testsanitizers: disable loca…
f38fca3 
…tion checking for clang

Pending a resolution to #65606, this CL marks clang's ASAN runtime as
unable to symbolize stack traces to unblock the LUCI clang builder.

For #65606.
For #65469.
Fixes #65640.

Change-Id: I649773085aff30e5703e7f7ac2c72a0430a015c2
Cq-Include-Trybots: luci.golang.try:go1.21-linux-amd64-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/562675
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit d94ab59)
Reviewed-on: https://go-review.googlesource.com/c/go/+/562999
gopherbot pushed a commit that referenced this issue Feb 16, 2024
@mknyszek @thanm
[release-branch.go1.22] cmd/cgo/internal/testsanitizers: disable loca…
d6a2719 
…tion checking for clang

Pending a resolution to #65606, this CL marks clang's ASAN runtime as
unable to symbolize stack traces to unblock the LUCI clang builder.

For #65606.
For #65469.
Fixes #65641.

Change-Id: I649773085aff30e5703e7f7ac2c72a0430a015c2
Cq-Include-Trybots: luci.golang.try:go1.22-linux-amd64-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/562675
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit d94ab59)
Reviewed-on: https://go-review.googlesource.com/c/go/+/563015
ezz-no pushed a commit to ezz-no/go-ezzno that referenced this issue Feb 18, 2024
@mknyszek @ezz-no
cmd/cgo/internal/testsanitizers: disable location checking for clang
8f5eb2a 
Pending a resolution to golang#65606, this CL marks clang's ASAN runtime as
unable to symbolize stack traces to unblock the LUCI clang builder.

For golang#65606.
Fixes golang#65469.

Change-Id: I649773085aff30e5703e7f7ac2c72a0430a015c2
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/562675
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
bradfitz pushed a commit to tailscale/go that referenced this issue Mar 5, 2024
@mknyszek @bradfitz
[release-branch.go1.22] cmd/cgo/internal/testsanitizers: disable loca…
20e5874 
…tion checking for clang

Pending a resolution to golang#65606, this CL marks clang's ASAN runtime as
unable to symbolize stack traces to unblock the LUCI clang builder.

For golang#65606.
For golang#65469.
Fixes golang#65641.

Change-Id: I649773085aff30e5703e7f7ac2c72a0430a015c2
Cq-Include-Trybots: luci.golang.try:go1.22-linux-amd64-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/562675
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit d94ab59)
Reviewed-on: https://go-review.googlesource.com/c/go/+/563015
romaindoumenc pushed a commit to TroutSoftware/go that referenced this issue Mar 6, 2024
@mknyszek @romaindoumenc
[release-branch.go1.22] cmd/cgo/internal/testsanitizers: disable loca…
e081268 
…tion checking for clang

Pending a resolution to golang#65606, this CL marks clang's ASAN runtime as
unable to symbolize stack traces to unblock the LUCI clang builder.

For golang#65606.
For golang#65469.
Fixes golang#65641.

Change-Id: I649773085aff30e5703e7f7ac2c72a0430a015c2
Cq-Include-Trybots: luci.golang.try:go1.22-linux-amd64-clang15
Reviewed-on: https://go-review.googlesource.com/c/go/+/562675
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit d94ab59)
Reviewed-on: https://go-review.googlesource.com/c/go/+/563015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mknyszek mknyszek

Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsFix The path to resolution is known, but the work has not been done.
Projects
Go Compiler / Runtime
Status: In Progress
Milestone
Backlog
Development

No branches or pull requests
2 participants
@mknyszek @gopherbot