Error on Github Actions: could not import C (cgo preprocessing failed)

[grongor]

• Yes, I'm using a binary release within 2 latest major releases. Only such installations are supported.
• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've included all information below (version, config, etc).

Version of golangci-lint

Run golangci/golangci-lint-action@v1
  with:
    version: v1.27

Config file

run:
    modules-download-mode: readonly

linters:
    enable-all: true
    disable:
        - depguard
        - dogsled
        - funlen
        - godox
        - gofmt
        - golint
        - gomnd
        - interfacer

linters-settings:
    gocritic:
        disabled-checks:
            - captLocal

    govet:
        enable-all: true
        disable:
            - shadow

    lll:
        tab-width: 4

    wsl:
        allow-cuddle-declarations: true

issues:
    max-issues-per-linter: 0
    max-same-issues: 0
    exclude-rules:
        # ignore unchecked errors and missing Close() calls in tests
        -   path: _test\.go
            linters: [errcheck, bodyclose]
        # ignore recommendations for variable names, e.g id => ID, uri => URI, ...
        -   text: "^ST1003:"
            linters: [stylecheck]
        # ignore line length limit in tests
        -   path: _test\.go
            text: line is \d+ characters
            linters: [lll]
        # ignore defer cuddle in tests
        -   path: _test\.go
            text: only one cuddle assignment allowed before defer statement
            linters: [wsl]
        # ignore expressions after assignment in tests
        -   path: _test\.go
            text: only cuddled expressions if assigning variable or using from line above
            linters: [wsl]
        # ignore scopelint in tests for "test" variable (probably bug, maybe remove this later)
        -   path: _test\.go
            text: Using the variable on range scope `test` in function literal
            linters: [scopelint]
        # ignore goerr113 dynamic errors definition error...not sure how to approach this correctly now
        -   text: do not define dynamic errors, use wrapped static errors instead
            linters: [goerr113]

Go environment

go version go1.14.4 linux/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/runner/.cache/go-build"
GOENV="/home/runner/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/runner/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/hostedtoolcache/go/1.14.4/x64"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/hostedtoolcache/go/1.14.4/x64/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/runner/work/go-snmp-proxy/go-snmp-proxy/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build154314408=/tmp/go-build -gno-record-gcc-switches"

Verbose output of running

level=info msg="[config_reader] Used config file .golangci.yml"
level=info msg="[lintersdb] Active 40 linters: [asciicheck bodyclose deadcode dupl errcheck gochecknoglobals gochecknoinits gocognit goconst gocritic gocyclo godot goerr113 goimports gomodguard goprintffuncname gosec gosimple govet ineffassign lll maligned misspell nakedret nestif nolintlint prealloc rowserrcheck scopelint staticcheck structcheck stylecheck testpackage typecheck unconvert unparam unused varcheck whitespace wsl]"
level=info msg="[loader] Go packages loading at mode 575 (exports_file|files|imports|types_sizes|compiled_files|deps|name) took 407.861258ms"
level=info msg="[runner/filename_unadjuster] Pre-built 0 adjustments in 4.949238ms"
level=info msg="[linters context/goanalysis] analyzers took 10.059387857s with top 10 stages: buildir: 8.312565252s, inspect: 344.899191ms, fact_purity: 299.720098ms, printf: 226.774459ms, findcall: 215.589417ms, ctrlflow: 196.536901ms, fact_deprecated: 177.667232ms, the_only_name: 95.11866ms, buildssa: 94.015571ms, unconvert: 39.563129ms"
level=warning msg="[runner] Can't run linter goanalysis_metalinter: bodyclose: failed prerequisites: [buildssa@github.com/grongor/go-snmp-proxy/snmpproxy: analysis skipped: errors in package: [/home/runner/work/go-snmp-proxy/go-snmp-proxy/snmpproxy/mib.go:9:8: could not import C (cgo preprocessing failed)]]"
level=info msg="[linters context/goanalysis] analyzers took 10.166717ms with top 10 stages: buildir: 9.767185ms, U1000: 399.532µs"
level=warning msg="[runner] Can't run linter unused: buildir: analysis skipped: errors in package: [/home/runner/work/go-snmp-proxy/go-snmp-proxy/snmpproxy/mib.go:9:8: could not import C (cgo preprocessing failed)]"
level=info msg="[runner] processing took 3.8µs with stages: skip_dirs: 500ns, nolint: 500ns, max_same_issues: 500ns, filename_unadjuster: 300ns, max_from_linter: 300ns, cgo: 300ns, skip_files: 200ns, exclude: 200ns, exclude-rules: 200ns, identifier_marker: 100ns, autogenerated_exclude: 100ns, uniq_by_line: 100ns, source_code: 100ns, path_prettifier: 100ns, max_per_file_from_linter: 100ns, diff: 100ns, path_shortener: 100ns"
level=info msg="[runner] linters took 7.8340658s with stages: goanalysis_metalinter: 7.618676219s, unused: 215.339577ms"
level=error msg="Running error: buildir: analysis skipped: errors in package: [/home/runner/work/go-snmp-proxy/go-snmp-proxy/snmpproxy/mib.go:9:8: could not import C (cgo preprocessing failed)]"
level=info msg="Memory: 84 samples, avg is 234.2MB, max is 339.8MB"
level=info msg="Execution took 8.256899979s"

Hello, I have an issue with golangci-lint in Github Actions. I tried using the official golangci/golangci-lint-action@v1, then tried manually installing the linter and then I even tried running it multiple times just to see if it's not a random fail. Neither run was succesful and I'm always getting some errors regarding could not import C (cgo preprocessing failed). Strange thing is that if I run the linter locally, everything works as expected (I even tried multiple versions...), see screenshot:

Here are the failed builds (with detailed info):

• using the Github Action: https://github.com/grongor/go-snmp-proxy/runs/734271995
• direct install, single try: https://github.com/grongor/go-snmp-proxy/runs/734293704
• direct install, multiple tries: https://github.com/grongor/go-snmp-proxy/runs/734297231

I also tried removing the golangci-lint locally, and running it in the same way as I do on CI ... and it works:

╭─grongor@grongor-nb ~/projects/grongor/go-snmp-proxy  ‹test› 
╰─➤  make clean                                                                                                                                                                                                                                                                                                          2 ↵
rm -rf bin
╭─grongor@grongor-nb ~/projects/grongor/go-snmp-proxy  ‹test*› 
╰─➤  make lint
curl --retry 5 -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s v1.27.0
golangci/golangci-lint info checking GitHub for tag 'v1.27.0'
golangci/golangci-lint info found version: 1.27.0 for v1.27.0/linux/amd64
golangci/golangci-lint info installed ./bin/golangci-lint
/home/grongor/projects/grongor/go-snmp-proxy/bin/golangci-lint run
╭─grongor@grongor-nb ~/projects/grongor/go-snmp-proxy  ‹test*› 
╰─➤  echo $?
0

[bombsimon]

The reason for this working on your local machine and not in the workflow is because you've also installed some dependent C libraries on your machine (net-snmp/net-snmp-config.h and net-snmp/mib_api.h). I get the same error on a fresh machine with any version of golangic-lint. For the linting to work the code needs to be able to compile. This is not possible without the headers as seen with go test.

$ go test ./...
# github.com/grongor/go-snmp-proxy/snmpproxy
snmpproxy/mib.go:6:10: fatal error: net-snmp/net-snmp-config.h: No such file or directory
 #include <net-snmp/net-snmp-config.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Reproducable MVP in Docker outside GitHub action:

FROM golang:latest                                                                
                                                                                  
WORKDIR /workspace                                                                
                                                                                  
RUN curl -sSfL \                                                                  
    https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | \
    sh -s -- -b $(go env GOPATH)/bin v1.27.0                                      
                                                                                  
RUN git clone https://github.com/grongor/go-snmp-proxy.git                        
                                                                                  
WORKDIR /workspace/go-snmp-proxy                                                  
                                                                                  
RUN git fetch origin && git checkout first-version                                
                                                                                  
                                                                                  
CMD [ "golangci-lint", "run" ]                                                    

$ docker build -t lint .
$ docker run --rm lint 
level=warning msg="[runner] Can't run linter goanalysis_metalinter: bodyclose: failed prerequisites: [buildssa@github.com/grongor/go-snmp-proxy/snmpproxy: analysis skipped: errors in package: [/workspace/go-snmp-proxy/snmpproxy/mib.go:9:8: could not import C (cgo preprocessing failed)]]"
level=warning msg="[runner] Can't run linter unused: buildir: analysis skipped: errors in package: [/workspace/go-snmp-proxy/snmpproxy/mib.go:9:8: could not import C (cgo preprocessing failed)]"
level=error msg="Running error: buildir: analysis skipped: errors in package: [/workspace/go-snmp-proxy/snmpproxy/mib.go:9:8: could not import C (cgo preprocessing failed)]"

So to sum it up, you need to add your header files in /usr/local/include in the action before running golangci-lint.

[grongor]

OMG! I should've figured that out...This was my first time playing with cgo so I wasn't sure about many things. I'm so sorry for wasting your time like that and thank you for taking the time to write such a detailed answer.

Just for reference, the solution was really simple:

-   name: Install dependencies
    run: sudo apt update && sudo apt install libsnmp-dev

[grongor]

By the way, do you @bombsimon think that it would be possible to improve the error message somehow, ideally telling the user that it wasn't able to include the required headers? That would immediately put me on the right path...at first I though "ok, golangci-lint doesn't work with C imports at all"...if so, I would take look it :)

[bombsimon]

I don't think golangci-lint should say what kind of compilation error occurs, although requests to make it more clear that it's a compilation error and that the code won't lint since it wont compile exist. Something in line with what you're asking about is discussed in #1043 and also related in #1172.

[charleskorn]

Do you have any suggestions on how to debug this issue? I'm seeing it for my project where everything works fine locally, and building on CI works without issue, but golangci-lint fails on CI. Is there some way to get more detailed information on what has gone wrong?

[jatinkapoor1313]

OMG! I should've figured that out...This was my first time playing with cgo so I wasn't sure about many things. I'm so sorry for wasting your time like that and thank you for taking the time to write such a detailed answer.

Just for reference, the solution was really simple:

-   name: Install dependencies
    run: sudo apt update && sudo apt install libsnmp-dev

still not clear what to do in order to resolve this