The results seem to be random

[ldez]

Short summary

The results seem to be random: two calls don't provide the same results.

I found the issue during the analysis of this issue golangci/golangci-lint#1653

Steps to reproduce the behavior

# https://github.com/kopia/kopia
$ git@github.com:kopia/kopia.git
$ git checkout 1f3b8d4d
$ mnd ./...

go-mnd version or commit ref

I use my PR #19 because without that I was not able to run the CLI.

Go version (output of 'go version')

$ go version
go version go1.15.7 linux/amd64

Operating system / Environment

$ go version
go version go1.15.7 linux/amd64

Expected behavior

$ GOARCH=amd64 GOOS=windows mnd ./... 2>&1 | grep splitter.go | wc -l
13
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=darwin mnd ./... 2>&1 | grep splitter.go  | wc -l
13

Actual behavior

linux

$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
12
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
12
$ GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep splitter.go  | wc -l
13

darwin

$ GOARCH=amd64 GOOS=darwin mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=darwin mnd ./... 2>&1 | grep splitter.go  | wc -l
10
$ GOARCH=amd64 GOOS=darwin mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=darwin mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=darwin mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=darwin mnd ./... 2>&1 | grep splitter.go  | wc -l
13
$ GOARCH=amd64 GOOS=darwin mnd ./... 2>&1 | grep splitter.go  | wc -l
12

windows

$ GOARCH=amd64 GOOS=windows mnd ./... 2>&1 | grep splitter.go | wc -l
10
$ GOARCH=amd64 GOOS=windows mnd ./... 2>&1 | grep splitter.go | wc -l
12
$ GOARCH=amd64 GOOS=windows mnd ./... 2>&1 | grep splitter.go | wc -l
13
$ GOARCH=amd64 GOOS=windows mnd ./... 2>&1 | grep splitter.go | wc -l
12
$ GOARCH=amd64 GOOS=windows mnd ./... 2>&1 | grep splitter.go | wc -l
12
$ GOARCH=amd64 GOOS=windows mnd ./... 2>&1 | grep splitter.go | wc -l
13
$ GOARCH=amd64 GOOS=windows mnd ./... 2>&1 | grep splitter.go | wc -l
13

[jkowalski]

Note: to repro this with Kopia repo use kopia/kopia@1f3b8d4 instead of latest master, since we've merged the fix to make linter no longer flaky in kopia/kopia@04b99c7

[ldez]

The root cause seems to be the argument checker:

condition

for i in {1..10}; do GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep 'in <condition> detected'  | wc -l; done
5
5
5
5
5
5
5
5
5

operation

for i in {1..10}; do GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep 'in <operation> detected'  | wc -l; done
18
18
18
18
18
18
18
18
18
18

argument

for i in {1..10}; do GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep 'in <argument> detected'  | wc -l; done
37
40
36
37
38
36
39
37
39
38

return

for i in {1..10}; do GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep 'in <return> detected'  | wc -l; done
4
4
4
4
4
4
4
4
4
4

assign

$ for i in {1..10}; do GOARCH=amd64 GOOS=linux mnd ./... 2>&1 | grep 'in <assign> detected'  | wc -l; done
17
17
17
17
17
17
17
17
17
17

[ldez]

I found the problem I will open a PR

[tommy-muehle]

PR is merged and new release published! Thanks for the fix.