cmd/go: go get fails to create zip for package containing empty files

[plastikfan]

What version of Go are you using (go version)?

$ go version
go version go1.19.7 linux/amd64

Does this issue reproduce with the latest release?

Yes, the latest 19 version. I can't use go20, because linting is broken in that version, which then breaks my entire build chain.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
 plastikfan@Janus  ~/dev/github/go/snivilised/arcadia λ   feat/add-logger ±  go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/plastikfan/.cache/go-build"
GOENV="/home/plastikfan/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/plastikfan/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/plastikfan/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.7"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/plastikfan/dev/github/go/snivilised/arcadia/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4218326177=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have created a library extendio, which contains quite a lot of test files (.txt, .flac and .jpeg) and since the content of the fles is irrelevant, (only their presence matters for the test cases) they are all empty files. However, when a client tries to go get the module, a whole load of errors are generated whilst trying to create the zip file.

To reproduce this error, just run:

go get github.com/snivilised/extendio

The really odd thing is that this doesnt fail in github actions, but that doesnt use go get, uses actions/checkout instead, which works fine.

What did you expect to see?

Nothing

What did you see instead?

 plastikfan@Janus  ~/dev/github/go/snivilised/arcadia λ   feat/add-logger ±  go get github.com/snivilised/extendio
go: downloading github.com/snivilised/extendio v0.1.1
go: github.com/snivilised/extendio: create zip: Test/data/MUSICO/DREAM-POP/Tori Amos/Strange Little Girls/02 - '97 Bonnie & Clyde.flac: malformed file path "Test/data/MUSICO/DREAM-POP/Tori Amos/Strange Little Girls/02 - '97 Bonnie & Clyde.flac": invalid char '\''
Test/data/MUSICO/ELECTRONIC-POP/Traci Lords/Control (Single)/01 - Control (Olympic 7'' edit).flac: malformed file path "Test/data/MUSICO/ELECTRONIC-POP/Traci Lords/Control (Single)/01 - Control (Olympic 7'' edit).flac": invalid char '\''
Test/data/MUSICO/POP/Madonna/Bedtime Stories/03 - I'd Rather Be Your Lover.flac: malformed file path "Test/data/MUSICO/POP/Madonna/Bedtime Stories/03 - I'd Rather Be Your Lover.flac": invalid char '\''
Test/data/MUSICO/POP/Madonna/Bedtime Stories/04 - Don't Stop.flac: malformed file path "Test/data/MUSICO/POP/Madonna/Bedtime Stories/04 - Don't Stop.flac": invalid char '\''
Test/data/MUSICO/POP/Madonna/MDNA/1.03 - I'm Addicted.flac: malformed file path "Test/data/MUSICO/POP/Madonna/MDNA/1.03 - I'm Addicted.flac": invalid char '\''
Test/data/MUSICO/PROGRESSIVE-HOUSE/Blue Amazon/Long Way Home (Vinyl Single)/01 - Long Way Home (Blue Amazon's Trip TV Mix).flac: malformed file path "Test/data/MUSICO/PROGRESSIVE-HOUSE/Blue Amazon/Long Way Home (Vinyl Single)/01 - Long Way Home (Blue Amazon's Trip TV Mix).flac": invalid char '\''

Now, I am aware there are a whole load of restrictions about files that can be contained within a zip file (from issue #41402), but none of the restrictions cover empty files. I don't know if the problem is because of empty files, I am just speculating, because it's not clear what the problem is. The message invalid char '\'' doesn't make sense because that char doesn't exist anywhere in the file content or within the path of any of the files. (I thought it might be that I had referenced a file via a path containing a '\'', which is invalid in linux, but this is not the case.)

If these empty files are the cause, then shouldn't there be a check when you add the file to a repo (or some other go specic step/cmd) to warning that an empty file in the repo will break the zip creation process invoked by go get?

All I need for now is a clear indication of why this is failing. Empty files may not be the cause and none of the other restrictions that I have read at constraints, fits the bill.

[bcmills]

What leads you to believe that the problem is that the files are empty? The error message is talking about the quote characters in the filenames themselves.

Per https://go.dev/ref/mod#zip-path-size-constraints:

File and directory names within a module may consist of Unicode letters, ASCII digits, the ASCII space character (U+0020), and the ASCII punctuation characters !#$%&()+,-.=@[]^_{}~. Note that package paths may not contain all these characters. See module.CheckFilePath and module.CheckImportPath for the differences.

[plastikfan]

Hi @bcmills, yeah I did another test in a different test repo and its not empty files. But all the files that I'm work with are all empty, there is no content with a "\", so I don't know whats going wrong. I am still investigating, but as far as I know, there are no paths that contain a "\" either.

[plastikfan]

Oh, this is staring me in the face. I misinterpretted the error. I thought the problem was \, where as the problem is the use of the ' character. I hadn't realised that the \ was being used an escape for the '. My bad, I do need to confirm though ...

Really, the error message needs to be changed from

invalid char '\''

to something that is human friendly:

invalid char '

[plastikfan]

Confirmed, closing