cmd/go: create zip: malformed file path: invalid char '·'

[navytux]

Hello up there.

I've tried to modularize my workspace and hit the error shown in subject.
Unlike #40749 here the "invalid" character is in file name that is outside of any go package just living in the same repository in a separate directory. Please find details below.

Thanks beforehand,
Kirill

What version of Go are you using (go version)?

$ go version
go version go1.15.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/kirr/.cache/go-build"
GOENV="/home/kirr/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/kirr/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/kirr/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/kirr/src/tools/go/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/kirr/src/tools/go/go/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/kirr/tmp/trashme/try1/wendelin.core/wcfs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build596838834=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.15.2 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.15.2
uname -sr: Linux 5.7.0-3-amd64
Distributor ID:	Debian
Description:	Debian GNU/Linux bullseye/sid
Release:	testing
Codename:	bullseye
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Debian GLIBC 2.31-3) stable release version 2.31.
gdb --version: GNU gdb (Debian 9.2-1) 9.2

What did you do?

$ go get lab.nexedi.com/kirr/neo/go/zodb@37dc79789954

What did you expect to see?

go get succeeds.

What did you see instead?

$ go get lab.nexedi.com/kirr/neo/go/zodb@37dc79789954
go: downloading lab.nexedi.com/kirr/neo v1.8.2-0.20200803093418-37dc79789954
go get lab.nexedi.com/kirr/neo/go/zodb@37dc79789954: create zip: malformed file path "t/time/20180321-neo1·P4.txt": invalid char '·'

[navytux]

A similar failure with ':' after workaround:

$ go get lab.nexedi.com/kirr/neo/go@t
go: downloading lab.nexedi.com/kirr/neo v1.8.2-0.20200915132355-8783ff63fc24
go get lab.nexedi.com/kirr/neo/go@t: create zip: malformed file path "t/time/time-soct12-neo1-neo2.txt-tgro20-txc:200µs4f": invalid char ':'

[jayconrod]

This is working as intended I'm afraid.

There are a number of constraints on file and directory names within modules. These constraints ensure that module zip files can be extracted consistently on a wide variety of file systems.

If you don't want to include those files in the module archive, you can add an empty go.mod file to the parent directory.

[navytux]

oh, I see. Ok...

[mewmew]

I know this issue is closed. So perhaps this should be a new issue.

@jayconrod I was just wondering if there is a go get flag to use git clone directly instead of downloading Go modules as ZIP files. The constraints listed in https://golang.org/ref/mod#zip-path-size-constraints are quite restrictive (e.g. max 500 MB), so being able to handle Go modules that fail to satisfy one of these constraints would be desirable; e.g. go get -zip=false github.com/foo/bar

Cheers,
Robin

[jayconrod]

@mewmew Sorry, no. While the go command can clone repositories using git in direct mode (with GOPROXY=direct or with an appropriate GONOPROXY setting), the same restrictions apply.

The goal here is to ensure that modules can be extracted and used consistently on a wide variety of platforms and file systems.

The constraint you mentioned in particular (500MB size limit) is nearly always enough for source code. You can exclude directories containing test data and other large files by placing a go.mod file (even an empty one) at the top of it.

[mewmew]

Thank you for the added background information @jayconrod.

Cheers,
Robin

[thediveo]

Unfortunately, I have unit tests that need to exactly reproduce with a test file that ends in .. This file name replicates the situation on a product and our tests need to reproduce this situation exactly. The product is a Linux-only product, so we really don't care about the "needs to work across all file systems". At the moment, Go breaks our product development. Thank you so much.

[MawKKe]

@thediveo as a workaround you could create those files dynamically in the test setup, i.e. those files with Go-incompatible names are never commited into the repo.

[thediveo]

True, but that's very cumbersome and error prone.