Test Go Library Examples

[mattnibs]

Add markdown test to run go vet on go library examples. Fix existing go library examples.

Closes #4703

[nwt]

I think docs/libraries/go.md needs an update.

[mattnibs]

@nwt updated

[philrz]

@mattnibs: Since I saw you made docs updates, I did zed-docs-site make dev so I could and walk through the docs as a user on a scratch Ubuntu 20.04 VM. It seems they're not working for me yet. Below is the output from my full session. Since your Go code references Arena and my go mod tidy output shows it downloading the last tagged release v1.15.0 that pre-dated Arena, I assume thats the source of the problem.

$ go version
go version go1.21.11 linux/amd64

$ mkdir example

$ cd example

$ go mod init example
go: creating new go.mod: module example

$ cat > main.go
package main

import (
    "fmt"
    "log"
    "os"

    "github.com/brimdata/zed"
    "github.com/brimdata/zed/zio/zsonio"
    "github.com/brimdata/zed/zson"
)

func main() {
    zctx := zed.NewContext()
    reader := zsonio.NewReader(zctx, os.Stdin)
    arena := zed.NewArena()
    for {
        val, err := reader.Read()
        if err != nil {
            log.Fatalln(err)
        }
        if val == nil {
            return
        }
        s := val.Deref(arena, "s")
        if s == nil {
            s = zctx.Missing(arena).Ptr()
        }
        fmt.Println(zson.String(s))
    }
}

$ go mod tidy
go: finding module for package github.com/brimdata/zed/zson
go: finding module for package github.com/brimdata/zed
go: finding module for package github.com/brimdata/zed/zio/zsonio
go: downloading github.com/brimdata/zed v1.15.0
go: found github.com/brimdata/zed in github.com/brimdata/zed v1.15.0
go: found github.com/brimdata/zed/zio/zsonio in github.com/brimdata/zed v1.15.0
go: found github.com/brimdata/zed/zson in github.com/brimdata/zed v1.15.0
go: downloading github.com/x448/float16 v0.8.4
go: downloading github.com/stretchr/testify v1.8.4
go: downloading golang.org/x/text v0.13.0
go: downloading github.com/segmentio/ksuid v1.0.2
go: downloading golang.org/x/sys v0.13.0
go: downloading github.com/apache/arrow/go/v14 v14.0.0
go: downloading github.com/pierrec/lz4/v4 v4.1.18
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/hashicorp/golang-lru/v2 v2.0.1
go: downloading go.uber.org/zap v1.23.0
go: downloading golang.org/x/sync v0.4.0
go: downloading github.com/aws/aws-sdk-go v1.36.17
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading golang.org/x/tools v0.14.0
go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2
go: downloading github.com/google/flatbuffers v23.5.26+incompatible
go: downloading github.com/klauspost/compress v1.16.7
go: downloading github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de
go: downloading github.com/agnivade/levenshtein v1.1.1
go: downloading github.com/axiomhq/hyperloglog v0.0.0-20191112132149-a4c4c47bc57f
go: downloading go.uber.org/atomic v1.7.0
go: downloading go.uber.org/multierr v1.8.0
go: downloading github.com/kr/text v0.2.0
go: downloading github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d
go: downloading github.com/klauspost/asmfmt v1.3.2
go: downloading github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8
go: downloading github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3
go: downloading github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c
go: downloading github.com/apache/thrift v0.17.0
go: downloading golang.org/x/exp v0.0.0-20231006140011-7918f672742d
go: downloading github.com/goccy/go-json v0.10.2
go: downloading github.com/zeebo/xxh3 v1.0.2
go: downloading google.golang.org/grpc v1.58.2
go: downloading google.golang.org/protobuf v1.31.0
go: downloading golang.org/x/term v0.13.0
go: downloading github.com/dgryski/go-metro v0.0.0-20180109044635-280f6062b5bc
go: downloading github.com/andybalholm/brotli v1.0.5
go: downloading github.com/golang/snappy v0.0.4
go: downloading github.com/klauspost/cpuid/v2 v2.2.5
go: downloading golang.org/x/mod v0.13.0
go: downloading golang.org/x/net v0.17.0
go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20231002182017-d307bd883b97
go: downloading github.com/jmespath/go-jmespath v0.4.0
go: downloading github.com/golang/protobuf v1.5.3

$ echo '{s:"hello"}{x:123}{s:"world"}' | go run .
# example
./main.go:16:18: undefined: zed.NewArena
./main.go:25:31: too many arguments in call to val.Deref
	have (unknown type, string)
	want (string)
./main.go:27:30: too many arguments in call to zctx.Missing
	have (unknown type)
	want ()

[philrz]

I discussed this one offline with @mattnibs and he confirmed my suspicion that the changes in this PR won't work until we tag the next Zed release.

In an attempt to "play user" with this in mind, I added the extra step of updating the go.mod so the zed line reflects the current tip of main:

require github.com/brimdata/zed v1.15.1-0.20240619181207-9c3f5fdcb1b0

Then I re-ran go mod tidy. At that point the "ZSON Reader" example ran ok for me. However, when I marched on to the "Local Lake Reader" example, I get a SEGV when I run it.

$ go run . ./scratch
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1629ccf]

goroutine 1 [running]:
go.uber.org/zap.(*Logger).clone(...)
	/Users/phil/.go/pkg/mod/go.uber.org/zap@v1.23.0/logger.go:278
go.uber.org/zap.(*Logger).Named(0x0, {0x258d379, 0x7})
	/Users/phil/.go/pkg/mod/go.uber.org/zap@v1.23.0/logger.go:156 +0x4f
github.com/brimdata/zed/lake/journal.newStore(0xc00001bdd0, 0x3618660?, {0xc00041fb70, 0x1, 0xc0005f5a70?})
	/Users/phil/.go/pkg/mod/github.com/brimdata/zed@v1.15.1-0.20240619181207-9c3f5fdcb1b0/lake/journal/store.go:82 +0x185
github.com/brimdata/zed/lake/journal.OpenStore({0x284f2c8?, 0x3618660?}, {0x28597f8?, 0xc0005a8300?}, 0x25?, 0x0?, {0xc00041fb70, 0x1, 0x1})
	/Users/phil/.go/pkg/mod/github.com/brimdata/zed@v1.15.1-0.20240619181207-9c3f5fdcb1b0/lake/journal/store.go:73 +0x6e
github.com/brimdata/zed/lake/pools.OpenStore({0x284f2c8?, 0x3618660?}, {0x28597f8?, 0xc0005a8300?}, 0xc00001b6b0?, 0xc00001b680?)
	/Users/phil/.go/pkg/mod/github.com/brimdata/zed@v1.15.1-0.20240619181207-9c3f5fdcb1b0/lake/pools/store.go:32 +0xc5
github.com/brimdata/zed/lake.(*Root).loadConfig(0xc0004d7d40, {0x284f2c8, 0x3618660})
	/Users/phil/.go/pkg/mod/github.com/brimdata/zed@v1.15.1-0.20240619181207-9c3f5fdcb1b0/lake/root.go:117 +0x87
github.com/brimdata/zed/lake.Open({0x284f2c8, 0x3618660}, {0x28597f8?, 0xc0005a8300?}, 0x27?, 0xc0005f5170)
	/Users/phil/.go/pkg/mod/github.com/brimdata/zed@v1.15.1-0.20240619181207-9c3f5fdcb1b0/lake/root.go:73 +0x59
github.com/brimdata/zed/lake/api.OpenLocalLake({0x284f2c8, 0x3618660}, 0x2589e5b?, {0xc0005dc660?, 0xc0005dc660?})
	/Users/phil/.go/pkg/mod/github.com/brimdata/zed@v1.15.1-0.20240619181207-9c3f5fdcb1b0/lake/api/local.go:36 +0x68
github.com/brimdata/zed/lake/api.OpenLake({0x284f2c8?, 0x3618660?}, 0xc00041fd18?, {0xc0005dc660?, 0x27?})
	/Users/phil/.go/pkg/mod/github.com/brimdata/zed@v1.15.1-0.20240619181207-9c3f5fdcb1b0/lake/api/api.go:49 +0xef
main.main()
	/Users/phil/work/example/main.go:24 +0x145
exit status 2

[mattnibs]

#5145

[nwt]

The addition to mdtest feels pretty clunky to me but I'm OK with merging it.

[mattnibs]

@nwt would you rather we just not test this?