Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

depsync: add xk6-depsync command #72

Closed
wants to merge 11 commits into from
Closed

depsync: add xk6-depsync command #72

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
b2111f6
depsync: add xk6-depsync command
roobre Sep 22, 2023
0be4201
depsync: fix typo
roobre Oct 5, 2023
f562cbf
README: add section describing `xk6-depsync`
roobre Oct 5, 2023
8ae8675
depsync: use golang.org/x/mod/modfile for parsing
roobre Oct 6, 2023
6bf8353
go.mod: tidy and vendor
roobre Oct 6, 2023
1a15a46
Makefile: add xk6-depsync to `build` target
roobre Oct 6, 2023
87a8947
depsync: defer close response body
roobre Nov 9, 2023
c71ad22
Dockerfile: revert addition of xk6-depsync
roobre Nov 10, 2023
622fc4f
fixup! README: add section describing `xk6-depsync`
roobre Nov 10, 2023
c9cc4ff
fixup! go.mod: tidy and vendor
roobre Nov 10, 2023
43ae42e
depsync: move command to its own module
roobre Nov 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,13 @@ help:
## clean: Removes any previously created build artifacts.
clean:
rm -f ./xk6
rm -f ./xk6-depsync
rm -f ./k6

## build: Builds the 'xk6' binary.
build:
go build -work ./cmd/xk6
go build -work ./cmd/xk6-depsync/

## format: Applies Go formatting to code.
format:
Expand Down
34 changes: 34 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,40 @@ Because the subcommands and flags are constrained to benefit rapid extension pro
- `XK6_K6_REPO` optionally sets the path to the main k6 repository. This is useful when building with k6 forks.


## `xk6-depsync`

In addition to `xk6`, this repository also includes the `xk6-depsync` command. `xk6-depsync` can be used to find common dependencies between a given go project, typically a k6 extension, and k6 core. For these common dependencies, `xk6-depsync` will look specifically for those that are required to be on a different version than what k6 uses, and generate a `go get` command to set it to the correct version. This is considered a good chance to minimize the chance of build-time errors and binary compatibility issues.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do understand that this is related to xk6 (somewhat) and as this it seems like a good idea to keep it in the same repo, but:

Having gone through the code - none of the new code is dependent on xk6 or ... even specific about xk6. But it does add more dependency and makes the README harder to read by making it bigger.

The tool seems to be useful as well for any other projects trying to align their dependencies between different versions.

As such it seems to me it will be more useful to:

  1. Move this whole thing to a separate repo.
  2. the tool to take in another module name as argument.
  3. document here that you can use that repo to sync dependencies.

Copy link
Member Author

@roobre roobre Nov 9, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that from an organizational point of view this tool should be its own module and/or repository.

However, considering that the tool itself is small and, for being a new tool, pretty much unused, I find hard to justify at this point of the tool's lifespan the operational burden of creating a new repository with is corresponding permissions, settings, ci/cd and release pipelines, etc.

I think we can always split the repo (or merge the command) if it starts making sense later on, while benefiting from the simpler operation and discoverability that sharing the repo provides.

Regarding:

But it does add more dependency

I agree and mentioned this here. I'll be okay with reverting 8ae8675 and switch back to the homemade string parser that does not have any external dependency at the cost of perhaps being less conformant with the go.mod spec, if you're okay with that tradeoff.

Copy link

@codebien codebien Nov 9, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roobre I miss this during my reviews, sorry. Why not create a dedicated go.mod file? This is an independent project in the end, being in the same repo doesn't force us to have the same project from Go's point of view.

If we can divide the Go Modules then I share the same feeling as you about having a different repository. A dedicated repo for only one file sounds a bit too much. IMHO.

Copy link
Member Author

@roobre roobre Nov 10, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codebien No problem, I think creating a different go.mod should be okay. As we already have a go.mod in the root though, do you have any preference about where to put the new one? Would cmd/xk6-depsync/go.mod be okay? Should I also vendor the deps of the new module?

Copy link

@codebien codebien Nov 10, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would cmd/xk6-depsync/go.mod be okay?

Yep, it should be the idiomatic way these days for it. If you can do a quick research to confirm even better.

Should I also vendor the deps of the new module?

I don't think it's required. It's a very simple tool. But I don't have a strong opinion here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave this a try. In order for go build to work, I had to add a go workspace. Otherwise the toolchain does not seem to understand it is a different module:

12:18:03 ~/Staging/xk6 $> go build ./cmd/xk6-depsync
main module (go.k6.io/xk6) does not contain package go.k6.io/xk6/cmd/xk6-depsync

I've tested using my fork that go install works as expected:

12:30:39 ~/Staging/xk6/cmd/xk6-depsync $> go install github.com/roobre/xk6/cmd/xk6-depsync@latest
go: downloading github.com/roobre/xk6/cmd/xk6-depsync v0.0.0-20231110113016-4811bd1cf67a

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codebien Eager to hear your thoughts on this :)

If you're not positive about the approach (go.work file and all) then perhaps we should go for the route that @mstoykov suggested and just create a new repo for this tool.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving the project to another repo also has the benefit of making the tool more general - taking the repo it is supposed to take in as an argument.

Also both tools won't get updated eveyr time the other is

At this point I have to say that :

  1. repos are free.
  2. CI for open source project is free.
  3. copying .github folders takes seconds

I would expect there is even a template repo that makes the above a single click

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roobre I'm fine with moving it. 👍


### `xk6-depsync` usage

`xk6-depsync` can be installed using `go install`:

```console
$ go install go.k6.io/xk6/cmd/xk6-depsync@latest
```

After that, it is ready to be used:

```bash
xk6-depsync
```

Running the command above on the root of a k6 extension will produce an output like the following:

```
2023/10/05 14:08:30 detected k6 core version v0.45.1
2023/10/05 14:08:30 Mismatched versions for gopkg.in/guregu/null.v3: v3.5.0 (this package) -> v3.3.0 (core)
2023/10/05 14:08:30 Mismatched versions for github.com/spf13/afero: v1.9.5 (this package) -> v1.1.2 (core)
2023/10/05 14:08:30 Mismatched versions for github.com/google/pprof: v0.0.0-20230426061923-93006964c1fc (this package) -> v0.0.0-20230207041349-798e818bf904 (core)
go get github.com/google/pprof@v0.0.0-20230207041349-798e818bf904 github.com/spf13/afero@v1.1.2 gopkg.in/guregu/null.v3@v3.3.0
```

The final line includes the `go get` command that, when run, will sync the versions of the detected shared dependencies to the version that k6 is using. `xk6-depsync` outputs this line to `stdout` so it can be piped to a shell, or redirected to a script for later use.

### Trivia

Go versions earlier than 1.21 have been observed to behave unexpectedly in some cases, sometimes ignoring some of the versions specified in the `go get` command, or unexpectedly upgrading/downgrading other dependencies. It is recommended to run the `go get` commands suggested by `xk6-depsync` with Go >= 1.21.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give an example of this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we faced this in the xk6-disruptor project. We introduced this code as a script in this commit, which at the time yielded the following command:

go get github.com/spf13/afero@v1.1.2 github.com/spf13/cobra@v1.4.0 golang.org/x/sys@v0.9.0 google.golang.org/grpc@v1.56.1

Running said command with Go 1.21 produced the following changes, which I would pretty much expect:

diff --git a/go.mod b/go.mod
index a8cc4d4..1cdab88 100644
--- a/go.mod
+++ b/go.mod
@@ -4,10 +4,11 @@ go 1.19
 
 require (
 	github.com/docker/docker v24.0.5+incompatible
+	github.com/docker/go-connections v0.4.0
 	github.com/dop251/goja v0.0.0-20230621100801-7749907a8a20
 	github.com/google/go-cmp v0.5.9
 	github.com/sirupsen/logrus v1.9.3
-	github.com/spf13/cobra v1.5.0
+	github.com/spf13/cobra v1.4.0
 	github.com/testcontainers/testcontainers-go v0.21.0
 	go.k6.io/k6 v0.46.0
 	k8s.io/api v0.27.4
@@ -24,7 +25,6 @@ require (
 	github.com/containerd/containerd v1.7.3 // indirect
 	github.com/cpuguy83/dockercfg v0.3.1 // indirect
 	github.com/docker/distribution v2.8.2+incompatible // indirect
-	github.com/docker/go-connections v0.4.0 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
 	github.com/emicklei/go-restful/v3 v3.10.1 // indirect
 	github.com/google/pprof v0.0.0-20230207041349-798e818bf904 // indirect
@@ -61,7 +61,7 @@ require (
 	github.com/golang/protobuf v1.5.3
 	github.com/google/gnostic v0.5.7-v3refs // indirect
 	github.com/google/gofuzz v1.2.0 // indirect
-	github.com/google/uuid v1.3.0
+	github.com/google/uuid v1.3.0 // indirect
 	github.com/imdario/mergo v0.3.15 // indirect
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
 	github.com/jhump/protoreflect v1.15.1
@@ -78,17 +78,17 @@ require (
 	github.com/pelletier/go-toml v1.9.5 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
 	github.com/serenize/snaker v0.0.0-20201027110005-a7ad2135616e // indirect
-	github.com/spf13/afero v1.2.2 // indirect
+	github.com/spf13/afero v1.1.2 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/testcontainers/testcontainers-go/modules/k3s v0.21.0
 	golang.org/x/net v0.11.0 // indirect
 	golang.org/x/oauth2 v0.7.0 // indirect
-	golang.org/x/sys v0.11.0 // indirect
+	golang.org/x/sys v0.9.0 // indirect
 	golang.org/x/term v0.9.0 // indirect
 	golang.org/x/text v0.10.0 // indirect
 	golang.org/x/time v0.3.0 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/grpc v1.57.0-dev
+	google.golang.org/grpc v1.56.1
 	google.golang.org/protobuf v1.31.0
 	gopkg.in/guregu/null.v3 v3.3.0 // indirect
 	gopkg.in/inf.v0 v0.9.1 // indirect

With Go 1.19, however, yielded these changes instead:

diff --git a/go.mod b/go.mod
index a8cc4d4..776d9b4 100644
--- a/go.mod
+++ b/go.mod
@@ -3,32 +3,38 @@ module github.com/grafana/xk6-disruptor
 go 1.19
 
 require (
-	github.com/docker/docker v24.0.5+incompatible
+	github.com/docker/docker v24.0.6+incompatible
+	github.com/docker/go-connections v0.4.0
 	github.com/dop251/goja v0.0.0-20230621100801-7749907a8a20
 	github.com/google/go-cmp v0.5.9
 	github.com/sirupsen/logrus v1.9.3
-	github.com/spf13/cobra v1.5.0
-	github.com/testcontainers/testcontainers-go v0.21.0
+	github.com/spf13/cobra v1.4.0
+	github.com/testcontainers/testcontainers-go v0.24.0
+	github.com/testcontainers/testcontainers-go/modules/k3s v0.24.0
 	go.k6.io/k6 v0.46.0
-	k8s.io/api v0.27.4
-	k8s.io/apimachinery v0.27.4
-	k8s.io/client-go v0.27.4
+	k8s.io/api v0.28.1
+	k8s.io/apimachinery v0.28.1
+	k8s.io/client-go v0.28.1
 	sigs.k8s.io/kind v0.13.0
 )
 
 require (
+	dario.cat/mergo v1.0.0 // indirect
 	github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
 	github.com/Microsoft/go-winio v0.6.1 // indirect
+	github.com/Microsoft/hcsshim v0.11.0 // indirect
 	github.com/bufbuild/protocompile v0.4.0 // indirect
 	github.com/cenkalti/backoff/v4 v4.2.1 // indirect
-	github.com/containerd/containerd v1.7.3 // indirect
+	github.com/containerd/containerd v1.7.6 // indirect
 	github.com/cpuguy83/dockercfg v0.3.1 // indirect
 	github.com/docker/distribution v2.8.2+incompatible // indirect
-	github.com/docker/go-connections v0.4.0 // indirect
 	github.com/docker/go-units v0.5.0 // indirect
 	github.com/emicklei/go-restful/v3 v3.10.1 // indirect
+	github.com/go-ole/go-ole v1.2.6 // indirect
+	github.com/google/gnostic-models v0.6.8 // indirect
 	github.com/google/pprof v0.0.0-20230207041349-798e818bf904 // indirect
 	github.com/klauspost/compress v1.16.6 // indirect
+	github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
 	github.com/magiconair/properties v1.8.7 // indirect
 	github.com/moby/patternmatcher v0.5.0 // indirect
 	github.com/moby/sys/sequential v0.5.0 // indirect
@@ -38,10 +44,18 @@ require (
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/opencontainers/image-spec v1.1.0-rc4 // indirect
 	github.com/opencontainers/runc v1.1.5 // indirect
+	github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
+	github.com/shirou/gopsutil/v3 v3.23.7 // indirect
+	github.com/shoenig/go-m1cpu v0.1.6 // indirect
+	github.com/tklauser/go-sysconf v0.3.11 // indirect
+	github.com/tklauser/numcpus v0.6.0 // indirect
+	github.com/yusufpapurcu/wmi v1.2.3 // indirect
 	golang.org/x/exp v0.0.0-20230510235704-dd950f8aeaea // indirect
-	golang.org/x/mod v0.9.0 // indirect
-	golang.org/x/tools v0.7.0 // indirect
-	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
+	golang.org/x/mod v0.10.0 // indirect
+	golang.org/x/tools v0.8.0 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20230525234030-28d5490b6b19 // indirect
+	gotest.tools/v3 v3.5.1 // indirect
+	k8s.io/kube-openapi v0.0.0-20230717233707-2695361300d9 // indirect
 )

Which were pretty unexpected to me, and were pretty much contradicting the command, which asked for google.golang.org/grpc@v1.56.1 and yet yielded:

-	google.golang.org/grpc v1.57.0-dev
+	google.golang.org/grpc v1.57.0


---

> This project originally forked from the [xcaddy](https://github.com/caddyserver/xcaddy) project. **Thank you!**
5 changes: 5 additions & 0 deletions cmd/xk6-depsync/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module go.k6.io/xk6/cmd/xk6-depsync

go 1.16

require golang.org/x/mod v0.14.0
44 changes: 44 additions & 0 deletions cmd/xk6-depsync/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.14.0 h1:dGoOF9QVLYng8IHTm7BAyWqCqSheQ5pYWGhzW00YJr0=
golang.org/x/mod v0.14.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
108 changes: 108 additions & 0 deletions cmd/xk6-depsync/main.go
View file
Open in desktop
roobre marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
// main is the main package for xk6-depsync, a script that checks and provides commands to synchronize
// common dependencies with k6 core.
package main

import (
"fmt"
"io"
"log"
"net/http"
"os"
"sort"
"strings"

"golang.org/x/mod/modfile"
)

const (
k6Core = "go.k6.io/k6"
k6GoModURL = "https://proxy.golang.org/" + k6Core + "/@v/%s.mod"
)

func main() {
gomod := "go.mod"
if len(os.Args) >= 2 {
gomod = os.Args[1]
}

file, err := os.Open(gomod)
if err != nil {
log.Fatalf("opening local go.mod: %v", err)
}

ownDeps, err := dependencies(file)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using https://pkg.go.dev/golang.org/x/mod@v0.13.0/modfile#Parse instead of custom parsing logic, it would be a more robust solution....

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've given this a swing in the latest two commits. I wouldn't say I'm against it, but I'm not very convinced either. I don't see the code significantly less complex or significantly shorter either.

LMK what you think about this. If you prefer it this way, I'm happy letting it stay, otherwise I can drop the commits and get back to the original implementation. Your call :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't recommend it because of the complexity or length of the code. If there is a well-maintained, high-quality library for reading a file format, it is advisable to use it. This way you can avoid that your own parser does not exactly follow the file format specification.

BTW this is just a suggestion, it's up to the code maintainer to decide :)
( @olegbespalov ? )

if err != nil {
log.Fatalf("reading dependencies: %v", err)
}

k6Version := ownDeps[k6Core]
if k6Version == "" {
log.Fatalf("k6 core %q not found in %q", k6Core, gomod)
}

log.Printf("detected k6 core version %s", k6Version)

k6CoreVersionedURL := fmt.Sprintf(k6GoModURL, k6Version)
response, err := http.Get(k6CoreVersionedURL)
if err != nil {
log.Fatalf("error fetching k6 go.mod: %v", err)
}

defer response.Body.Close()

if response.StatusCode != http.StatusOK {
log.Fatalf("got HTTP status %d for %s", response.StatusCode, k6CoreVersionedURL)
}

coreDeps, err := dependencies(response.Body)
if err != nil {
log.Fatalf("reading k6 core dependencies: %v", err)
}

//nolint:prealloc // Number of mismatched deps cannot be accurately estimated.
var mismatched []string
for dep, version := range ownDeps {
coreVersion, inCore := coreDeps[dep]
if !inCore {
continue
}

if version == coreVersion {
continue
}

log.Printf("Mismatched versions for %s: %s (this package) -> %s (core)", dep, version, coreVersion)
mismatched = append(mismatched, fmt.Sprintf("%s@%s", dep, coreVersion))
}

if len(mismatched) == 0 {
log.Println("All deps are in sync, nothing to do.")
return
}

// TODO: Use slices.Sort when we move to a go version that has it on the stdlib.
sort.Strings(mismatched)

//nolint:forbidigo // We are willingly writing to stdout here.
fmt.Printf("go get %s\n", strings.Join(mismatched, " "))
olegbespalov marked this conversation as resolved.
Show resolved Hide resolved
}

// dependencies reads a go.mod file from an io.Reader and returns a map of dependency name to their specified versions.
func dependencies(reader io.Reader) (map[string]string, error) {
data, err := io.ReadAll(reader)
if err != nil {
return nil, fmt.Errorf("reading contents: %w", err)
}

gomod, err := modfile.Parse("go.mod", data, nil)
if err != nil {
return nil, fmt.Errorf("parsing contents: %w", err)
}

deps := make(map[string]string)
for _, req := range gomod.Require {
deps[req.Mod.Path] = req.Mod.Version
}

return deps, nil
}
6 changes: 6 additions & 0 deletions go.work
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
go 1.16

use (
.
./cmd/xk6-depsync
)
42 changes: 42 additions & 0 deletions go.work.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=