Skip to content
This repository has been archived by the owner on Jun 28, 2023. It is now read-only.

Go 1.16 support and some general improvements #350

Merged
merged 5 commits into from
Jul 21, 2021
Merged

Go 1.16 support and some general improvements #350

merged 5 commits into from
Jul 21, 2021

Conversation

daenney
Copy link
Contributor

@daenney daenney commented May 19, 2021

  • Move go-neb to be built by 1.16 and update the Dockerfile to use newer bases
    Just for general house keeping purposes and getting access to new features in go
  • Replace the deprecated golint with staticcheck and fix the linter warnings
    Golint has been deprecated and frozen
  • Fix the sqlite3 package version which also resolves a build issue on newer gcc's
    The v2 tags were erroneous, move us back to v1 tags
  • Migrate CI from Travis to Github Actions
    We've started to use GHA for more things on matrix-org, this moves neb in the same direction. The one thing we lose now is the Travis-CI webhook notifications, but given GHA will poke the author of the PR if their builds fail that feels like an acceptable trade-off. You can see what a build ends up looking like over here
  • Upgrade to the current release of mautrix-go
    We need this in order to be able to build with newer versions of Go

daenney added 4 commits May 18, 2021 14:42
golint has been deprecated and archived. There's no 1:1 match for it,
but staticcheck is a checker the golint README recommends and has tons
of useful checks.

This commit enables all checks, and then disables:
* ST1000: Incorrect or missing package comment
* ST1005: Incorrectly formatted error string, necessary due to the fact
  that all our error strings are capitalised, which they should not be
* SA1019: Locally in goneb.go because we have to rework how we use the
  prometheus package there

It also fixes a number of other errors surfaced by staticheck.
v2 was a set of erroneous tags, the latest is v1.14.7 which also fixes a
vet warning related to building sqlite3 on gcc10+.
@daenney
Copy link
Contributor Author

daenney commented May 19, 2021

Though all the tests pass, one test is logging a panic:

=== RUN   TestRespondToEcho
time="2021-05-19T08:55:31Z" level=info msg="Incoming request" req.id=smW335jjZJSf req.method=POST req.path=/admin/configureClient
time="2021-05-19T08:55:31Z" level=warning msg="Device ID is not set which will result in E2E encryption/decryption not working"
time="2021-05-19T08:55:31Z" level=error msg="Failed to load next_batch token" error="sql: no rows in result set" user_id="@link:hyrule"
time="2021-05-19T08:55:31Z" level=info msg="Created new client" auto_join_rooms=true device_id= since= sync=true user_id="@link:hyrule"
time="2021-05-19T08:55:31Z" level=info msg="Responding (372 bytes)" code=200 req.id=smW335jjZJSf req.method=POST req.path=/admin/configureClient
time="2021-05-19T08:55:31Z" level=info msg="Incoming request" req.id=Bk1hytf2pIFO req.method=POST req.path=/admin/configureService
time="2021-05-19T08:55:31Z" level=info msg="Incoming configure service request" req.id=Bk1hytf2pIFO req.method=POST req.path=/admin/configureService service_id=test_echo_service service_type=echo service_user_id="@link:hyrule"
time="2021-05-19T08:55:31Z" level=info msg="Responding (72 bytes)" code=200 req.id=Bk1hytf2pIFO req.method=POST req.path=/admin/configureService
time="2021-05-19T08:55:31Z" level=error msg="Fatal Sync() error" error="ProcessResponse panicked! since= panic=RoundTrip: Unhandled request: POST /_matrix/client/r0/keys/upload\nHandlers: 2\ngoroutine 12 [running]:\nruntime/debug.Stack(0xc000706e30, 0xba9080, 0xc0001aaa20)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/runtime/debug/stack.go:24 +0x9f\nmaunium.net/go/mautrix.(*DefaultSyncer).ProcessResponse.func1(0x0, 0x0, 0xc000707d18)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/sync.go:133 +0x76\npanic(0xba9080, 0xc0001aaa20)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/runtime/panic.go:965 +0x1b9\ngithub.com/matrix-org/go-neb.(*matrixTripper).RoundTrip(0xc0000aaa78, 0xc000112300, 0xc0000aaa78, 0x0, 0x0)\n\t/home/runner/work/go-neb/go-neb/testutil_test.go:43 +0x33f\nnet/http.send(0xc000112300, 0xd96080, 0xc0000aaa78, 0x0, 0x0, 0x0, 0xc0000102c8, 0x1e, 0x1, 0x0)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/net/http/client.go:251 +0x454\nnet/http.(*Client).send(0xc0001a9890, 0xc000112300, 0x0, 0x0, 0x0, 0xc0000102c8, 0x0, 0x1, 0x419ffb)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/net/http/client.go:175 +0xff\nnet/http.(*Client).do(0xc0001a9890, 0xc000112300, 0x0, 0x0, 0x0)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/net/http/client.go:717 +0x45f\nnet/http.(*Client).Do(...)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/net/http/client.go:585\nmaunium.net/go/mautrix.(*Client).executeCompiledRequest(0xc0004441a0, 0xc000112300, 0x0, 0xee6b2800, 0xb7c920, 0xc0000102b8, 0x0, 0x0, 0x0, 0x0, ...)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:393 +0xe7\nmaunium.net/go/mautrix.(*Client).MakeFullRequest(0xc0004441a0, 0xcb0dfd, 0x4, 0xc000784c00, 0x2f, 0x0, 0xb8d6e0, 0xc00045b6f0, 0x0, 0x0, ...)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:360 +0x1e5\nmaunium.net/go/mautrix.(*Client).MakeRequest(...)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:284\nmaunium.net/go/mautrix.(*Client).UploadKeys(0xc0004441a0, 0xc00045b6f0, 0x1a, 0xc00045b700, 0x1)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:1184 +0x258\nmaunium.net/go/mautrix/crypto.(*OlmMachine).ShareKeys(0xc00048c000, 0x0, 0x4e, 0xc00045a9d0)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/crypto/machine.go:421 +0x1a5\nmaunium.net/go/mautrix/crypto.(*OlmMachine).ProcessSyncResponse(0xc00048c000, 0xc00046c4d0, 0x0, 0x0, 0x767d18)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/crypto/machine.go:180 +0x305\ngithub.com/matrix-org/go-neb/clients.(*BotClient).syncCallback(0xc0001a5220, 0xc00046c4d0, 0x0, 0x0, 0xcf5f01)\n\t/home/runner/work/go-neb/go-neb/clients/bot_client.go:113 +0x75\nmaunium.net/go/mautrix.(*DefaultSyncer).ProcessResponse(0xc000296e60, 0xc00046c4d0, 0x0, 0x0, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/sync.go:138 +0x110\nmaunium.net/go/mautrix.(*Client).SyncWithContext(0xc0004441a0, 0xda3a50, 0xc0000a6010, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:245 +0x22d\nmaunium.net/go/mautrix.(*Client).Sync(...)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:201\ngithub.com/matrix-org/go-neb/clients.(*BotClient).Sync(0xc0001a5220)\n\t/home/runner/work/go-neb/go-neb/clients/bot_client.go:170 +0x185\ncreated by github.com/matrix-org/go-neb/clients.(*Clients).initClient\n\t/home/runner/work/go-neb/go-neb/clients/clients.go:417 +0xd65\n" user_id="@link:hyrule"

I don't really know why, it seems the handler that gets installed for keys/upload somehow goes missing during the test. Since the tests pass, I'm not actually sure what to make of it

@daenney
Copy link
Contributor Author

daenney commented May 19, 2021

@kegsay @neilalexander Could I bother you for a review on this?

Copy link

@neilalexander neilalexander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be better to use golangci-lint, which includes staticcheck. There's a sample config in the Dendrite repository, if we're looking to settle on a standard.

@daenney
Copy link
Contributor Author

daenney commented May 19, 2021

Might be better to use golangci-lint, which includes staticcheck. There's a sample config in the Dendrite repository, if we're looking to settle on a standard.

I was thinking about that, but due to the amount of extra linters that it has and the state of go-neb that becomes a lot of extra things to fix. I'm happy to do that, but I'd prefer to do it as a follow-up PR.

@Mic92
Copy link

Mic92 commented Jul 20, 2021

I realized you also incorporate my changes: https://github.com/matrix-org/go-neb/pull/351/files

@daenney
Copy link
Contributor Author

daenney commented Jul 20, 2021

@neilalexander @kegsay Could I get another review and/or sign-off please? I'd like to move this forward.

@daenney
Copy link
Contributor Author

daenney commented Jul 20, 2021

Though all the tests pass, one test is logging a panic:

=== RUN   TestRespondToEcho
time="2021-05-19T08:55:31Z" level=info msg="Incoming request" req.id=smW335jjZJSf req.method=POST req.path=/admin/configureClient
time="2021-05-19T08:55:31Z" level=warning msg="Device ID is not set which will result in E2E encryption/decryption not working"
time="2021-05-19T08:55:31Z" level=error msg="Failed to load next_batch token" error="sql: no rows in result set" user_id="@link:hyrule"
time="2021-05-19T08:55:31Z" level=info msg="Created new client" auto_join_rooms=true device_id= since= sync=true user_id="@link:hyrule"
time="2021-05-19T08:55:31Z" level=info msg="Responding (372 bytes)" code=200 req.id=smW335jjZJSf req.method=POST req.path=/admin/configureClient
time="2021-05-19T08:55:31Z" level=info msg="Incoming request" req.id=Bk1hytf2pIFO req.method=POST req.path=/admin/configureService
time="2021-05-19T08:55:31Z" level=info msg="Incoming configure service request" req.id=Bk1hytf2pIFO req.method=POST req.path=/admin/configureService service_id=test_echo_service service_type=echo service_user_id="@link:hyrule"
time="2021-05-19T08:55:31Z" level=info msg="Responding (72 bytes)" code=200 req.id=Bk1hytf2pIFO req.method=POST req.path=/admin/configureService
time="2021-05-19T08:55:31Z" level=error msg="Fatal Sync() error" error="ProcessResponse panicked! since= panic=RoundTrip: Unhandled request: POST /_matrix/client/r0/keys/upload\nHandlers: 2\ngoroutine 12 [running]:\nruntime/debug.Stack(0xc000706e30, 0xba9080, 0xc0001aaa20)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/runtime/debug/stack.go:24 +0x9f\nmaunium.net/go/mautrix.(*DefaultSyncer).ProcessResponse.func1(0x0, 0x0, 0xc000707d18)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/sync.go:133 +0x76\npanic(0xba9080, 0xc0001aaa20)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/runtime/panic.go:965 +0x1b9\ngithub.com/matrix-org/go-neb.(*matrixTripper).RoundTrip(0xc0000aaa78, 0xc000112300, 0xc0000aaa78, 0x0, 0x0)\n\t/home/runner/work/go-neb/go-neb/testutil_test.go:43 +0x33f\nnet/http.send(0xc000112300, 0xd96080, 0xc0000aaa78, 0x0, 0x0, 0x0, 0xc0000102c8, 0x1e, 0x1, 0x0)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/net/http/client.go:251 +0x454\nnet/http.(*Client).send(0xc0001a9890, 0xc000112300, 0x0, 0x0, 0x0, 0xc0000102c8, 0x0, 0x1, 0x419ffb)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/net/http/client.go:175 +0xff\nnet/http.(*Client).do(0xc0001a9890, 0xc000112300, 0x0, 0x0, 0x0)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/net/http/client.go:717 +0x45f\nnet/http.(*Client).Do(...)\n\t/opt/hostedtoolcache/go/1.16.4/x64/src/net/http/client.go:585\nmaunium.net/go/mautrix.(*Client).executeCompiledRequest(0xc0004441a0, 0xc000112300, 0x0, 0xee6b2800, 0xb7c920, 0xc0000102b8, 0x0, 0x0, 0x0, 0x0, ...)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:393 +0xe7\nmaunium.net/go/mautrix.(*Client).MakeFullRequest(0xc0004441a0, 0xcb0dfd, 0x4, 0xc000784c00, 0x2f, 0x0, 0xb8d6e0, 0xc00045b6f0, 0x0, 0x0, ...)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:360 +0x1e5\nmaunium.net/go/mautrix.(*Client).MakeRequest(...)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:284\nmaunium.net/go/mautrix.(*Client).UploadKeys(0xc0004441a0, 0xc00045b6f0, 0x1a, 0xc00045b700, 0x1)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:1184 +0x258\nmaunium.net/go/mautrix/crypto.(*OlmMachine).ShareKeys(0xc00048c000, 0x0, 0x4e, 0xc00045a9d0)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/crypto/machine.go:421 +0x1a5\nmaunium.net/go/mautrix/crypto.(*OlmMachine).ProcessSyncResponse(0xc00048c000, 0xc00046c4d0, 0x0, 0x0, 0x767d18)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/crypto/machine.go:180 +0x305\ngithub.com/matrix-org/go-neb/clients.(*BotClient).syncCallback(0xc0001a5220, 0xc00046c4d0, 0x0, 0x0, 0xcf5f01)\n\t/home/runner/work/go-neb/go-neb/clients/bot_client.go:113 +0x75\nmaunium.net/go/mautrix.(*DefaultSyncer).ProcessResponse(0xc000296e60, 0xc00046c4d0, 0x0, 0x0, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/sync.go:138 +0x110\nmaunium.net/go/mautrix.(*Client).SyncWithContext(0xc0004441a0, 0xda3a50, 0xc0000a6010, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:245 +0x22d\nmaunium.net/go/mautrix.(*Client).Sync(...)\n\t/home/runner/go/pkg/mod/maunium.net/go/mautrix@v0.9.12/client.go:201\ngithub.com/matrix-org/go-neb/clients.(*BotClient).Sync(0xc0001a5220)\n\t/home/runner/work/go-neb/go-neb/clients/bot_client.go:170 +0x185\ncreated by github.com/matrix-org/go-neb/clients.(*Clients).initClient\n\t/home/runner/work/go-neb/go-neb/clients/clients.go:417 +0xd65\n" user_id="@link:hyrule"

I don't really know why, it seems the handler that gets installed for keys/upload somehow goes missing during the test. Since the tests pass, I'm not actually sure what to make of it

@kegsay Do you have any thoughts on this? I'm still kinda puzzled by the test. It feels wrong that it's logging a panic there.

@kegsay
Copy link
Member

kegsay commented Jul 21, 2021

Let's leave it for now, but if you can file an issue for it that'd be great.

@daenney daenney merged commit 2592801 into matrix-org:master Jul 21, 2021
@daenney daenney deleted the daenney/go-116 branch July 21, 2021 09:49
daenney added a commit that referenced this pull request Jul 21, 2021
Forgot to do this as part of #350, which flipped CI to GitHub Actions.
daenney added a commit that referenced this pull request Jul 21, 2021
I noticed that after the rebase for #351 CI didn't run, whereas it should (since we're upgrading packages). This happens because as part of #350 we forgot to include go.mod|sum files as triggers.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants