Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates minio-go to v7.0.37 from v7.0.32 #5702

Merged
merged 2 commits into from
Sep 19, 2022

Conversation

davinci26
Copy link
Contributor

@davinci26 davinci26 commented Sep 19, 2022

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

fixes #5701

Changelog: minio/minio-go@v7.0.32...v7.0.37

Signed-off-by: Sotiris Nanopoulos sotiris.nanopoulos@reddit.com

Verification

Unit/integration tests

fixes thanos-io#5701

Changelog: minio/minio-go@v7.0.32...v7.0.37

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Copy link
Contributor

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @davinci26 For the minio issue you mentioned, how did you find it in Thanos? Are you just seeing goroutine leaks on the store gateway?

@SuperQ
Copy link
Contributor

SuperQ commented Sep 19, 2022

We're seeing panic: send on closed channel, which causes our Thanos Stores to crash.

@davinci26
Copy link
Contributor Author

It surfaces as:

panic: send on closed channel

goroutine 10097536 [running]:
github.com/minio/minio-go/v7.(*Object).doGetRequest(_, {{0x0, 0x0, 0x0}, 0x0, 0x0, 0x0, 0x0, 0x1, 0x1, ...})
  /go/pkg/mod/github.com/minio/minio-go/v7@v7.0.32-0.20220706200439-ef3e45ed9cdb/api-get-object.go:319 +0xdc
github.com/minio/minio-go/v7.(*Object).Read(0xc0b91441c0, {0x0?, 0x0?, 0x0?})
  /go/pkg/mod/github.com/minio/minio-go/v7@v7.0.32-0.20220706200439-ef3e45ed9cdb/api-get-object.go:399 +0x248
github.com/thanos-io/objstore/providers/s3.(*Bucket).getRange(0xc0004cbf90, {0x2586188, 0xc3d0ed7880}, {0xc015a4c870, 0x28}, 0x65900, 0x32c80)
  /go/pkg/mod/github.com/thanos-io/objstore@v0.0.0-20220715165016-ce338803bc1e/providers/s3/s3.go:428 +0x20e
github.com/thanos-io/objstore/providers/s3.(*Bucket).GetRange(0x7f97aa2f30d8?, {0x2586188?, 0xc3d0ed7880?}, {0xc015a4c870?, 0x0?}, 0x10000c00009cf00?, 0xc0b3934c68?)
  /go/pkg/mod/github.com/thanos-io/objstore@v0.0.0-20220715165016-ce338803bc1e/providers/s3/s3.go:445 +0x28
github.com/thanos-io/objstore.(*metricBucket).GetRange(0xc000b54540, {0x2586188, 0xc3d0ed7880}, {0xc015a4c870, 0x28}, 0x0?, 0x0?)
  /go/pkg/mod/github.com/thanos-io/objstore@v0.0.0-20220715165016-ce338803bc1e/objstore.go:526 +0xbd
github.com/thanos-io/objstore.TracingBucket.GetRange({{0x2596cf0?, 0xc000b54540?}}, {0x2586188?, 0xc3d0ed7880?}, {0xc015a4c870, 0x28}, 0x0?, 0x0?)
  /go/pkg/mod/github.com/thanos-io/objstore@v0.0.0-20220715165016-ce338803bc1e/tracing.go:50 +0x22c
github.com/thanos-io/thanos/pkg/store/cache.(*CachingBucket).fetchMissingSubranges.func1()
  /app/pkg/store/cache/caching_bucket.go:424 +0x103
golang.org/x/sync/errgroup.(*Group).Go.func1()
  /go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:75 +0x64
created by golang.org/x/sync/errgroup.(*Group).Go
  /go/pkg/mod/golang.org/x/sync@v0.0.0-20220722155255-886fb9371eb4/errgroup/errgroup.go:72 +0xa5

Which I pattern match with minio/mc#4202 but I also see goroutines spike to 2k as well.

image

Copy link
Member

@squat squat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find

@yeya24 yeya24 merged commit d1405e4 into thanos-io:main Sep 19, 2022
@yeya24
Copy link
Contributor

yeya24 commented Sep 29, 2022

It is probably a little bit late. Do we still want to have a v0.28.1 release for this fix?

@diranged
Copy link

diranged commented Oct 3, 2022

We're seeing this fairly regularly... can we get a v0.28.1 release with it?

yeya24 pushed a commit that referenced this pull request Oct 4, 2022
* Updates minio-go to v7.0.37 from v7.0.32

fixes #5701

Changelog: minio/minio-go@v7.0.32...v7.0.37

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

* Adds changelog entry

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Co-authored-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
yeya24 pushed a commit that referenced this pull request Oct 4, 2022
* Updates minio-go to v7.0.37 from v7.0.32

fixes #5701

Changelog: minio/minio-go@v7.0.32...v7.0.37

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

* Adds changelog entry

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
yeya24 pushed a commit that referenced this pull request Oct 6, 2022
* Updates minio-go to v7.0.37 from v7.0.32

fixes #5701

Changelog: minio/minio-go@v7.0.32...v7.0.37

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

* Adds changelog entry

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>
yeya24 added a commit that referenced this pull request Oct 6, 2022
* Updates minio-go to v7.0.37 from v7.0.32 (#5702)

* Updates minio-go to v7.0.37 from v7.0.32

fixes #5701

Changelog: minio/minio-go@v7.0.32...v7.0.37

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

* Adds changelog entry

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>

* update changelog and version

Signed-off-by: Ben Ye <benye@amazon.com>

add line

Signed-off-by: Ben Ye <benye@amazon.com>

* update version to 0.28.1

Signed-off-by: Ben Ye <benye@amazon.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Co-authored-by: Sotiris Nanopoulos <sonanopo@microsoft.com>
@yeya24
Copy link
Contributor

yeya24 commented Oct 6, 2022

@diranged New release is out. Let me know if it works or not.

squat pushed a commit that referenced this pull request Oct 7, 2022
* Cut 0.28.0-rc.0 (#5632)

* cut 0.28.0-rc.0

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* address review comments

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* mention experimental/hidden features

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* include #5607 in changelog

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* cut v0.28.0 (#5647)

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* Cut 0.28.1 (#5757)

* Updates minio-go to v7.0.37 from v7.0.32 (#5702)

* Updates minio-go to v7.0.37 from v7.0.32

fixes #5701

Changelog: minio/minio-go@v7.0.32...v7.0.37

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

* Adds changelog entry

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>

* update changelog and version

Signed-off-by: Ben Ye <benye@amazon.com>

add line

Signed-off-by: Ben Ye <benye@amazon.com>

* update version to 0.28.1

Signed-off-by: Ben Ye <benye@amazon.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Co-authored-by: Sotiris Nanopoulos <sonanopo@microsoft.com>

Signed-off-by: Ben Ye <ben.ye@bytedance.com>
Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Co-authored-by: Ben Ye <ben.ye@bytedance.com>
Co-authored-by: Sotiris Nanopoulos <sonanopo@microsoft.com>
utukJ pushed a commit to utukJ/thanos that referenced this pull request Oct 13, 2022
* Cut 0.28.0-rc.0 (thanos-io#5632)

* cut 0.28.0-rc.0

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* address review comments

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* mention experimental/hidden features

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* include thanos-io#5607 in changelog

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* cut v0.28.0 (thanos-io#5647)

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

Signed-off-by: Ben Ye <ben.ye@bytedance.com>

* Cut 0.28.1 (thanos-io#5757)

* Updates minio-go to v7.0.37 from v7.0.32 (thanos-io#5702)

* Updates minio-go to v7.0.37 from v7.0.32

fixes thanos-io#5701

Changelog: minio/minio-go@v7.0.32...v7.0.37

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

* Adds changelog entry

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>

* update changelog and version

Signed-off-by: Ben Ye <benye@amazon.com>

add line

Signed-off-by: Ben Ye <benye@amazon.com>

* update version to 0.28.1

Signed-off-by: Ben Ye <benye@amazon.com>

Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Co-authored-by: Sotiris Nanopoulos <sonanopo@microsoft.com>

Signed-off-by: Ben Ye <ben.ye@bytedance.com>
Signed-off-by: Sotiris Nanopoulos <sotiris.nanopoulos@reddit.com>
Signed-off-by: Ben Ye <benye@amazon.com>
Co-authored-by: Ben Ye <ben.ye@bytedance.com>
Co-authored-by: Sotiris Nanopoulos <sonanopo@microsoft.com>
Signed-off-by: utukj <utukphd@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bump minio-go v0.7.0.32 to latest
5 participants