Skip to content

Commit

Permalink
store/proxy: properly check if context has ended
Browse files Browse the repository at this point in the history
How the code was before it could happen that we might receive some
series from the stream however by the time we'd send them back to the
reader, it would not read it anymore since the deadline would have been
exceeded.

Properly use a `select` here to get out of the goroutine if the deadline
has been exceeded.

Might potentially fix a problem where we see one goroutine hanging
constantly (and thus blocking from work being done):

```
goroutine profile: total 126
25 @ 0x42f62f 0x40502b 0x405001 0x404de5 0xe7435b 0x45cc41
	0xe7435a	github.com/improbable-eng/thanos/pkg/store.startStreamSeriesSet.func1+0x18a	/go/src/github.com/improbable-eng/thanos/pkg/store/proxy.go:318
```
  • Loading branch information
Giedrius Statkevičius committed Apr 25, 2019
1 parent f58d417 commit 853df6d
Showing 1 changed file with 8 additions and 5 deletions.
13 changes: 8 additions & 5 deletions pkg/store/proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -294,10 +294,6 @@ func startStreamSeriesSet(
return
}

if ctx.Err() != nil {
return
}

if err != nil {
wrapErr := errors.Wrapf(err, "receive series from %s", s.name)
if partialResponse {
Expand All @@ -315,7 +311,14 @@ func startStreamSeriesSet(
s.warnCh.send(storepb.NewWarnSeriesResponse(errors.New(w)))
continue
}
s.recvCh <- r.GetSeries()

select {
case s.recvCh <- r.GetSeries():
continue
case <-ctx.Done():
return
}

}
}()
return s
Expand Down

0 comments on commit 853df6d

Please sign in to comment.