Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix race condition in Cache.Clear #133

Merged
merged 2 commits into from
Mar 16, 2020
Merged

Fix race condition in Cache.Clear #133

merged 2 commits into from
Mar 16, 2020

Conversation

martinmr
Copy link
Contributor

@martinmr martinmr commented Feb 5, 2020

There are race conditions when Set or Del are called at the same time
as Clear. Instead of creating a new channel, Clear now processes all the
entries from the channel.

Fixes #132


This change is Reviewable

There are race conditions when Set or Del are called at the same time
than Clear or Close since the latter functions modify setBuf.
Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @jarifibrahim, @karlmcguire, and @martinmr)


cache.go, line 299 at r1 (raw file):

	// Swap out the setBuf channel.
	c.setBufLock.Lock()
	c.setBuf = make(chan *item, setBufSize)

Instead of make chan. I could do a for loop and empty the chan out.

Let's not use locks.

Copy link
Contributor Author

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on @jarifibrahim, @karlmcguire, and @manishrjain)


cache.go, line 299 at r1 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

Instead of make chan. I could do a for loop and empty the chan out.

Let's not use locks.

Done.

@martinmr martinmr changed the title Protect setBuf with a RWLock. Fix race condition in Cache.Clear Feb 8, 2020
Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge this, if this works? :lgtm:

Reviewed 1 of 1 files at r2.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @jarifibrahim and @karlmcguire)

Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @jarifibrahim and @karlmcguire)

@martinmr martinmr merged commit 62cb731 into master Mar 16, 2020
@martinmr martinmr deleted the martinmr/setbuf-lock branch March 16, 2020 19:49
@erikdubbelboer erikdubbelboer mentioned this pull request Jun 15, 2020
jarifibrahim pushed a commit to dgraph-io/badger that referenced this pull request Jun 30, 2020
The update brings following changes from ristretto into badger

```
f66de99 Improve handling of updated items (dgraph-io/ristretto#168)
aec7994 Fix flaky TestCacheDel test (dgraph-io/ristretto#169)
a093fe6 Add benchmark plots to the project repo (dgraph-io/ristretto#166)
49dc42c Add Anurag as codeowner (dgraph-io/ristretto#158)
7a3f2d3 z: use MemHashString and xxhash.Sum64String (dgraph-io/ristretto#153)
9c31bb2 Check conflict key before updating expiration map. (dgraph-io/ristretto#154)
62cb731 Fix race condition in Cache.Clear (dgraph-io/ristretto#133)
9af1934 Docs and whitespace changes for readability. (dgraph-io/ristretto#143)
dbc185e Add changelog. (dgraph-io/ristretto#142)
ff325ad Remove key from policy after TTL eviction (dgraph-io/ristretto#130)
2dd5ff5 Use the require library in all the tests. (dgraph-io/ristretto#128)
7c48141 Use require in all tests in cache_test.go (dgraph-io/ristretto#127)
51e97ad Sets with TTL (dgraph-io/ristretto#122)
d3e7c37 Add benchmarks for math.rand and fastrand (dgraph-io/ristretto#118)
593823e Integrate fixes from PR dgraph-io/ristretto#91. (dgraph-io/ristretto#126)
29b4dd7 Fix comments. (dgraph-io/ristretto#123)
ddf345c Removed workflows directory. (dgraph-io/ristretto#124)
eb104d0 Add martinmr to CODEOWNERS file. (dgraph-io/ristretto#125)
```
jarifibrahim pushed a commit to dgraph-io/badger that referenced this pull request Jul 6, 2020
The update brings following changes from ristretto into badger

```
f66de99 Improve handling of updated items (dgraph-io/ristretto#168)
aec7994 Fix flaky TestCacheDel test (dgraph-io/ristretto#169)
a093fe6 Add benchmark plots to the project repo (dgraph-io/ristretto#166)
49dc42c Add Anurag as codeowner (dgraph-io/ristretto#158)
7a3f2d3 z: use MemHashString and xxhash.Sum64String (dgraph-io/ristretto#153)
9c31bb2 Check conflict key before updating expiration map. (dgraph-io/ristretto#154)
62cb731 Fix race condition in Cache.Clear (dgraph-io/ristretto#133)
9af1934 Docs and whitespace changes for readability. (dgraph-io/ristretto#143)
dbc185e Add changelog. (dgraph-io/ristretto#142)
ff325ad Remove key from policy after TTL eviction (dgraph-io/ristretto#130)
2dd5ff5 Use the require library in all the tests. (dgraph-io/ristretto#128)
7c48141 Use require in all tests in cache_test.go (dgraph-io/ristretto#127)
51e97ad Sets with TTL (dgraph-io/ristretto#122)
d3e7c37 Add benchmarks for math.rand and fastrand (dgraph-io/ristretto#118)
593823e Integrate fixes from PR dgraph-io/ristretto#91. (dgraph-io/ristretto#126)
29b4dd7 Fix comments. (dgraph-io/ristretto#123)
ddf345c Removed workflows directory. (dgraph-io/ristretto#124)
eb104d0 Add martinmr to CODEOWNERS file. (dgraph-io/ristretto#125)
```

(cherry picked from commit 09dfa66)
jarifibrahim pushed a commit to dgraph-io/badger that referenced this pull request Oct 2, 2020
The update brings following changes from ristretto into badger

```
f66de99 Improve handling of updated items (dgraph-io/ristretto#168)
aec7994 Fix flaky TestCacheDel test (dgraph-io/ristretto#169)
a093fe6 Add benchmark plots to the project repo (dgraph-io/ristretto#166)
49dc42c Add Anurag as codeowner (dgraph-io/ristretto#158)
7a3f2d3 z: use MemHashString and xxhash.Sum64String (dgraph-io/ristretto#153)
9c31bb2 Check conflict key before updating expiration map. (dgraph-io/ristretto#154)
62cb731 Fix race condition in Cache.Clear (dgraph-io/ristretto#133)
9af1934 Docs and whitespace changes for readability. (dgraph-io/ristretto#143)
dbc185e Add changelog. (dgraph-io/ristretto#142)
ff325ad Remove key from policy after TTL eviction (dgraph-io/ristretto#130)
2dd5ff5 Use the require library in all the tests. (dgraph-io/ristretto#128)
7c48141 Use require in all tests in cache_test.go (dgraph-io/ristretto#127)
51e97ad Sets with TTL (dgraph-io/ristretto#122)
d3e7c37 Add benchmarks for math.rand and fastrand (dgraph-io/ristretto#118)
593823e Integrate fixes from PR dgraph-io/ristretto#91. (dgraph-io/ristretto#126)
29b4dd7 Fix comments. (dgraph-io/ristretto#123)
ddf345c Removed workflows directory. (dgraph-io/ristretto#124)
eb104d0 Add martinmr to CODEOWNERS file. (dgraph-io/ristretto#125)
```
manishrjain pushed a commit to outcaste-io/outserv that referenced this pull request Jul 6, 2022
The update brings following changes from ristretto into badger

```
f66de99 Improve handling of updated items (dgraph-io/ristretto#168)
aec7994 Fix flaky TestCacheDel test (dgraph-io/ristretto#169)
a093fe6 Add benchmark plots to the project repo (dgraph-io/ristretto#166)
49dc42c Add Anurag as codeowner (dgraph-io/ristretto#158)
7a3f2d3 z: use MemHashString and xxhash.Sum64String (dgraph-io/ristretto#153)
9c31bb2 Check conflict key before updating expiration map. (dgraph-io/ristretto#154)
62cb731 Fix race condition in Cache.Clear (dgraph-io/ristretto#133)
9af1934 Docs and whitespace changes for readability. (dgraph-io/ristretto#143)
dbc185e Add changelog. (dgraph-io/ristretto#142)
ff325ad Remove key from policy after TTL eviction (dgraph-io/ristretto#130)
2dd5ff5 Use the require library in all the tests. (dgraph-io/ristretto#128)
7c48141 Use require in all tests in cache_test.go (dgraph-io/ristretto#127)
51e97ad Sets with TTL (dgraph-io/ristretto#122)
d3e7c37 Add benchmarks for math.rand and fastrand (dgraph-io/ristretto#118)
593823e Integrate fixes from PR dgraph-io/ristretto#91. (dgraph-io/ristretto#126)
29b4dd7 Fix comments. (dgraph-io/ristretto#123)
ddf345c Removed workflows directory. (dgraph-io/ristretto#124)
eb104d0 Add martinmr to CODEOWNERS file. (dgraph-io/ristretto#125)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Race condition in setBuf
2 participants