Skip to content

Conversation

@sarutak
Copy link
Member

@sarutak sarutak commented Aug 18, 2021

What changes were proposed in this pull request?

This PR upgrades Caffeine to 2.9.2.
Caffeine was introduced in SPARK-34309 (#31517). At the time that PR was opened, the latest version of caffeine was 2.9.1 but now 2.9.2 is available.

Why are the changes needed?

2.9.2 have the following improvements (https://github.com/ben-manes/caffeine/releases/tag/v2.9.2).

  • Fixed reading an intermittent null weak/soft value during a concurrent write
  • Fixed extraneous eviction when concurrently removing a collected entry after a writer resurrects it with a new mapping
  • Fixed excessive retries of discarding an expired entry when the fixed duration period is extended, thereby resurrecting it

Does this PR introduce any user-facing change?

No.

How was this patch tested?

CIs.

@github-actions github-actions bot added the BUILD label Aug 18, 2021
@sarutak
Copy link
Member Author

sarutak commented Aug 18, 2021

Considering the discussion, it's a little bit possible that that PR is reverted but it's better to upgrade it if we retain that change.

Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good w/ this.

@SparkQA
Copy link

SparkQA commented Aug 18, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47074/

@SparkQA
Copy link

SparkQA commented Aug 18, 2021

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47074/

@SparkQA
Copy link

SparkQA commented Aug 18, 2021

Test build #142573 has finished for PR 33772 at commit a6c53b1.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@sarutak
Copy link
Member Author

sarutak commented Aug 18, 2021

Thanks. Merging to master.

@sarutak sarutak closed this in 281b00a Aug 18, 2021
@dongjoon-hyun
Copy link
Member

+1, LGTM, too.

HyukjinKwon pushed a commit that referenced this pull request Aug 22, 2021
…Guava Cache"

### What changes were proposed in this pull request?
This pr revert the change of SPARK-34309, includes:

- #31517
- #33772

### Why are the changes needed?

1. No really performance improvement in Spark
2. Added an additional dependency

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass the Jenkins or GitHub Action

Closes #33784 from LuciferYang/revert-caffeine.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants