Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix SearchableSnapshotsPersistentCacheIntegTests.testCacheSurviveRestart #66354

Merged

Conversation

tlrx
Copy link
Member

@tlrx tlrx commented Dec 15, 2020

The test SearchableSnapshotsPersistentCacheIntegTests.testCacheSurviveRestart fails on CI when executing this last verification:

assertThat(persistentCache.getNumDocs(), equalTo(0L));

This assert verifies that the persistent cache does not contain any cache file documents after the index is deleted. At the time this verification is done, it is possible that the CacheService hasn't processed yet the shard deletion notification (in markShardAsEvictedInCache) and hasn't evicted yet the CacheFiles associated to the deleted shard (these evictions are executed using an AbstractRunnable in the generic thread pool).

This pull request sets a large cache synchronization interval setting so that no synchronization are likely to be executed in the middle of the test. It also busy-asserts that the persistent cache has deletions (ie, cache files evictions were processed) before synchronization the cache again and checks that there are no more documents in the persistent cache.

Closes #66278

@tlrx tlrx added >test Issues or PRs that are addressing/adding tests :Distributed/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs v8.0.0 v7.11.0 labels Dec 15, 2020
@elasticmachine elasticmachine added the Team:Distributed Meta label for distributed team label Dec 15, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

Copy link
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@tlrx tlrx merged commit 0a1cec2 into elastic:master Dec 17, 2020
@tlrx tlrx deleted the fix-SearchableSnapshotsPersistentCacheIntegTests branch December 17, 2020 11:04
tlrx added a commit to tlrx/elasticsearch that referenced this pull request Dec 17, 2020
tlrx added a commit to tlrx/elasticsearch that referenced this pull request Dec 17, 2020
@tlrx
Copy link
Member Author

tlrx commented Dec 17, 2020

Thanks Henning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed Meta label for distributed team >test Issues or PRs that are addressing/adding tests v7.11.1 v7.12.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SearchableSnapshotsPersistentCacheIntegTests. testCacheSurviveRestart failure
4 participants