Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow searchable snapshot cache service to periodically fsync cache files #64696

Merged
merged 30 commits into from
Nov 23, 2020
Merged

Allow searchable snapshot cache service to periodically fsync cache files #64696

Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
012eee2
Periodically fsync searchable snapshots cache files
tlrx Nov 5, 2020
499f12b
Merge branch 'master' into periodic-fsync
tlrx Nov 9, 2020
31299c1
Fix spotless
tlrx Nov 9, 2020
bf0e660
Merge branch 'master' into periodic-fsync
tlrx Nov 12, 2020
9085f81
Merge branch 'master' into periodic-fsync
tlrx Nov 13, 2020
fb10c6f
use registering mechanism to sync cache files
tlrx Nov 16, 2020
f5d9559
Merge branch 'master' into periodic-fsync
tlrx Nov 16, 2020
684e01e
Use a queue
tlrx Nov 16, 2020
6b238d6
revert changes in Cache
tlrx Nov 17, 2020
eabe562
do not remove from queue on eviction
tlrx Nov 17, 2020
af47aaa
noop fsyncListener
tlrx Nov 17, 2020
437c8ea
limit iterations
tlrx Nov 17, 2020
c20bf75
assert IOE
tlrx Nov 17, 2020
365812d
min 1s
tlrx Nov 17, 2020
8fe920b
scaledRandomIntBetween(1, 120)
tlrx Nov 17, 2020
74f728f
Use runnable
tlrx Nov 17, 2020
5091a5d
mutualize FSyncTrackingFileSystemProvider
tlrx Nov 17, 2020
f69dde9
check evictions
tlrx Nov 17, 2020
395845d
use atomic long to count cache files
tlrx Nov 17, 2020
082b76d
[] + synchronized assert
tlrx Nov 18, 2020
ea34570
private
tlrx Nov 20, 2020
54d051d
provider.tearDown()
tlrx Nov 20, 2020
e8ad9be
randomPopulateAndReads
tlrx Nov 20, 2020
6066924
set interval before start
tlrx Nov 20, 2020
e9721c4
deleteIfExists
tlrx Nov 20, 2020
bd1ce13
lock & waitfor termination
tlrx Nov 20, 2020
46adb3c
Merge branch 'master' into periodic-fsync
tlrx Nov 20, 2020
04a4d77
missing close
tlrx Nov 20, 2020
db24043
Merge branch 'master' into periodic-fsync
tlrx Nov 23, 2020
10216ea
nits
tlrx Nov 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions ...apshots/src/main/java/org/elasticsearch/xpack/searchablesnapshots/cache/CacheService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ boolean isCacheFileToSync(CacheFile cacheFile) {
* non empty set of completed ranges this method also fsync the shard's snapshot cache directory, which is the parent directory of the
* cache entry. Note that cache files might be evicted during the synchronization.
*/
protected void synchronizeCache() {
protected synchronized void synchronizeCache() {
henningandersen marked this conversation as resolved.
Show resolved Hide resolved
long count = 0L;
final Set<Path> cacheDirs = new HashSet<>();
final long startTimeNanos = threadPool.relativeTimeInNanos();
Expand All @@ -262,10 +262,8 @@ protected void synchronizeCache() {
break;
}
final CacheFile cacheFile = cacheFilesToSync.poll();
if (cacheFile == null) {
logger.debug("stopping cache synchronization (no more cache files to fsync)");
break;
}
assert cacheFile != null;

final long value = numberOfCacheFilesToSync.decrementAndGet();
assert value >= 0 : value;
final Path cacheFilePath = cacheFile.getFile();
Expand Down Expand Up @@ -294,7 +292,7 @@ protected void synchronizeCache() {
if (logger.isDebugEnabled()) {
final long elapsedNanos = threadPool.relativeTimeInNanos() - startTimeNanos;
logger.debug(
"cache files synchronization is done ({} cache files synchronized in {})",
"cache files synchronization is done ([{}] cache files synchronized in [{}])",
count,
TimeValue.timeValueNanos(elapsedNanos)
);
Expand Down