Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ui] Thumbnail cache #1861

Merged
merged 29 commits into from
Feb 9, 2023
Merged
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
ff6358e
[ui] new Thumbnail disk cache
mugulmd Jan 5, 2023
1819eac
[ui] override default thumbnail directory with environment variable
mugulmd Jan 5, 2023
7566544
[ui] remove old thumbnails from disk cache when launching app
mugulmd Jan 6, 2023
bfdcfb5
[ui] ThumbnailCache: use logging instead of print
mugulmd Jan 6, 2023
b372946
[ui] ThumbnailCache: check cache dir existence before cleaning
mugulmd Jan 6, 2023
a96d886
[ui] override default time limit for thumbnails on disk with env var
mugulmd Jan 6, 2023
220bcfb
[ui] maximum number of thumbnails on disk
mugulmd Jan 6, 2023
60c8e77
[ui] create thumbnails asynchronously
mugulmd Jan 9, 2023
af2c9c0
[ui] dispatch thumbnail creation signal from grid
mugulmd Jan 9, 2023
88c85e9
[ui] thumbnail busy indicator when loading
mugulmd Jan 10, 2023
cd95589
[ui] ImageGallery: default value for grid view cache buffer
mugulmd Jan 10, 2023
268f608
[ui] ThumbnailCache: use QStandardPath for default cache dir
mugulmd Jan 16, 2023
dfb120c
[ui] ThumbnailCache: limit number of worker threads to 3
mugulmd Jan 16, 2023
a2e6a68
[ui] ImageDelegate: check URL is not empty instead of OS call
mugulmd Jan 16, 2023
7abbf1b
[ui] ThumbnailCache: reduce number of system calls
mugulmd Jan 16, 2023
8551135
[ui] ThumbnailCache: catch FileNotFoundError when removing thumbnails
mugulmd Jan 16, 2023
80174e2
[ui] ImageGallery: faster thumbnail dispatch using caller ID
mugulmd Jan 16, 2023
7e371ed
[ui] ThumbnailCache: resolve default cache dir in initialize method
mugulmd Jan 16, 2023
0926b20
[ui] fallback for dispatching thumbnails in case cellID changed
mugulmd Jan 17, 2023
374b9ab
[ui] ThumbnailCache: replace thread pool with custom thread spawning
mugulmd Jan 17, 2023
3f96778
[ui] ThumbnailCache: check if thumbnail has been created by another t…
mugulmd Jan 17, 2023
ae3184d
[ui] send thumbnail request with timer to avoid deadlocks
mugulmd Jan 17, 2023
dd235dc
[ui] clear thumbnail requests when changing viewpoints
mugulmd Jan 17, 2023
2f72f9e
[ui] ImageDelegate: increase thumbnail timer to 5s
mugulmd Jan 23, 2023
c482331
[ui] ThumbnailCache: asynchronous cache cleaning
mugulmd Jan 23, 2023
9b0be36
[ui] improve thumbnails quality
mugulmd Jan 26, 2023
db8b387
[ui] ImageGallery: do not reduce thumbnail resolution
fabiencastan Feb 8, 2023
db8d00e
[ui] ImageGallery thumbnails: no need for caching and expensive smoot…
fabiencastan Feb 8, 2023
c758bf0
[ui] thumbnail cache: use thread pool to manage thread lifetime autom…
mugulmd Feb 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions meshroom/ui/components/thumbnail.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import os
from pathlib import Path
import stat
import hashlib
import time
import logging
Expand Down Expand Up @@ -173,22 +174,28 @@ def clean():
# Scan thumbnail directory and gather all thumbnails to remove
toRemove = []
remaining = []
for entry in os.scandir(ThumbnailCache.thumbnailDir):
mugulmd marked this conversation as resolved.
Show resolved Hide resolved
if not entry.is_file():
for f_name in os.listdir(ThumbnailCache.thumbnailDir):
pathname = os.path.join(ThumbnailCache.thumbnailDir, f_name)

# System call to get current item info
f_stat = os.stat(pathname, follow_symlinks=False)

# Check if this is a regular file
if not stat.S_ISREG(f_stat.st_mode):
continue

# Compute storage duration since last usage of thumbnail
lastUsage = os.path.getmtime(entry.path)
lastUsage = f_stat.st_mtime
storageTime = now - lastUsage
logging.debug(f'[ThumbnailCache] Thumbnail {entry.name} has been stored for {storageTime}s')
logging.debug(f'[ThumbnailCache] Thumbnail {f_name} has been stored for {storageTime}s')

if storageTime > ThumbnailCache.storageTimeLimit * 3600 * 24:
# Mark as removable if storage time exceeds limit
logging.debug(f'[ThumbnailCache] {entry.name} exceeded storage time limit')
toRemove.append(entry.path)
logging.debug(f'[ThumbnailCache] {f_name} exceeded storage time limit')
toRemove.append(pathname)
else:
# Store path and last usage time for potentially sorting and removing later
remaining.append((entry.path, lastUsage))
remaining.append((pathname, lastUsage))

# Remove all thumbnails marked as removable
for path in toRemove:
Expand Down