Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: cache issues #292

Merged
merged 6 commits into from
Mar 20, 2024
Merged

Fix: cache issues #292

merged 6 commits into from
Mar 20, 2024

Conversation

saikonen
Copy link
Collaborator

  • Adds a check that tmpdir exists when launching workers, and creates the dir if it is missing. This fixes workers not being able to launch if for any reason the cache directory gets wiped.
  • Adds log cache process to the cache healthcheck monitoring, so it will correctly be restarted in case of issues.

@saikonen saikonen added the ready-for-review Ready for thorough review. label Jan 27, 2022
@saikonen saikonen added ready-for-review Ready for thorough review. and removed ready-for-review Ready for thorough review. labels Jan 31, 2022
@ruial
Copy link

ruial commented Jul 19, 2022

This fix is very much needed, the LogCacheStore tends to die after a while and the service requires a manual restart.

Could someone please review? @oavdeev @obgibson @jackie-ob

@@ -77,7 +77,7 @@ async def start_caches(self, app):

async def _monitor_restart_requests(self):
while True:
for _cache in [self.artifact_cache, self.dag_cache]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should merge this change.

@saikonen saikonen merged commit 0d4ac65 into master Mar 20, 2024
6 checks passed
@saikonen saikonen deleted the fix/cache-issues branch March 20, 2024 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-review Ready for thorough review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants