Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash diff server when its process pool dies #49

Merged
merged 3 commits into from
Jan 1, 2021

Conversation

Mr0grog
Copy link
Member

@Mr0grog Mr0grog commented Dec 14, 2020

If the server's process pool (used for actually running the diffs) fails repeatedly, assume it just can't be started and crash the whole server so a process manager can clean things up and totally start over. You can retain the old behavior (where the server keeps trying to start a new process pool if the old one is broken) by setting the RESTART_BROKEN_DIFFER environment variable to true.

Fixes #42.

If the server's process pool (used for actually running the diffs) fails repeatedly, assume it just can't be started and crash the whole server so a process manager can clean things up and totally start over. You can retain the old behavior (where the server keeps trying to start a new process pool if the old one is broken) by setting the `RESTART_BROKEN_DIFFER` environment variable to `true`.
@Mr0grog Mr0grog force-pushed the 42-crash-in-a-blaze-of-glory-when-process-pool-dies branch from 27e83e8 to 8e79ece Compare December 14, 2020 23:09
@Mr0grog Mr0grog merged commit 600e9e0 into main Jan 1, 2021
@Mr0grog Mr0grog deleted the 42-crash-in-a-blaze-of-glory-when-process-pool-dies branch January 1, 2021 22:15
@Mr0grog Mr0grog restored the 42-crash-in-a-blaze-of-glory-when-process-pool-dies branch January 1, 2021 22:20
@Mr0grog Mr0grog deleted the 42-crash-in-a-blaze-of-glory-when-process-pool-dies branch January 1, 2021 22:23
Mr0grog added a commit to edgi-govdata-archiving/web-monitoring-ops that referenced this pull request Jan 1, 2021
This is a test of edgi-govdata-archiving/web-monitoring-diff#49. We want this to run for a few days and make sure there aren't stability problems. If there are, we'll flip the default for `RESTART_BROKEN_DIFFER` to true instead of false.
@Mr0grog
Copy link
Member Author

Mr0grog commented Jan 1, 2021

This has been released and deployed as part of v0.1.2rc1. If it seems stable in production over a few days, we’ll make a final release as-is. If not, we’ll flip the default for RESTART_BROKEN_DIFFER (so it’s True) and release that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Server should crash if diff process pool is irrecoverable
1 participant