Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 remove dead providers from coordinator #4010

Merged
merged 2 commits into from
May 16, 2024

Conversation

imilchev
Copy link
Member

This should minimize the impact of errors we have observed with the scan API. If a provider crashes, it doesn't get properly cleaned up and results in endless loop of failed scans because no provider instance is available. This change makes sure we remove dead providers after each scan, such that the next scan can start with a fresh provider.

This still has the issue that if we have 1 scan with 300 assets and the provider crashes after scanning the first asset, we will get 299 errors for the remaining assets. The issue should be gone with the next scan. A better approach is to detect that a provider instance is dead before connecting to it, if that is the case, we should implement restart logic. That should happen inside the runtime, since we keep a reference to the RunningProvider in all the runtimes for every asset of the scan. We need to make sure we update the same RunningProvider pointer with a new provider instance.

I added a TODO to the code where I think this logic should be implemented.

imilchev added 2 commits May 16, 2024 11:27
Signed-off-by: Ivan Milchev <ivan@mondoo.com>
Signed-off-by: Ivan Milchev <ivan@mondoo.com>
Copy link
Contributor

Test Results

2 994 tests  +1   2 993 ✅ +1   1m 34s ⏱️ +6s
  329 suites ±0       1 💤 ±0 
   23 files   ±0       0 ❌ ±0 

Results for commit aa17e7d. ± Comparison against base commit 10c73b7.

@preslavgerchev preslavgerchev merged commit bdc7ec4 into main May 16, 2024
15 checks passed
@preslavgerchev preslavgerchev deleted the ivan/coordinator-remove-dead-providers branch May 16, 2024 10:57
@github-actions github-actions bot locked and limited conversation to collaborators May 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants