Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1682638 - Multiple fixes: Don't let the queue run, handle empty databases #1398

Merged
merged 10 commits into from
Dec 17, 2020

Conversation

badboy
Copy link
Member

@badboy badboy commented Dec 16, 2020

I identified 2 problems, that in combination cause the crashes in bug 1682638.

  1. Panic on shutdown

See document.
When init fails for some reason we never set a global Glean object. However on shutdown we try to unblock the dispatcher, which might have some recording tasks in there already. These should never be executed without a Glean.

The first 4 commits address that: we know when it is initialized, so we can avoid emptying the dispatcher if it hasn't.

  1. Empty database files

For reasons unknown to me right now the database file was empty on at least the machine of one tester.
Rkv in safe-mode considers empty files as invalid files. I filed this upstream.
The solution (for now) is to remove that file and start over again.


For both cases I added new tests.

@badboy badboy requested a review from Dexterp37 December 16, 2020 13:51
@auto-assign auto-assign bot requested a review from travis79 December 16, 2020 13:52
@badboy badboy removed the request for review from travis79 December 16, 2020 13:58
glean-core/rlb/tests/common/mod.rs Show resolved Hide resolved
glean-core/rlb/tests/fog_simple.rs Outdated Show resolved Hide resolved
glean-core/rlb/tests/fog_simple.rs Outdated Show resolved Hide resolved
glean-core/rlb/tests/fog_init_fails.rs Outdated Show resolved Hide resolved
glean-core/rlb/tests/common/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@chutten chutten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drive by comment pile.

glean-core/rlb/tests/common/mod.rs Outdated Show resolved Hide resolved
glean-core/rlb/tests/common/mod.rs Outdated Show resolved Hide resolved
glean-core/rlb/tests/common/mod.rs Outdated Show resolved Hide resolved
This test as of this commit fails.
There's no point in waiting for it to finish initialization if we're
shutting down. There might be multiple operations queued, which could
block for a long time. We should not hold off shutdown in those cases.

Note: If Glean was already fully initialized shutdown will wait for
these operations to finish before it shuts down.
Invalid means: either empty or not bincode-deserializable.
`test_reset_glean` will already do the right thing:

* Destroys the Glean object if any
* Resets the dispatcher, but only if Glean was initialized.
@badboy badboy force-pushed the dont-crash-if-init-fails branch from a25ab44 to c0abb67 Compare December 16, 2020 16:33
@badboy badboy requested review from Dexterp37 and chutten December 16, 2020 16:40
Copy link
Contributor

@Dexterp37 Dexterp37 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@badboy would you consider dropping the FOG prefix from the filenames and the references in the test files?

@badboy badboy requested a review from Dexterp37 December 16, 2020 17:18
Copy link
Contributor

@Dexterp37 Dexterp37 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you :)

@badboy badboy merged commit 7ad9998 into main Dec 17, 2020
@badboy badboy deleted the dont-crash-if-init-fails branch December 17, 2020 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants