[RFC] Give SegmentWriter a cachable Searcher #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When repeatedly calling `update_document()`, each call instantiates a `Searcher`, which is expensive. The `SegmentWriter` already has a write lock on the index, so if the conditions are right, let's cache a `Searcher`, override its closing method to keep it open, and close it on `_finish()` if it's around.
Also don't cache a `Searcher` if the `SegmentWriter` is closed.
cclauss
pushed a commit
to cclauss/whoosh-1
that referenced
this pull request
Jan 4, 2024
Merge the main branch from whoosh-community/whoosh into our main branch.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When repeatedly calling
update_document(), each call instantiates aSearcher, which is expensive. TheSegmentWriteralready has a write lock on the index, so if the conditions are right, let's cache aSearcher, override itsclose()method to keep it open, and close it on_finish()if it's around.I think this is safe, but I don't know enough about whoosh to be certain. All tests are still passing, and I didn't break any django-haystack tests with this change either.