avoid excessive memory consumption in picker #8127
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pascalkuthe
added
E-easy
cd-a
reviewed
Aug 31, 2023
pascalkuthe
changed the title
the-mikedavis
approved these changes
Aug 31, 2023
dgkf
pushed a commit
to dgkf/helix
that referenced
this pull request
Jan 30, 2024
* avoid excessive memory consumption from file picker * fix typos Co-authored-by: Chris <75008413+cd-a@users.noreply.github.com> --------- Co-authored-by: Chris <75008413+cd-a@users.noreply.github.com>
mtoohey31
pushed a commit
to mtoohey31/helix
that referenced
this pull request
Jun 2, 2024
* avoid excessive memory consumption from file picker * fix typos Co-authored-by: Chris <75008413+cd-a@users.noreply.github.com> --------- Co-authored-by: Chris <75008413+cd-a@users.noreply.github.com>
smortime
pushed a commit
to smortime/helix
that referenced
this pull request
Jul 10, 2024
* avoid excessive memory consumption from file picker * fix typos Co-authored-by: Chris <75008413+cd-a@users.noreply.github.com> --------- Co-authored-by: Chris <75008413+cd-a@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We currently keep the picker around after it's closed to allow it to be reopened. This can lead to excessive memory consumption for very large pickers (like when crawling home with nucleo). This PR ensures that we drop the last picker immediately when a new picker is opened, that we stop streaming in new matches into the closed picker n the background and do not store pickers with more than 100_000 items.
In the long term, a better solution would be to restart the stream (recrawl the FS) for such large pickers instead, but that requires larger changes.