Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utilize disk_log for search backend sync #538

Closed
scohen opened this issue Jan 3, 2024 · 0 comments
Closed

Utilize disk_log for search backend sync #538

scohen opened this issue Jan 3, 2024 · 0 comments
Labels
enhancement New feature or request indexing

Comments

@scohen
Copy link
Collaborator

scohen commented Jan 3, 2024

Whenever we sync, we have to write the entire database to the disk, this can be hundreds of megs and can take some time. Instead, we should write deltas using disk_log, and load them after we load the table. We can then rewrite the entire index much more occasionally in a background job.

We can also remove the explicit calls to sync in the search store and let the backend handle writing its state.

@scohen scohen added enhancement New feature or request indexing labels Jan 3, 2024
@scohen scohen closed this as completed Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request indexing
Projects
None yet
Development

No branches or pull requests

1 participant