Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store index ids outside of configuration when content is disabled #653

Closed
davidmezzetti opened this issue Jan 30, 2024 · 0 comments
Closed
Assignees
Milestone

Comments

@davidmezzetti
Copy link
Member

When a new embeddings instance is created, the default settings do not store content. In this mode, index ids are added to a separate array and stored directly in the embeddings configuration.

#652 is changing the default configuration format from pickle to json. Given this change, it's best to store ids in a file separate from configuration.

This change will support reading existing embeddings indexes with config["ids"] set. All save calls moving forward will store ids in a separate file, even for existing indexes.

This change has no impacts to indexes with content storage enabled.

@davidmezzetti davidmezzetti added this to the v6.4.0 milestone Jan 30, 2024
@davidmezzetti davidmezzetti self-assigned this Jan 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant