Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Local registry index cache #66

Closed
DCjanus opened this issue Oct 5, 2020 · 1 comment · Fixed by #69
Closed

Local registry index cache #66

DCjanus opened this issue Oct 5, 2020 · 1 comment · Fixed by #69
Labels
enhancement New feature or request

Comments

@DCjanus
Copy link
Member

DCjanus commented Oct 5, 2020 •
edited
Loading

For now, we get latest version of crate via HTTP call, each crate cost a HTTP call, which might be expensive.

If we keep a full index local cache, and timing sync it, that might be cheaper and faster.

Here are my ideas:

  1. Download github archive file every 5 mins and load everything in memory, for now, crates.io-index archive file is 35MB.
  2. Clone and read via git2 crate, incremental synchronization every 1 min.
  3. Download archive file when first launch, incremental synchronization via github api
@robjtede robjtede changed the title [improve] Local registry index cache Local registry index cache Oct 5, 2020
@robjtede robjtede added the enhancement New feature or request label Oct 5, 2020
@robjtede robjtede mentioned this issue Oct 5, 2020
Open
@paolobarbolini paolobarbolini mentioned this issue Oct 10, 2020
Merged
@DCjanus
Copy link
Member Author

DCjanus commented Oct 14, 2020

Not available in recently

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Clone crates.io-index instead of querying it through GitHub's API deps-rs/deps.rs
2 participants
@robjtede @DCjanus