-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ReposVerb: allow registering vanilla Git repos #258
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good, some minor suggestions.
@jrbriggs: Thanks for the feedback. I'll respond to that soon. I'm marking this PR as WIP for the remainder of the holidays. I'm going to go a bit nuts adding stuff to this PR until I feel I have a "complete" picture of the feature. Then, we can split it into smaller pieces for review when people have bandwidth. |
Signed-off-by: Derrick Stolee <stolee@gmail.com>
This also allows re-registering Scalar repos if they somehow become unregistered. (We will follow-up with a way to un-watch later.) The only required update to the maintenance steps right now is to make the fetch-commits-and-trees step be a no-op for non-Scalar repos. Eventually, we will want that to run a "git fetch" (in a way that does not update the refs/remotes space). Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
When we run functional tests inside the Scalar source code, we need our vanilla Git repos to be recognized inside that repo. The old logic would look for "src/.git" first, finding the Scalar repo before looking just for ".git". This makes the code simpler, too! Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
These commits are cherry-picked from #258 with slight modifications. The functionality here makes sense for repos cloned by Scalar, allowing us to focus later on the interactions with vanilla Git repos. 1. Add `ConfigStep` for running the Git config commands at clone time and as the service starts up. This already has some split for vanilla Git repos, but that code does not get exercised. Resolves #252. 2. The `PackfileMaintenanceStep` attempts to repack all but the largest pack when those packs sum to less than 2gb. 3. Some cleanup things here and there. 4. `scalar repos (add|list)` allows tracking the registered repos. Resolves some of #214.
Replaces #258. Resolves #254. If you have a "vanilla" Git repo, then run `scalar repos add` in that repo. Scalar will detect the repo root, register that root with the service, and the maintenance steps will run. The way we identify a vanilla Git repo (now) is if the object cache dir is not given by config and hence we do all operations on the local `.git` folder. This will not work with Git worktrees, but we will also not discover a `.git` _folder_ (worktrees use a `.git` _file_) so the enlistment will be invalid. * The `commit-graph` task will generate the incremental commit-graph files using the `--reachable` option. This will be a no-op if the user has not changed their refs. * The `packfile` task will write a `multi-pack-index` and do the expire/repack logic. To actually do something, the batch-size is decreased if the total pack size is smaller than 2gb. (This was done in #272.) * The `loose-objects` task will delete loose objects already in pack-files and put them into a new `from-loose` pack. This required no change. * The `fetch-commits-and-trees` task is renamed the `fetch` task. On a vanilla Git repo, it will simply call `git fetch <remote> --no-update-remote-refs +refs/head/*:refs/hidden/<remote>/*` for each `<remote>` given by `git remote`. This downloads a pack-file with the new objects not reachable form the current refs, but also does not update `refs/remotes/`. The user will see their refs update as normal, but the pack download is much smaller. * The `config` task (added by #272) runs the necessary Git config. This step is run in `scalar repos add`, but also in the background with the service. All config options become optional for vanilla Git repos, so a user can opt-out of the settings we recommend.
Part of #254 and half of #214. Resolves #252.
The
scalar repos
verb allows two modes:scalar repos add
: add the current enlistment to the list being watched by the Scalar service.scalar repos list
: list the current set of enlistments being watched by the Scalar service.scalar repos add
inside a Scalar or Git repository will register the repo with the Scalar service. The service will start running maintenance on that repo.The way we identify a vanilla Git repo (now) is if the object cache dir is not given by config and hence we do all operations on the local
.git
folder. This will not work with Git worktrees, but we will also not discover a.git
folder (worktrees use a.git
file) so the enlistment will be invalid.The
commit-graph
task will generate the incremental commit-graph files using the--reachable
option. This will be a no-op if the user has not changed their refs.The
packfile
task will write amulti-pack-index
and do the expire/repack logic. To actually do something, the batch-size is decreased if the total pack size is smaller than 2gb.The
loose-objects
task will delete loose objects already in pack-files and put them into a newfrom-loose
pack.The
fetch-commits-and-trees
task is renamed thefetch
task. On a vanilla Git repo, it will simply callgit fetch origin --no-update-remote-refs +refs/head/*:refs/hidden/*
. This downloads a pack-file with the new objects not reachable form the current refs, but also does not updaterefs/remotes/
. The user will see their refs update as normal, but the pack download is much smaller.The new
config
task runs the necessary Git config. This step is run atclone
andadd
time, but also in the background with the service. Some config options are removed as required for vanilla Git repos.