Skip to content

Update 'git maintenance' to match upstream #301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 39 commits into from
Dec 15, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
28cb5e6
maintenance: add prefetch task
derrickstolee Sep 25, 2020
252cfb7
maintenance: add loose-objects task
derrickstolee Sep 25, 2020
3e220e6
maintenance: create auto condition for loose-objects
derrickstolee Sep 25, 2020
18e449f
midx: enable core.multiPackIndex by default
derrickstolee Sep 25, 2020
efdd2f0
midx: use start_delayed_progress()
derrickstolee Sep 25, 2020
52fe41f
maintenance: add incremental-repack task
derrickstolee Sep 25, 2020
a13e3d0
maintenance: auto-size incremental-repack batch
derrickstolee Sep 25, 2020
e841a79
maintenance: add incremental-repack auto condition
derrickstolee Sep 25, 2020
1942d48
maintenance: optionally skip --auto process
derrickstolee Aug 28, 2020
b08ff1f
maintenance: add --schedule option and config
derrickstolee Sep 11, 2020
4950b2a
for-each-repo: run subcommands on configured repos
derrickstolee Sep 11, 2020
0c18b70
maintenance: add [un]register subcommands
derrickstolee Sep 11, 2020
2fec604
maintenance: add start/stop subcommands
derrickstolee Sep 11, 2020
a4cb1a2
maintenance: create maintenance.strategy config
derrickstolee Oct 15, 2020
61f7a38
maintenance: use 'incremental' strategy by default
derrickstolee Oct 15, 2020
0016b61
maintenance: add troubleshooting guide to docs
derrickstolee Oct 15, 2020
31345d5
maintenance: extract platform-specific scheduling
derrickstolee Nov 24, 2020
16c5690
maintenance: include 'cron' details in docs
derrickstolee Nov 24, 2020
f86987e
maintenance: use launchctl on macOS
derrickstolee Nov 24, 2020
d386975
maintenance: use Windows scheduled tasks
derrickstolee Dec 9, 2020
aaf9d18
t7900: fix test failures when invoked individually via --run
sunshineco Nov 30, 2020
59d499b
t7900: make macOS-specific test work on Windows
sunshineco Nov 30, 2020
171a9f0
fixup! maintenance: add troubleshooting guide to docs
derrickstolee Dec 15, 2020
13907a3
fixup! maintenance: recommended schedule in register/start
derrickstolee Dec 15, 2020
67e7386
fixup! maintenance: add start/stop subcommands
derrickstolee Dec 15, 2020
6476a98
fixup! maintenance: add [un]register subcommands
derrickstolee Dec 15, 2020
cdd31e1
fixup! for-each-repo: run subcommands on configured repos
derrickstolee Dec 15, 2020
fc11f21
fixup! maintenance: add --schedule option and config
derrickstolee Dec 15, 2020
0ce59a1
fixup! maintenance: optionally skip --auto process
derrickstolee Dec 15, 2020
ad4506e
fixup! maintenance: add incremental-repack auto condition
derrickstolee Dec 15, 2020
326391a
fixup! maintenance: auto-size incremental-repack batch
derrickstolee Dec 15, 2020
db81d4a
fixup! maintenance: add incremental-repack task
derrickstolee Dec 15, 2020
b2ec78d
fixup! midx: use start_delayed_progress()
derrickstolee Dec 15, 2020
77508e6
fixup! midx: enable core.multiPackIndex by default
derrickstolee Dec 15, 2020
ade97b6
fixup! maintenance: create auto condition for loose-objects
derrickstolee Dec 15, 2020
13f4eb7
fixup! maintenance: add loose-objects task
derrickstolee Dec 15, 2020
db84c64
fixup! maintenance: add prefetch task
derrickstolee Dec 15, 2020
7a2faf2
Merge current ds/maintenance-part-4 into vfs-2.29.0
derrickstolee Dec 15, 2020
2864f7c
maintenance: care about gvfs.sharedCache config
derrickstolee Dec 15, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions Documentation/config/maintenance.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,21 @@ maintenance.auto::
`git maintenance run --auto` after doing their normal work. Defaults
to true.

maintenance.strategy::
This string config option provides a way to specify one of a few
recommended schedules for background maintenance. This only affects
which tasks are run during `git maintenance run --schedule=X`
commands, provided no `--task=<task>` arguments are provided.
Further, if a `maintenance.<task>.schedule` config value is set,
then that value is used instead of the one provided by
`maintenance.strategy`. The possible strategy strings are:
+
* `none`: This default setting implies no task are run at any schedule.
* `incremental`: This setting optimizes for performing small maintenance
activities that do not delete any data. This does not schedule the `gc`
task, but runs the `prefetch` and `commit-graph` tasks hourly and the
`loose-objects` and `incremental-repack` tasks daily.

maintenance.<task>.enabled::
This boolean config option controls whether the maintenance task
with name `<task>` is run when no `--task` option is specified to
Expand Down
126 changes: 122 additions & 4 deletions Documentation/git-maintenance.txt
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,18 @@ register::
for running in the background without disrupting foreground
processes.
+
If your repository has no `maintenance.<task>.schedule` configuration
values set, then Git will use a recommended default schedule that performs
background maintenance that will not interrupt foreground commands. The
default schedule is as follows:
The `register` subcomand will also set the `maintenance.strategy` config
value to `incremental`, if this value is not previously set. The
`incremental` strategy uses the following schedule for each maintenance
task:
+
--
* `gc`: disabled.
* `commit-graph`: hourly.
* `prefetch`: hourly.
* `loose-objects`: daily.
* `incremental-repack`: daily.
--
+
`git maintenance register` will also disable foreground maintenance by
setting `maintenance.auto = false` in the current repository. This config
Expand Down Expand Up @@ -216,6 +218,122 @@ Further, the `git gc` command should not be combined with
but does not take the lock in the same way as `git maintenance run`. If
possible, use `git maintenance run --task=gc` instead of `git gc`.

The following sections describe the mechanisms put in place to run
background maintenance by `git maintenance start` and how to customize
them.

BACKGROUND MAINTENANCE ON POSIX SYSTEMS
---------------------------------------

The standard mechanism for scheduling background tasks on POSIX systems
is cron(8). This tool executes commands based on a given schedule. The
current list of user-scheduled tasks can be found by running `crontab -l`.
The schedule written by `git maintenance start` is similar to this:

-----------------------------------------------------------------------
# BEGIN GIT MAINTENANCE SCHEDULE
# The following schedule was created by Git
# Any edits made in this region might be
# replaced in the future by a Git command.

0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly
0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily
0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly

# END GIT MAINTENANCE SCHEDULE
-----------------------------------------------------------------------

The comments are used as a region to mark the schedule as written by Git.
Any modifications within this region will be completely deleted by
`git maintenance stop` or overwritten by `git maintenance start`.

The `crontab` entry specifies the full path of the `git` executable to
ensure that the executed `git` command is the same one with which
`git maintenance start` was issued independent of `PATH`. If the same user
runs `git maintenance start` with multiple Git executables, then only the
latest executable is used.

These commands use `git for-each-repo --config=maintenance.repo` to run
`git maintenance run --schedule=<frequency>` on each repository listed in
the multi-valued `maintenance.repo` config option. These are typically
loaded from the user-specific global config. The `git maintenance` process
then determines which maintenance tasks are configured to run on each
repository with each `<frequency>` using the `maintenance.<task>.schedule`
config options. These values are loaded from the global or repository
config values.

If the config values are insufficient to achieve your desired background
maintenance schedule, then you can create your own schedule. If you run
`crontab -e`, then an editor will load with your user-specific `cron`
schedule. In that editor, you can add your own schedule lines. You could
start by adapting the default schedule listed earlier, or you could read
the crontab(5) documentation for advanced scheduling techniques. Please
do use the full path and `--exec-path` techniques from the default
schedule to ensure you are executing the correct binaries in your
schedule.


BACKGROUND MAINTENANCE ON MACOS SYSTEMS
---------------------------------------

While macOS technically supports `cron`, using `crontab -e` requires
elevated privileges and the executed process does not have a full user
context. Without a full user context, Git and its credential helpers
cannot access stored credentials, so some maintenance tasks are not
functional.

Instead, `git maintenance start` interacts with the `launchctl` tool,
which is the recommended way to schedule timed jobs in macOS. Scheduling
maintenance through `git maintenance (start|stop)` requires some
`launchctl` features available only in macOS 10.11 or later.

Your user-specific scheduled tasks are stored as XML-formatted `.plist`
files in `~/Library/LaunchAgents/`. You can see the currently-registered
tasks using the following command:

-----------------------------------------------------------------------
$ ls ~/Library/LaunchAgents/org.git-scm.git*
org.git-scm.git.daily.plist
org.git-scm.git.hourly.plist
org.git-scm.git.weekly.plist
-----------------------------------------------------------------------

One task is registered for each `--schedule=<frequency>` option. To
inspect how the XML format describes each schedule, open one of these
`.plist` files in an editor and inspect the `<array>` element following
the `<key>StartCalendarInterval</key>` element.

`git maintenance start` will overwrite these files and register the
tasks again with `launchctl`, so any customizations should be done by
creating your own `.plist` files with distinct names. Similarly, the
`git maintenance stop` command will unregister the tasks with `launchctl`
and delete the `.plist` files.

To create more advanced customizations to your background tasks, see
launchctl.plist(5) for more information.


BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
-----------------------------------------

Windows does not support `cron` and instead has its own system for
scheduling background tasks. The `git maintenance start` command uses
the `schtasks` command to submit tasks to this system. You can inspect
all background tasks using the Task Scheduler application. The tasks
added by Git have names of the form `Git Maintenance (<frequency>)`.
The Task Scheduler GUI has ways to inspect these tasks, but you can also
export the tasks to XML files and view the details there.

Note that since Git is a console application, these background tasks
create a console window visible to the current user. This can be changed
manually by selecting the "Run whether user is logged in or not" option
in Task Scheduler. This change requires a password input, which is why
`git maintenance start` does not select it by default.

If you want to customize the background tasks, please rename the tasks
so future calls to `git maintenance (start|stop)` do not overwrite your
custom tasks.


GIT
---
Expand Down
Loading