Thu, 26 Sep 2024 20:32:11 +0200

Add experimental 'git survey' builtin (#5174)

This introduces `git survey` to Git for Windows ahead of upstream for
the express purpose of getting the path-based analysis in the hands of
more folks.

The inspiration of this builtin is
[`git-sizer`](https://github.com/github/git-sizer), but since that
command relies on `git cat-file --batch` to get the contents of objects,
it has limits to how much information it can provide.

This is mostly a rewrite of the `git survey` builtin that was introduced
into the `microsoft/git` fork in microsoft/git#667. That version had a
lot more bells and whistles, including an analysis much closer to what
`git-sizer` provides.

The biggest difference in this version is that this one is focused on
using the path-walk API in order to visit batches of objects based on a
common path. This allows identifying, for instance, the path that is
contributing the most to the on-disk size across all versions at that
path.

For example, here are the top ten paths contributing to my local Git
repository (which includes `microsoft/git` and `gitster/git`):

```
TOP FILES BY DISK SIZE
============================================================================
                                    Path | Count | Disk Size | Inflated Size
-----------------------------------------+-------+-----------+--------------
                       whats-cooking.txt |  1373 |  11637459 |      37226854
             t/helper/test-gvfs-protocol |     2 |   6847105 |      17233072
                      git-rebase--helper |     1 |   6027849 |      15269664
                          compat/mingw.c |  6111 |   5194453 |     463466970
             t/helper/test-parse-options |     1 |   3420385 |       8807968
                  t/helper/test-pkt-line |     1 |   3408661 |       8778960
      t/helper/test-dump-untracked-cache |     1 |   3408645 |       8780816
            t/helper/test-dump-fsmonitor |     1 |   3406639 |       8776656
                                po/vi.po |   104 |   1376337 |      51441603
                                po/de.po |   210 |   1360112 |      71198603
```

This kind of analysis has been helpful in identifying the reasons for
growth in a few internal monorepos. Those findings motivated the changes
in #5157 and #5171.

With this early version in Git for Windows, we can expand the reach of
the experimental tool in advance of it being contributed to the upstream
project.

Unfortunately, this will mean that in the next `microsoft/git` rebase,
@jeffhostetler's version will need to be pulled out since there are
enough conflicts. These conflicts include how tables are stored and
generated, as the version in this PR is slightly more general to allow
for different kinds of data.

Thu, 26 Sep 2024 13:10:40 +0200

Introduce 'git backfill' to get missing blobs in a partial clone (#5172)

This change introduces the `git backfill` command which uses the path
walk API to download missing blobs in a blobless partial clone.

By downloading blobs that correspond to the same file path at the same
time, we hope to maximize the potential benefits of delta compression
against multiple versions.

These downloads occur in a configurable batch size, presenting a
mechanism to perform "resumable" clones: `git clone --filter=blob:none`
gets the commits and trees, then `git backfill` will download all
missing blobs. If `git backfill` is interrupted partway through, it can
be restarted and will redownload only the missing objects.

When combining blobless partial clones with sparse-checkout, `git
backfill` will assume its `--sparse` option and download only the blobs
within the sparse-checkout. Users may want to do this as the repo size
will still be smaller than the full repo size, but commands like `git
blame` or `git log -L` will not suffer from many one-by-one blob
downloads.

Future directions should consider adding a pathspec or file prefix to
further focus which paths are being downloaded in a batch.

Wed, 25 Sep 2024 16:48:41 -0400

Add path walk API and its use in 'git pack-objects' (#5171)

This is a follow up to #5157 as well as motivated by the RFC in
gitgitgadget/git#1786.

We have ways of walking all objects, but it is focused on visiting a
single commit and then expanding the new trees and blobs reachable from
that commit that have not been visited yet. This means that objects
arrive without any locality based on their path.

Add a new "path walk API" that focuses on walking objects in batches
according to their type and path. This will walk all annotated tags, all
commits, all root trees, and then start a depth-first search among all
paths in the repo to collect trees and blobs in batches.

The most important application for this is being fast-tracked to Git for
Windows: `git pack-objects --path-walk`. This application of the path
walk API discovers the objects to pack via this batched walk, and
automatically groups objects that appear at a common path so they can be
checked for delta comparisons.

This use completely avoids any name-hash collisions (even the collisions
that sometimes occur with the new `--full-name-hash` option) and can be
much faster to compute since the first pass of delta calculations does
not waste time on objects that are unlikely to be diffable.

Some statistics are available in the commit messages.

Tue, 24 Sep 2024 13:39:58 +0200

Merge 'readme' into HEAD

Add a README.md for GitHub goodness.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Tue, 24 Sep 2024 12:07:32 +0200

ci: work around a problem with HTTP/2 vs libcurl v8.10.0 (#5165)

This patch implements a work-around, basically for the sake of Git for
Windows v2.46.2's CI build: The `macos-13` runner images still reference
libcurl v8.10.0, which has a known flaw that breaks t5559.30. Let's
detect this situation and skip that test case in that case.

While at it, also backport the work-around for a flaky test case
(t5512.40 "helper with refspec capability fails gracefully").

This fixes https://github.com/git-for-windows/git/issues/5159.

Wed, 18 Sep 2024 09:46:09 +0200

Merge 'readme' into HEAD

Add a README.md for GitHub goodness.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Mon, 16 Sep 2024 12:27:15 +0200

Backport `linux32` CI fix from upstream Git (#5145)

The `linux32` jobs are seeing this error:

    Error: This request has been automatically failed because it uses a
    deprecated version of `actions/upload-artifact: v1`. Learn more:

https://github.blog/changelog/2024-02-13-deprecation-notice-v1-and-v2-of-the-artifact-actions/

To maintain CI builds that typically succeed, let's cherry-pick this
into Git for Windows' `main` branch.

Tue, 13 Aug 2024 14:01:04 +0200

lazycurl: prepare for new upstream Git requirements (#5102)

An [upcoming change in
Git](https://github.com/git/git/commit/0ca365c2ed48084974c7081bdfe3189094a2b993)
wants it to call `curl_version_info()`, which the lazy cURL stuff hereby
learns about.

Wed, 7 Aug 2024 22:22:38 +0200

[bugfix] fix wsl mode bits error in subdir (#4660)

Support for wsl mode bits was previously added to git, but there was a
bug because the filenames provided by fscache did not contain paths.

This commit fixes the issue.

The previous feature is added in PR #4438,
but at that time I didn't tested so much.
Sorry to have this bug.

To test this feature, set core.wslcompat to ture and core.filemode to
true and make sure repo is on NTFS.

Wed, 7 Aug 2024 19:34:48 +0200

Do not redefine _FILE_STAT_LX_INFORMATION (#5092)

The struct definition as well as the other defines are already defined
in Windows SDK >= 10.1.26100.1.