Rebase to v2.27.0-rc0 #2626
Merged
Rebase to v2.27.0-rc0 #2626
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Perforce allows you commit files and directories with the same name,
so you could have files //depot/foo and //depot/foo/bar both checked
in. A p4 sync of a repository in this state fails. Deleting one of
the files recovers the repository.
When this happens we want git-p4 to recover in the same way as
perforce.
Note that Perforce has this change in their 2017.1 version:
Bugs fixed in 2017.1
#1489051 (Job git-for-windows#2170) **
Submitting a file with the same name as an existing depot
directory path (or vice versa) will now be rejected.
so people hopefully will not creating damaged Perforce repos
anymore, but "git p4" needs to be able to interact with already
corrupt ones.
Signed-off-by: Andrew Oakley <andrew@adoakley.name>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Tree entries are sorted in path order, meaning that directory names get
a slash ('/') appended implicitly. Git fsck checks if trees contains
consecutive duplicates, but due to that ordering there can be
non-consecutive duplicates as well if one of them is a directory and the
other one isn't. Such a tree cannot be fully checked out.
Find these duplicates by recording candidate file names on a stack and
check candidate directory names against that stack to find matches.
Suggested-by: Brandon Williams <bwilliamseng@gmail.com>
Original-test-by: Brandon Williams <bwilliamseng@gmail.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When computing changed-path Bloom filters for a commit, we need to know if the commit has a parent or not. If the commit is not parsed, then its parent pointer will be NULL. As far as I can tell, the only opportunity to reach this code without parsing the commit is inside "test-tool bloom get_filter_for_commit" but it is best to be safe. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
In Documentation/technical/commit-graph-format.txt, the definition of the BIDX chunk specifies the length is a number of 8-byte words. During development we discovered that using 8-byte words in the Murmur3 hash algorithm causes issues with big-endian versus little- endian machines. Thus, the hash algorithm was adapted to work on a byte-by-byte basis. However, this caused a change in the definition of a "word" in bloom.h. Now, a "word" is a single byte, which allows filters to be as small as two bytes. These length-two filters are demonstrated in t0095-bloom.sh, and a larger filter of length 25 is demonstrated as well. The original point of using 8-byte words was for alignment reasons. It also presented opportunities for extremely sparse Bloom filters when there were a small number of changes at a commit, creating a very low false-positive rate. However, modifying the format at this point is unlikely to be a valuable exercise. Also, this use of single-byte granularity does present opportunities to save space. It is unclear if 8-byte alignment of the filters would present any meaningful performance benefits. Modify the format document to reflect reality. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When computing a changed-path Bloom filter, we need to take the files that changed from the diff computation and extract the parent directories. That way, a directory pathspec such as "Documentation" could match commits that change "Documentation/git.txt". However, the current code does a poor job of this process. The paths are added to a hashmap, but we do not check if an entry already exists with that path. This can create many duplicate entries and cause the filter to have a much larger length than it should. This means that the filter is more sparse than intended, which helps the false positive rate, but wastes a lot of space. Properly use hashmap_get() before hashmap_add(). Also be sure to include a comparison function so these can be matched correctly. This has an effect on a test in t0095-bloom.sh. This makes sense, there are ten changes inside "smallDir" so the total number of paths in the filter should be 11. This would result in 11 * 10 bits required, and with 8 bits per byte, this results in 14 bytes. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
As diff_tree_oid() computes a diff, it will terminate early if the total number of changed paths is strictly larger than max_changes. This includes the directories that changed, not just the file paths. However, only the file paths are reflected in the resulting diff queue's "nr" value. Use the "num_changes" from diffopt to check if the diff terminated early. This is incredibly important, as it can result in incorrect filters! For example, the first commit in the Linux kernel repo reports only 471 changes, but since these are nested inside several directories they expand to 513 "real" changes, and in fact the total list of changes is not reported. Thus, the computed filter for this commit is incorrect. Demonstrate the subtle difference by using one fewer file change in the 'get bloom filter for commit with 513 changes' test. Before, this edited 513 files inside "bigDir" which hit this inequality. However, dropping the file count by one demonstrates how the previous inequality was incorrect but the new one is correct. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rather than teaching only one operation, like 'git fetch', how to write down throughput to traces, we can learn about a wide range of user operations that may seem slow by adding tooling to the progress library itself. Operations which display progress are likely to be slow-running and the kind of thing we want to monitor for performance anyways. By showing object counts and data transfer size, we should be able to make some derived measurements to ensure operations are scaling the way we expect. Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Code cleanup.
* tb/shallow-cleanup:
shallow: use struct 'shallow_lock' for additional safety
shallow.h: document '{commit,rollback}_shallow_file'
shallow: extract a header file for shallow-related functions
commit: make 'commit_graft_pos' non-static
The object walk with object filter "--filter=tree:0" can now take advantage of the pack bitmap when available. * tb/bitmap-walk-with-tree-zero-filter: pack-bitmap: pass object filter to fill-in traversal pack-bitmap.c: support 'tree:0' filtering pack-bitmap.c: make object filtering functions generic list-objects-filter: treat NULL filter_options as "disabled"
Minor in-code comments and documentation updates around credential API. * cb/credential-doc-fixes: credential: document protocol updates credential: update gitcredentials documentation credential: correct order of parameters for credential_match credential: update description for credential_from_url_gently
Teach "am", "commit", "merge" and "rebase", when they are run with the "--quiet" option, to pass "--quiet" down to "gc --auto". * jc/auto-gc-quiet: auto-gc: pass --quiet down from am, commit, merge and rebase auto-gc: extract a reusable helper from "git fetch"
Random bits of FAQ. * ss/faq-ignore: gitfaq: files in .gitignore are tracked
Random bits of FAQ. * ss/faq-fetch-pull: gitfaq: fetching and pulling a repository
Instead of always building all branches at GitHub via Actions, users can specify which branches to build. * jk/ci-only-on-selected-branches: ci: allow per-branch config for GitHub Actions
Code clean-up. * dd/bloom-sparse-fix: bloom: fix `make sparse` warning
Rewriting various parts of "git submodule" in C continues. * ss/submodule-set-url-in-c: submodule: port subcommand 'set-url' from shell to C
The code to skip unmerged paths in the index when sparse checkout is in use would have made out-of-bound access of the in-core index when the last path was unmerged, which has been corrected. * ds/sparse-updates-oob-access-fix: unpack-trees: avoid array out-of-bounds error
Serving a "git fetch" client over "git://" and "ssh://" protocols using the on-wire protocol version 2 was buggy on the server end when the client needs to make a follow-up request to e.g. auto-follow tags. * cc/upload-pack-v2-fetch-fix: upload-pack: clear filter_options for each v2 fetch command
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git bugreport" learned to report enabled hooks in the repository. * es/bugreport-with-hooks: bugreport: collect list of populated hooks
"git bisect replay" had trouble with input files when they used CRLF line ending, which has been corrected. * cw/bisect-replay-with-dos: bisect: allow CRLF line endings in "git bisect replay" input
Doc update. * jc/codingstyle-compare-with-null: CodingGuidelines: do not ==/!= compare with 0 or '\0' or NULL
"rebase -i" segfaulted when rearranging a sequence that has a fix-up that applies another fix-up (which may or may not be a fix-up of yet another step). * js/rebase-autosquash-double-fixup-fix: rebase --autosquash: fix a potential segfault
"git p4" learned to recover from a (broken) state where a directory and a file are recorded at the same path in the Perforce repository the same way as their clients do. * ao/p4-d-f-conflict-recover: git-p4: recover from inconsistent perforce history
"git fsck" ensures that the paths recorded in tree objects are sorted and without duplicates, but it failed to notice a case where a blob is followed by entries that sort before a tree with the same name. This has been corrected. * rs/fsck-duplicate-names-in-trees: fsck: report non-consecutive duplicate names in trees
Code cleanup and typofixes * ds/bloom-cleanup: completion: offer '--(no-)patch' among 'git log' options bloom: use num_changes not nr for limit detection bloom: de-duplicate directory entries Documentation: changed-path Bloom filters use byte words bloom: parse commit before computing filters test-bloom: fix usage typo bloom: fix whitespace around tab length
"git multi-pack-index repack" has been taught to honor some repack.* configuration variables. * sn/midx-repack-with-config: multi-pack-index: respect repack.packKeptObjects=false midx: teach "git multi-pack-index repack" honor "git repack" configurations
Test fix for a topic already in 'master' and meant for 'maint'. * jt/t5500-unflake: t5500: count objects through stderr, not trace
Teach codepaths that show progress meter to also use the start_progress() and the stop_progress() calls as a "region" to be traced. * es/trace-log-progress: trace2: log progress time and throughput
…er/fscache_nfd fscache: add not-found directory cache to fscache
…er/add_preload_fscache add: use preload-index and fscache for performance
…xcludes_with_fscache dir.c: make add_excludes aware of fscache during status
fetch-pack.c: enable fscache for stats under .git/objects
…t_flush checkout.c: enable fscache for checkout again Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…_index Enable the filesystem cache (fscache) in refresh_index().
…-gfw fscache: use FindFirstFileExW to avoid retrieving the short name
…ter-status-gfw status: disable and free fscache at the end of the status command
…e-gfw fscache: add GIT_TEST_FSCACHE support
…ter-add-gfw At the end of the add command, disable and free the fscache
…ics-gfw fscache: add fscache hit statistics
This brings substantial wins in performance because the FSCache is now per-thread, being merged to the primary thread only at the end, so we do not have to lock (except while merging). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…safe-enable-gfw fscache: make fscache_enable() thread safe
…DirectoryFile-gfw fscache: teach fscache to use NtQueryDirectoryFile
When updating the skip-worktree bits in the index to align with new values in a sparse-checkout file, Git scans the entire working directory with lstat() calls. In a sparse-checkout, many of these lstat() calls are for paths that do not exist. Enable the fscache feature during this scan. In a local test of a repo with ~2.2 million paths, updating the index with `git read-tree -m -u HEAD` with a sparse-checkout file containing only `/.gitattributes` improved from 2-3 minutes to 15-20 seconds. More work could be done to stop running lstat() calls when recursing into directories that are known to not exist.
We already avoid traversing NTFS junction points in `git clean -dfx`. With this topic branch, we do that when the FSCache is enabled, too. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This was pull request git-for-windows#1645 from ZCube/master Support windows container. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Specify symlink type in .gitattributes
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Handle Ctrl+C in Git Bash nicely Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This branch allows third-party tools to call `git status --no-lock-index` to avoid lock contention with the interactive Git usage of the actual human user. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…ored-directory-gracefully Phase out `--show-ignored-directory` gracefully
This topic branch re-adds the deprecated --stdin/-z options to `git reset`. Those patches were overridden by a different set of options in the upstream Git project before we could propose `--stdin`. We offered this in MinGit to applications that wanted a safer way to pass lots of pathspecs to Git, and these applications will need to be adjusted. Instead of `--stdin`, `--pathspec-from-file=-` should be used, and instead of `-z`, `--pathspec-file-nul`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add a README.md for GitHub goodness. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Apart from commits that made it upstream already, the range-diff only contains changes in the context. Here is a slightly annotated output of
git range-diff 'origin/master^{/^Start.the.merging-rebase}..origin/master' 'rebase-to-v2.27.0-rc0^{/^Start.the.merging-rebase}..rebase-to-v2.27.0-rc0(whereorigin/masteris 01c50ad andrebase-to-v2.27.0-rc0is fb008e4):Note: the "slight annotation" was performed mostly automatically:
There were three commits that made it upstream which were not annotated automatically: 32ce88d, b11a3e0 and 7665009. I added that annotation manually.