Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip-worktree support, sparse checkout impl #1495

Open
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

lmmx
Copy link

@lmmx lmmx commented Feb 12, 2025

This PR adds and refines functionality around sparse checkout, including:

  • Skip-worktree bit support in IndexEntry, preserving extended_flags so that excluded files actually stay out of the working tree.
  • Updates to write_index so that index entries with extended flags are written as version 3 or higher.
  • New TDD tests in test_porcelain.py for cone-mode sparse checkout (currently skipped, as cone mode is not fully implemented yet, but the tests are in place for future development). I did my best but I decided to submit now and not potentially spoil it. (I can take these out but I am confident I'll be able to do cone mode with a bit more thought)
  • Amended tests in test_index.py
  • Local modifications checks in the porcelain sparse_checkout function, so we can raise or force-remove on excluded paths.

All tests now pass (aside from the intentionally skipped cone-mode ones). I ran make check and hope this can land.

lmmx added 10 commits February 12, 2025 00:45
Previously, IndexEntry did not store or expose the extended_flags field, causing any skip-worktree bits to be discarded when reading or writing the index. This patch adds flags and extended_flags fields to IndexEntry, and updates from_serialized/serialize to preserve them.

In particular:
- We store both flags and extended_flags from SerializedIndexEntry.
- We clear/set stage bits in flags, preserving the rest (including the
  extended flag).
- We introduce a helper method set_skip_worktree() to enable or disable
  the skip-worktree bit in extended_flags, and set/unset FLAG_EXTENDED
  as needed.

With these changes, Dulwich can now retain and manipulate skip-worktree
bits when reading and writing .git/index, enabling sparse checkout
functionality to match Git’s behavior.
@lmmx lmmx requested a review from jelmer as a code owner February 12, 2025 02:59
@lmmx lmmx changed the title Add skip-worktree support and TDD for cone-mode sparse checkout Skip-worktree support, sparse checkout impl and cone-mode TDD Feb 12, 2025
@lmmx lmmx changed the title Skip-worktree support, sparse checkout impl and cone-mode TDD Skip-worktree support, sparse checkout impl Feb 12, 2025
@lmmx
Copy link
Author

lmmx commented Feb 12, 2025

Removed the cone mode TDDs (doesn't make sense to introduce them prematurely) and ran the same tests the CI runs (I thought it was just make check!)

make check
python -m mypy dulwich
python -m ruff check .
python -m ruff format check .

Should go green now 🤞

dulwich/porcelain.py Outdated Show resolved Hide resolved
dulwich/porcelain.py Outdated Show resolved Hide resolved
tests/test_porcelain.py Outdated Show resolved Hide resolved
uv.lock Outdated Show resolved Hide resolved
@jelmer
Copy link
Owner

jelmer commented Feb 12, 2025

Thanks for working on this - overall looks great, but some minor comments inline and one larger one about placement of the code in porcelain.py.

@lmmx
Copy link
Author

lmmx commented Feb 12, 2025

Thanks for reviewing! Yes would be great to land this and extend to further steps.

Apologies for the roughness, did not mean to include a uv lockfile, and yes I meant to remove those codenames.

Will revise and resubmit shortly

@lmmx
Copy link
Author

lmmx commented Feb 13, 2025

  • All comments addressed
  • Tests pass
  • make check passes

Ready for re-review @jelmer 😎

@jelmer
Copy link
Owner

jelmer commented Feb 15, 2025

Changes look good; please take a look at the ruff failures

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants