porcelain.status: skip traversal of gitignore untracked dirs

[pmrowla]

When using porcelain.get_untracked_paths with exclude_ignored, currently every untracked path is checked to see if it is gitignored. If a directory is gitignored, we can skip checking the remainder of the files inside that directory - from the documentation for gitignore:

Git doesn’t list excluded directories for performance reasons, so any patterns on contained files have no effect, no matter where they are defined.

Changes after this PR:

• porcelain.get_untracked_files: no longer walks ignored directories for performance reasons. The path to the ignored dir (including trailing slash) will be yielded if exclude_ignored=False, but individual files within the directory will not be included.
• porcelain.add: now only returns path to ignored directory in the (added, ignored) tuple (previously all files inside ignored dirs would be returned)
• porcelain.status: now only returns path to ignored directory when called with ignored=True (previously all files inside ignored dirs would be returned)

[codecov]

Codecov Report

Merging #853 (24ec0da) into master (c60b83b) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #853      +/-   ##
==========================================
+ Coverage   84.67%   84.69%   +0.01%     
==========================================
  Files          90       90              
  Lines       22019    22047      +28     
  Branches     2354     2358       +4     
==========================================
+ Hits        18645    18673      +28     
  Misses       2961     2961              
  Partials      413      413              

Impacted Files	Coverage Δ
dulwich/porcelain.py	81.01% <100.00%> (+0.29%)	⬆️
dulwich/tests/test_porcelain.py	99.91% <100.00%> (+<0.01%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c60b83b...24ec0da. Read the comment docs.

[pmrowla]

porcelain.add could probably also benefit from this?

Although the current behavior is to return the tuple of (added, ignored) where ignored contains all ignored paths, so it would need to be changed to return ignored directory paths instead of the files inside the directory

[jelmer]

The changes look good - any chance you could add a test to make sure this doesn't regress?

[jelmer]

I think it would make sense to change the behaviour of add as well - we should make sure to document the change in behaviour in NEWS.

[pmrowla]

This test should catch any future regression, if we recurse into the ignored subdir here, we will end up yielding subdir/ignored as well.

[pmrowla]

@jelmer this is also ready for review