Respect .gitignore file in commits

[Wauplin]

Solves #1826 and follow-up after server-side change (https://github.com/huggingface/moon-landing/pull/8007 - private link).

I have now added a respect_gitignore: bool = True parameter to create_commit and upload_folder to respect the gitignore file, if it exists.

By default we check if a .gitignore file is present in the commit itself. Otherwise the one hosted on the Hub is used. Only the gitignore file at root is respected (not in subdirectories).

I have adapted the documentation and added a few tests for it.

Note: as mentioned by @julien-c in #1826 (comment), let's consider this as a "bug-fix" and not a "breaking change" as we can expect that no-one was intentionally uploading ignored files on purpose.

---

# Test to upload my `huggingface_hub/` folder to the Hub
>>> huggingface-cli upload test_tmp_upload ../huggingface_hub
Skipped upload for 53 LFS file(s) (ignored by gitignore file).
Skipped 597 file(s) in commit (ignored by gitignore file).
https://huggingface.co/Wauplin/test_tmp_upload/tree/main/.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[LysandreJik]

Yes! And I agree that this is a bugfix.

The implementation looks good to me.

[Wauplin]

Thanks for the review @LysandreJik! Failing tests are unrelated so I'll merge this one and fix the CI in a separate PR.

[julien-c]

sorry i'm late to this, but are you sure we even needed a respect_gitignore param? Feels unlikely that anyone will ever want to set it to False no?

[Wauplin]

My feeling was that in some cases users still want to force upload a file. But yeah, maybe unlikely so I'd be fine with removing it (it has not been released yet so should be ok).

[Wauplin]

Opened a PR to remove respect_gitignore before it's too late 😄 #1876