feature: git clone specific tag instead of clone and checkout

[hmaarrfk]

I'm trying to use boa to speed up build times for pytorch in hopes of gaining a few minutes on CIs.

I'm hoping that I can save time cloning a gigantic repository.

However, git clone and checkout fail for shallow clones.

I feel like git should be able to clone a single branch or tag with the command

git clone --branch GIT_REV --depth 1 URL

however, it seems that the checkout happens after the clone.

Is there a way to run them both at the same time?

Cloning a specific branch: 18MB

git clone --branch v1.9.0 --depth 1 git@github.com:pytorch/pytorch.git
Cloning into 'pytorch'...
remote: Enumerating objects: 9310, done.
remote: Counting objects: 100% (9310/9310), done.
remote: Compressing objects: 100% (8275/8275), done.
Receiving objects: 100% (9310/9310), 18.08 MiB | 9.03 MiB/s, done.
remote: Total 9310 (delta 1322), reused 2845 (delta 828), pack-reused 0
Resolving deltas: 100% (1322/1322), done.
Note: switching to 'd69c22dd61a2f006dcfe1e3ea8468a3ecaf931aa'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Cloning full tree: 500MB

$ git clone git@github.com:pytorch/pytorch.git
Cloning into 'pytorch'...
remote: Enumerating objects: 618076, done.
remote: Counting objects: 100% (6125/6125), done.
remote: Compressing objects: 100% (2169/2169), done.
remote: Total 618076 (delta 4619), reused 5224 (delta 3947), pack-reused 611951
Receiving objects: 100% (618076/618076), 507.99 MiB | 41.92 MiB/s, done.
Resolving deltas: 100% (500231/500231), done.

[hmaarrfk]

This seems to be a design choice from conda-build https://github.com/conda/conda-build/blob/master/conda_build/source.py#L236

[wolfv]

maybe you could use the zip download if it's on github?

E.g. https://github.com/pytorch/pytorch/archive/refs/heads/1.7.zip

[wolfv]

And yeah, we just use conda-build for fetching the source so I'd also have to go look into conda-build docs / or the conda-build code to figure this out :)

[rgommers]

maybe you could use the zip download if it's on github?

Those are the GitHub-generated archives for a tag, not proper sdists. I.e., the git submodules will be missing. So that won't work.

Fixing in conda-build sounds good, I can't see a reason this would not work as an opt-in option.

[wolfv]

I'd be happy with censoring the code from conda-build for checking out source code in boa. It might get us a higher dev velocity and we can push changes upstream later on

[hmaarrfk]

understood. I figured I would post it upstream as well.

Good to know that this is useful for boa.

[wolfv]

Sorry, not censoring -- vendoring is what I meant to type. :)