Add git worktree command [was: hg share -> git worktree]

[tiran]

I used hg share a lot to share a single hg clone between multiple working directories. It's convenient to have a lean copy for each branch of Python (2.7, 3.5, 3.6). git has a similar feature called worktree, https://git-scm.com/docs/git-worktree . It took me a bit to find it. Other developers may find the feature useful, too. What do you think about documenting git worktree in the dev guide?

$ git clone git@github.com:python/cpython.git
$ git remote rename origin upstream
$ git worktree add -b 2.7 ../2.7 upstream/2.7
$ git worktree add -b 3.5 ../3.5 upstream/3.5
$ git worktree add -b 3.6 ../3.6 upstream/3.6
$ git worktree list
/tmp/cpython  e7ffb99 [master]
/tmp/3.6      2a35c93 [3.6]
/tmp/3.5      9cd7e17 [3.5]
/tmp/2.7      5c32988 [2.7]

[methane]

Oh, I used git-new-workdir (in git's contrib directory).
git worktree seems better. Thank you.

[tiran]

https://github.com/blog/2042-git-2-5-including-multiple-worktrees-and-triangular-workflows#git-worktree-one-git-repository-with-multiple-working-trees

[ncoghlan]

+1 from me. I'd suggest looking up where we used to document hg share and adding a similar example for git worktree.

[warsaw]

+1 for documenting worktree, it's a great feature of git especially when you might be working in multiple branches often (e.g. in our case devel, 3.6, and 3.5).

Also, +1 for documenting triangular workflows. Maybe I should open a new issue on this but as I was reading the devguide I noticed that it suggested pushing to origin, i.e. the main repo. In our case since many people won't have permission to do that, I think it's better to recommend forking the main repo and pushing branches for pull requests to your own fork. Generally I still call the main remote 'origin' and my own remote 'warsaw', but the page @tiran references calls the main repo 'upstream' and your own fork 'origin', which probably makes for a better default recommendation. Anybody with git experience will know how to season to taste.

[tiran]

I have been using upstream and origin with the described semantic for years. It seems to be pretty standard and even the recommended way on github, too. https://help.github.com/articles/configuring-a-remote-for-a-fork/

I wonder, should core devs really push to the main repos? In all my projects (work and private) I always push to my private clone and then use PR to merge from my private copy to upstream.

[matrixise]

@tiran normally we should propose a PR

[warsaw]

On Feb 11, 2017, at 06:08 AM, Christian Heimes wrote: I wonder, should core devs really push to the main repos?

No, we should also push to our forks. That way, core devs more closely use the same workflow non-core devs use, plus it doesn't clutter up the main repo with additional branches (that often, but not always, get pruned after a merge). I personally push to the main repos for some of my projects, but it really depends on the size of the contributor community. For something as large and diverse as Python, I think everyone should push to their forks.

[brettcannon]

The only reason someone should end up with a personal branch on python/cpython is if they are fixing a spelling mistake in the browser and they will be deleting their branch in less than 24 hours. Otherwise everything else should go to a personal fork and get merged in through a PR.

And it sounds like people like the idea of documenting git worktree so I don't see why you shouldn't go for it, @tiran !

[zhangyangyu]

+1 on this. IIRC there was a similar part about hg in devguide before. It's disappointing to not see such a part which does help my workflow.

[tiran]

I might be able to squeeze in some time next week. If somebody else is going to beat me instead ... go ahead! I wouldn't mind.

[ezio-melotti]

+1 for me too.

A good place where to add this would be the tool setup section of the pullrequest page with a link to it in version control setup section of the setup page.

FWIW in #103 I also suggested to add more git commands to the pull request page, and this will be a good starting point.

[ezio-melotti]

Another good place to add this would be the remotes setup section of the committing page (even though I don't like that the info are currently scattered around a few different page, and I'm going to propose some refactoring in a separate issue).

[ncoghlan]

I just set this up myself, and it seems very nice. From my cpython dir, it was just a matter of doing:

git checkout 3.6
git worktree add ../py36 3.6
git checkout 3.5
git worktree add ../py35 3.5
git checkout 2.7
git worktree add ../py27 2.7
git checkout master

(I didn't check if you could use "origin/X.Y" directly as the source branches for the worktrees)

[serhiy-storchaka]

How to create the second worktree for the same branch (for debug build)? What is git equivalent of following hg commands?

hg share -U cpython cpython-debug
cd cpython-debug
hg update -C default

[methane]

Use git worktree add --force (or -f) or git worktree add --detach.
See man git-worktree for detail.

[tiran]

@ncoghlan -b 3.6 performs git checkout 3.6 in one go.

@serhiy-storchaka You can also use out-of-tree builds to build different flavors from the same checkout. I use ccache to speed up compilation, too.

[serhiy-storchaka]

Thank you @methane and @tiran.

[Mariatta]

@tiran if you're not working on this, can I unassign you and add help-wanted label instead?
Perhaps we can find a new contributor at the sprint to work on this issue :)

[terryjreedy]

For me, this is the most needed addition to the devguide.

[terryjreedy]

There are two versions above of how to set shares (worktrees) for 3.6, 3.5, 2.7. Equivalent? I presume that if I want to test a 3.6 backport, I cd into ../3.6 and do the same PR fetch into a new branch as I currently do in cpython. What is not completely clear to me is how to efficiently update multiple repositories, with one fetch from remote. This was easy with hg.

One hg-git difference seems to be that hg commands have a '--repository xyz' option to specify the local repository that is the target of the command. Git commands seems to lack this, always using the cwd as target. For multiple workspaces, this would mean multiple cd commands. Correct?

[terryjreedy]

@Mariatta , see Tiran's Feb 16 message (it said, in effect, 'yes').

[pfmoore]

@terryjreedy There's a -C option for git that's similar to hg's -R

[terryjreedy]

Thank you. By experimenting, I discovered that 'git help git' opens the local git.html man(1) page that both lists and explains the git options that go before specific subcommands. "-C : Run as if git was started in instead of the current working directory."

[Mariatta]

Ah I didn't see that 😅 Thanks @terryjreedy 😃

[terryjreedy]

git worktree add ../36 3.6
seems to work without first checking out 3.6 in the master repository. In fact, once 3.6 is checked out in the worktree directory, it is no longer possible to check it out in the original repository.
pcbuild/build.bat in the new directory builds 3.6.1+.

[willingc]

I'm going to close this issue since the move to git workflow is pretty dated at this point in time. For folks interested in the discussion, the closed issue still keeps that available through search.

[erlend-aasland]

I believe we should keep this open; adding info about git worktree can be useful for a lot of developers.

[erlend-aasland]

Also, please use "Close as not planned" when closing issues as wont-fix, duplicate, outdated, invalid, etc.

[willingc]

@erlend-aasland If we want to keep this open, let's scope the issue down to "Add info about git worktree". Mentioning mercurial really makes little sense within the current workflow.

I didn't know we had a "Close at not planned" label. Thanks for the tip.

[erlend-aasland]

@erlend-aasland If we want to keep this open, let's scope the issue down to "Add info about git worktree". Mentioning mercurial really makes little sense within the current workflow.

Quoting the OP:

What do you think about documenting git worktree in the dev guide?

AFAICS, that is the scope of this issue.

[merwok]

(Close as not planned is not a project label, but a recent github feature accessible in a dropdown in the close button)

[hugovk]

Please see PR #1255.