Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempting to clone pypy repo failed for me. #62

Open
kcr opened this issue Mar 21, 2013 · 15 comments
Open

Attempting to clone pypy repo failed for me. #62

kcr opened this issue Mar 21, 2013 · 15 comments
Labels

Comments

@kcr
Copy link

kcr commented Mar 21, 2013

...
progress revision 61099 'master' (61100/62605)
fatal: Empty path component found in input
fast-import: dumping crash report to /home/kcr/src/pypy/.git/fast_import_crash_8317
fatal: Error while running fast-import
kcr@ordinator$ Traceback (most recent call last):
File "/usr/bin/git-remote-gitifyhg", line 9, in
load_entry_point('gitifyhg==0.8', 'console_scripts', 'git-remote-gitifyhg')()
File "/usr/lib/python2.7/dist-packages/gitifyhg/gitifyhg.py", line 262, in main
HGRemote(*[x.decode('utf-8') for x in sys.argv[1:3]]).process()
File "/usr/lib/python2.7/dist-packages/gitifyhg/gitifyhg.py", line 172, in process
getattr(self, 'do_%s' % command)(parser)
File "/usr/lib/python2.7/dist-packages/gitifyhg/gitifyhg.py", line 252, in do_import
HGImporter(self, parser).process()
File "/usr/lib/python2.7/dist-packages/gitifyhg/hgimporter.py", line 87, in process
self.hgremote.headnode[1])
File "/usr/lib/python2.7/dist-packages/gitifyhg/hgimporter.py", line 209, in process_ref
output(data)
File "/usr/lib/python2.7/dist-packages/gitifyhg/util.py", line 40, in output
print >> actual_stdout, msg
IOError: [Errno 32] Broken pipe

command was

git clone gitifyhg::ssh://hg@bitbucket.org/pypy/pypy

gitifyhg from the 0.8 tag, git version 1.7.10.4, hg 2.2.2

@fingolfin
Copy link
Collaborator

Thanks for the report. This used to work, so it looks like a regression. In any case, we definitely should fix this.

@felipec
Copy link
Contributor

felipec commented Apr 5, 2013

Works fine in remote-hg:

felipec/git@a160978

@PaulPrice
Copy link
Contributor

After observing that #69 did not fix this, and that the master of git-remote-hg doesn't work either, I noticed the commit felipec/git@a160978 is intended to fix this problem. I've adapted that commit to gitifyhg. Thanks, Felipe!

@dusty-phillips
Copy link
Owner

One thing I don't like about github is how commenting on commits and pull requests are separated. I meant to put this here:

Hi Paul,

You're doing a wonderful job of making me feel productive! ;) I thought I was going to do this on Sunday but it didn't happen.

I'm merging this patch as is but I'd like to point out that we should not have to call os.path.relpath here, because path is a path.py object which has a relpath method. However, path.path.relpath() does not accept the 'start' parameter, so we can't use it. I've filed jaraco/path#20 on path.py for this.

Your next pull request should be an update to add your name to the Readme as a contributor. Though, if you keep it up you'll have to move it to maintainers pretty quickly. :-)

I think we'll be calling 0.8.2 the Paul Price release!

dusty-phillips pushed a commit that referenced this issue Apr 15, 2013
Based on felipec/git@a160978, with thanks to Felipe Contreras.
@PaulPrice
Copy link
Contributor

I use gitifyhg for my work (to avoid using Mercurial), so I'm happy to be able to contribute to improving it.
I'm looking forward to 0.8.2, but please let's not call it anything silly.

OK, my attempt to clone pypy just now failed (but it got further than before). Will paste the traceback from my desktop....

@PaulPrice
Copy link
Contributor

progress revision 62999 'master' (63000/63382)
progress revision 63099 'master' (63100/63382)
progress revision 63199 'master' (63200/63382)
progress revision 63299 'master' (63300/63382)
WARNING: "Branch 'dist' has more than one head, consider merging"
[...]
WARNING: "Branch 'unicode-objspace' has more than one head, consider merging"
WARNING: "Branch 'reflex-support' has more than one head, consider merging"
error: there are still refs under 'refs/hg/origin/branches/stdlib-unification'
error: Unable to lock refs/hg/origin/branches/stdlib-unification
fatal: Error while running fast-import
price@neverland:~/test $ Traceback (most recent call last):
  File "/home/price/Software/gitifyhg/bin/git-remote-testgitifyhg", line 19, in <module>
    gitifyhg.main()
  File "/home/price/Software/gitifyhg/gitifyhg/gitifyhg.py", line 321, in main
    HGRemote(*[x.decode('utf-8') for x in args]).process()
  File "/home/price/Software/gitifyhg/gitifyhg/gitifyhg.py", line 197, in process
    self.marks.store()
  File "/home/price/Software/gitifyhg/gitifyhg/util.py", line 197, in store
    with self.storage_path.open('w') as file:
  File "/home/price/local/Linux.x86_64/lib/python2.7/site-packages/path.py-3.0.1-py2.7.egg/path.py", line 589, in open
    return open(self, mode)
IOError: [Errno 2] No such file or directory: path(u'/home/price/test/pypy/.git/hg/bc0c25890da4851631fef4c9786192c2b5377b6d/marks-hg')

@dusty-phillips
Copy link
Owner

Ugh, wouldn't it be nice to be able to test this without waiting half an hour for pypy to clone? At least it's a good repo for uncovering problems.

The IOError is a red herring; that's showing up because git has deleted the failed repo by the time gitifyhg tries to update the marks. Really, marks should only be updated if the command was successful.

The unable to lock error is a new one to me and a mystery. Runing with DEBUG_GITIFYHG may yield some more insight. There's clearly something funny going on with the git-remote protocol. However, cloning pypy with debugging output enabled may take a day to complete. It would be ideal to come up with a unit test that reproduces the issue.

@PaulPrice
Copy link
Contributor

Started the run with DEBUG_GITIFYHG; will check it in the morning. (:

@PaulPrice
Copy link
Contributor

The output is 11GB! I'm not going to post it all here. (:
But here's a sample.

[...]
WARNING: "Branch 'stdlib-unification/py3k' has more than one head, consider merging"
DEBUG: 'OUT: ? refs/heads/branches/stdlib-unification/py3k'
[...]
DEBUG: 'OUT: ? refs/heads/branches/stdlib-unification'
[...]
DEBUG: 'INPUT: import refs/tags/pypy-2.0-beta2'
DEBUG: 'OUT: reset refs/hg/origin/tags/pypy-2.0-beta2'
DEBUG: 'OUT: from :63129'
DEBUG: 'OUT: '
DEBUG: 'INPUT: '
DEBUG: 'OUT: done'
error: there are still refs under 'refs/hg/origin/branches/stdlib-unification'
error: Unable to lock refs/hg/origin/branches/stdlib-unification
fatal: Error while running fast-import
DEBUG: 'INPUT: '
Traceback (most recent call last):
  File "/home/price/Software/gitifyhg/bin/git-remote-testgitifyhg", line 19, in <module>
    gitifyhg.main()
  File "/home/price/Software/gitifyhg/gitifyhg/gitifyhg.py", line 321, in main
    HGRemote(*[x.decode('utf-8') for x in args]).process()
  File "/home/price/Software/gitifyhg/gitifyhg/gitifyhg.py", line 197, in process
    self.marks.store()
  File "/home/price/Software/gitifyhg/gitifyhg/util.py", line 197, in store
    with self.storage_path.open('w') as file:
  File "/home/price/local/Linux.x86_64/lib/python2.7/site-packages/path.py-3.0.1-py2.7.egg/path.py", line 589, in open
    return open(self, mode)
IOError: [Errno 2] No such file or directory: path(u'/home/price/test/pypy/.git/hg/c58fbfe850579675ea5d00a2a177d7b514440104/marks-hg')

It seems to me that this is likely related to the existence of a branch stdlib-unification/py3k in addition to branch stdlib-unification.

@dusty-phillips
Copy link
Owner

You're probably right, I think there's a couple evil lines in gitifyhg that split on '/'. The best way to test this would be to write a unit test with branches that share a prefix and have a '/'. No sense cloning pypy with 11 GB of output (!!!!) every time ;-)

@jedbrown
Copy link
Collaborator

It seems to me that this is likely related to the existence of a branch stdlib-unification/py3k in addition to branch stdlib-unification.

Yes, git does not allow this.

$ git branch x
$ git branch x/y
error: unable to resolve reference refs/heads/x/y: Not a directory
fatal: Failed to lock ref for update: Not a directory

Git branch names are like directory names, and literally exist in directories until they are written to the pack file.

@dusty-phillips
Copy link
Owner

Bummer. That disproves my comments on #75 I'd still like to find a neater solution for this than replacing / with SLASH, though. any ideas? Even if we only replaced it in the case where a common prefix is already known to exists, but the thought of maintaining that scares me.

So in git, having this naming structure is legal:

feature/hello
feature/world

but having this structure is not legal

feature
feature/hello

Is there some way we can do substitution only in the second case when we know that an error is going to occur? I'm thinking not, because if feature/hello exists and feature is added later, we have a different problem than if feature exists first and feature/hello is added later.

Even using a slightly less ugly substitution than SLASH would make me feel more content, but I don't know what to suggest. SLASH is at least explicit and not likely to conflict with other apis.

@felipec
Copy link
Contributor

felipec commented Apr 16, 2013

This was pointed out by Junio http://article.gmane.org/gmane.comp.version-control.git/220034. Which shows why patch review by git developers is valuable.

dusty-phillips pushed a commit that referenced this issue Apr 30, 2013
Fixes #80.

This is a controversial fix in some ways; an earlier commit explicitly turned on cloning of closed branches. However, I feel that the default should be to not clone closed branches. There are a few reasons for this.

The main one is that a closed branch in mercurial is generally considered the equivalent of a deleted branch in git. Therefore, we do not want the deleted branch in git.

A second one is that cloning closed branches can take a great deal of time that is, for the above reason, completely wasted.

A third one is that we have a few problems with branch naming (eg #78, #75, #62) that are caused by conflicting branched names. This fix does not solve those issues, but when the conflicting branches are closed branches, it does prevent them from causing a clone to fail outright.
@dusty-phillips
Copy link
Owner

I haven't tested it on pypy yet, but I suspect the workaround for #80 may address this issue. It depends if either of the conflicting branches is a closed branch in hg or not.

@PaulPrice
Copy link
Contributor

I believe it works in remote-hg because it doesn't clone the closed branches, so I expect gitifyhg would work now too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants