Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IPFS add -r hang (many files, progress bar disappears) #1408

Closed
sroerick opened this issue Jun 21, 2015 · 7 comments
Closed

IPFS add -r hang (many files, progress bar disappears) #1408

sroerick opened this issue Jun 21, 2015 · 7 comments
Assignees

Comments

@sroerick
Copy link
Contributor

Using FreeBSD:

I was experimenting with adding a large number of files to IPFS, and found some errors.

Some specific files caused add -r to hang. By hanging, I mean that the progress bar disappeared, there was no indication of what file was presently transferring, and there did not appear to be any disk activity or significant cpu activity.

I don't know that there was any specific quality to the files. I could solve the issue by removing the file, or by adding the file by itself using "ipfs add". Once the file had been hashed on its own, "ipfs add -r" resumed working.

A traceback of the ipfs daemon hanging can be found here: http://sportsteam.link/~roerick/goreport2.txt

More info on IRC here:
https://botbot.me/freenode/ipfs/2015-06-21/?msg=42494305&page=3

@jbenet
Copy link
Member

jbenet commented Jun 22, 2015

@sroerick progress bar disappearing? that's a new one. can you repro it reliably?

hang in general sounds related to the "Metadata" problem (i.e. make our Metadata object that addresses symlinks, fifos, etc)).

cc @whyrusleeping

@jbenet jbenet changed the title IPFS add -r hangs when adding many files. IPFS add -r hang (many files, progress bar disappears) Jun 22, 2015
@rtlong
Copy link

rtlong commented Jul 18, 2015

I think I may be seeing this as well. I've been noticing it with a build I've been using for about a month, but just the hang symptom, progress bar was still visible. When the daemon is running, it would hang, when I stop the daemon, I can add the whole directory without issue.

Today, I've built from master@8668f40915829d78916b94ed9bcfb1ff53fb0cb9 and I see this as described. I'm attempting to add the entire go-ipfs repo at that same commit. With the daemon running, it hangs at

added QmfUTFdG8NS62w6cijrCk4zF4uGcsE912n9uv9hLz8AYQh Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash

If I kill the daemon and retry, I see it error on one of the next items

added QmPSUxktbw5MHBMLZ6xeNCNoTeT3q3UPC7fY7p7pKZCnSr Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash/install.dist.sh
added QmTPrmfCgCoYzY1zAicV5rbQyvH5epLJsxtM5rnMQef23Q Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash/main.go
added QmfUTFdG8NS62w6cijrCk4zF4uGcsE912n9uv9hLz8AYQh Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash
added QmSDVUKYpYULR84uSMzUEWHSEhMgCg3dkHovrxM9XoLoZM Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash.go
added QmNTfMrYWYzT2cYALdJuZuYJzX1VF9AcPtfTE11hAUXVXL Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash_test.go
added QmeJuigWUjn78w7d6ahgtxCx9VGADg2USaNX42rG8m2Qht Godeps/_workspace/src/github.com/jbenet/go-multihash/opts/README.md
added QmWX9xWtnFVbc1CdyPCjPL6AX

Full output and stack trace from command which halted before I SIGQUIT it: https://gist.github.com/rtlong/07c7d7f9c0f4c02f0100

@jbenet
Copy link
Member

jbenet commented Jul 18, 2015

@whyrusleeping this is a really bad UX bug. we should fix this asap

@whyrusleeping
Copy link
Member

marking as fixed. Reopen if you can reproduce (this may not be entirely caused by the symlink issue, but its likely)

@jbenet
Copy link
Member

jbenet commented Sep 3, 2015

I believe @davidar or @rht reproed it without symlinks.

@rht
Copy link
Contributor

rht commented Sep 3, 2015

@davidar ipfs/team-mgmt#29 (comment)
(Also I was able to simulate an err in httpclient that makes ipfs add hangs like in the pre-symlink era where it should have returned err)

The non-symlink part of this issue is a duplicate of #1072

@rht
Copy link
Contributor

rht commented Sep 3, 2015

To clarify, there are 2 types of bugs here remaining:

  1. ipfs add breaks ipfs add breaks somewhere in the middle of upload #1072. To reproduce Sprint Aug 24 team-mgmt#29 (comment)
  2. ipfs add hangs (previously symlink&namedpipe but now namedpipe) IPFS add -r returns a "no such file or directory" error.  #1409. To reproduce, just ipfs add a namedpipe, or it is like adding return 0, nil here https://github.com/ipfs/go-ipfs/blob/master/commands/http/multifilereader.go#L103

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants