IPFS add -r hang (many files, progress bar disappears)

[sroerick]

Using FreeBSD:

I was experimenting with adding a large number of files to IPFS, and found some errors.

Some specific files caused add -r to hang. By hanging, I mean that the progress bar disappeared, there was no indication of what file was presently transferring, and there did not appear to be any disk activity or significant cpu activity.

I don't know that there was any specific quality to the files. I could solve the issue by removing the file, or by adding the file by itself using "ipfs add". Once the file had been hashed on its own, "ipfs add -r" resumed working.

A traceback of the ipfs daemon hanging can be found here: http://sportsteam.link/~roerick/goreport2.txt

More info on IRC here:
https://botbot.me/freenode/ipfs/2015-06-21/?msg=42494305&page=3

[jbenet]

@sroerick progress bar disappearing? that's a new one. can you repro it reliably?

hang in general sounds related to the "Metadata" problem (i.e. make our Metadata object that addresses symlinks, fifos, etc)).

cc @whyrusleeping

[rtlong]

I think I may be seeing this as well. I've been noticing it with a build I've been using for about a month, but just the hang symptom, progress bar was still visible. When the daemon is running, it would hang, when I stop the daemon, I can add the whole directory without issue.

Today, I've built from master@8668f40915829d78916b94ed9bcfb1ff53fb0cb9 and I see this as described. I'm attempting to add the entire go-ipfs repo at that same commit. With the daemon running, it hangs at

added QmfUTFdG8NS62w6cijrCk4zF4uGcsE912n9uv9hLz8AYQh Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash

If I kill the daemon and retry, I see it error on one of the next items

added QmPSUxktbw5MHBMLZ6xeNCNoTeT3q3UPC7fY7p7pKZCnSr Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash/install.dist.sh
added QmTPrmfCgCoYzY1zAicV5rbQyvH5epLJsxtM5rnMQef23Q Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash/main.go
added QmfUTFdG8NS62w6cijrCk4zF4uGcsE912n9uv9hLz8AYQh Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash
added QmSDVUKYpYULR84uSMzUEWHSEhMgCg3dkHovrxM9XoLoZM Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash.go
added QmNTfMrYWYzT2cYALdJuZuYJzX1VF9AcPtfTE11hAUXVXL Godeps/_workspace/src/github.com/jbenet/go-multihash/multihash_test.go
added QmeJuigWUjn78w7d6ahgtxCx9VGADg2USaNX42rG8m2Qht Godeps/_workspace/src/github.com/jbenet/go-multihash/opts/README.md
added QmWX9xWtnFVbc1CdyPCjPL6AX

Full output and stack trace from command which halted before I SIGQUIT it: https://gist.github.com/rtlong/07c7d7f9c0f4c02f0100

[jbenet]

@whyrusleeping this is a really bad UX bug. we should fix this asap

[whyrusleeping]

marking as fixed. Reopen if you can reproduce (this may not be entirely caused by the symlink issue, but its likely)

[jbenet]

I believe @davidar or @rht reproed it without symlinks.

[rht]

@davidar ipfs/team-mgmt#29 (comment)
(Also I was able to simulate an err in httpclient that makes ipfs add hangs like in the pre-symlink era where it should have returned err)

The non-symlink part of this issue is a duplicate of #1072

[rht]

To clarify, there are 2 types of bugs here remaining:

1. ipfs add breaks ipfs add breaks somewhere in the middle of upload #1072. To reproduce Sprint Aug 24 team-mgmt#29 (comment)
2. ipfs add hangs (previously symlink&namedpipe but now namedpipe) IPFS add -r returns a "no such file or directory" error.  #1409. To reproduce, just ipfs add a namedpipe, or it is like adding return 0, nil here https://github.com/ipfs/go-ipfs/blob/master/commands/http/multifilereader.go#L103