Multipart Error: unexpected line in Next(), IPFS not reading everything from Part.

[travisperson]

Found on dev0.4.0 5bf3a59

There are a few issues open that talk about this multipart, but none that I can see address (this might be slightly related to #1688).

Here is a curl request that sometimes fails:

curl 'http://localhost:5001/api/v0/add?stream-channels=true' \
    -H 'content-type: multipart/form-data; boundary=4yshodsbhhestt9i1h8aa2r5g4rhpvie9x3smmppagiudikr5a7lpes7kchaor' \
    -H 'Connection: keep-alive' \
    --data-binary $'--4yshodsbhhestt9i1h8aa2r5g4rhpvie9x3smmppagiudikr5a7lpes7kchaor\r\nContent-Disposition: file; filename="hello.txt"\r\nContent-Type: application/octet-stream\r\n\r\nworld\r\n--4yshodsbhhestt9i1h8aa2r5g4rhpvie9x3smmppagiudikr5a7lpes7kchaor\r\nContent-Disposition: file; filename="ipfs.txt"\r\nContent-Type: application/octet-stream\r\n\r\nis cool\r\n--4yshodsbhhestt9i1h8aa2r5g4rhpvie9x3smmppagiudikr5a7lpes7kchaor\r\nContent-Disposition: file; filename="readme.md"\r\nContent-Type: application/octet-stream\r\n\r\nwow\r\n--4yshodsbhhestt9i1h8aa2r5g4rhpvie9x3smmppagiudikr5a7lpes7kchaor--'

Most commonly this will fail with:
ERROR commands/h: err: multipart: unexpected line in Next(): "wow\r\n" handler.go:265

Results:

{
  "Name": "hello.txt",
  "Hash": "QmNyJpQkU1cEkBwMDhDNFstr42q55mqG5GE5Mgwug4xyGk"
}
{
  "Name": "ipfs.txt",
  "Hash": "QmfBfWxn32sGXbkgDoeNLT5rBcTehYffMyeU5LFTTByt6V"
}
{
  "Name": "readme.md",
  "Hash": "QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH"
}

But I've also seen it fail on
ERROR commands/h: err: multipart: unexpected line in Next(): "is cool\r\n" handler.go:265

Results:

{
  "Name": "hello.txt",
  "Hash": "QmNyJpQkU1cEkBwMDhDNFstr42q55mqG5GE5Mgwug4xyGk"
}
{
  "Name": "ipfs.txt",
  "Hash": "QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH"
}

The basic issue here is someplace IPFS is not reading everything from the part returned by NextPart. This results in the next section of the multipart not being a boundary and, as the error says, we find an unexpected NextPart.

You can see this by inspecting both of the last Hashs returned in the failures above. Both have a hash of QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH

Which is

{
  "Links": [],
  "Data": "\u0008\u0002\u0018\u0000"
}

Sometimes I see:
ERROR commands/h: err: multipart: NextPart: http: invalid Read on closed Body handler.go:265

A successful run (no errors) produces:

{
  "Name": "hello.txt",
  "Hash": "QmNyJpQkU1cEkBwMDhDNFstr42q55mqG5GE5Mgwug4xyGk"
}
{
  "Name": "ipfs.txt",
  "Hash": "QmfBfWxn32sGXbkgDoeNLT5rBcTehYffMyeU5LFTTByt6V"
}
{
  "Name": "readme.md",
  "Hash": "QmbxMp5x7RbqHSTrrKQCYjccw2obvgcz2GEyz3gEkixhDS"
}

[whyrusleeping]

What version of go were you using? I just remembered that the bug was fixed by upgrading to go 1.5.2

[travisperson]

Compiled ipfs with go version go1.5.2 linux/amd64 (pulled down go 1.5.2 from golang.org/dl/ last night).

[whyrusleeping]

cool, I'll take a look at this.

[whyrusleeping]

I can reproduce this on 0.4.1 and 0.4.2, but not on 0.4.3-rc3, so either:

• it was fixed in 0.4.3

or

• it was fixed in whatever go versions have been released since we built 0.4.2 for distributions

[whyrusleeping]

Alright, gonna assume we fixed it. Tried building 0.4.3-rc3 with go1.6 (which is what ipfs 0.4.2 was built with) and could not reproduce the issue.

[aniket-kumar]

@whyrusleeping facing same issue in go-ipfs_v0.4.3_linux-amd64. Any help will be appreciated.

[Kubuxu]

Please update to newer version, 0.4.3 is ancient.