MIME boundary and headers from an API call leak to file contents

[ion1]

Whenever you add a file of 0x2f3ff bytes via the daemon API, the file that happens to be added after it gets merged to it with a MIME multipart boundary and headers belonging to the API request in between.

@eminence reported that this only happens with the binary from gobuilder.me but not with one he built, and the binary for the daemon is relevant, the binary for the client is not.

% mkdir test && perl -we 'print "a" x 0x2f3ff' >test/a && printf 'hello\n' >test/b && ipfs add -r test && rm -fr test                           
added QmbUP8iiockT1s5cBYc5BhzZo6w8DCj9Amj37jfnGhovFx test/a
added QmTEmiuQEQKMPDCFXiqtdMcRFvmbJHftBuYL6WbAotNVAQ test

% ipfs cat QmbUP8iiockT1s5cBYc5BhzZo6w8DCj9Amj37jfnGhovFx | tail -c 300
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
--688cb596c381f436dae2e51d05362bf8e27f08225f70cb0712bfc1b7e0b8
Content-Disposition: file; filename="test%2Fb"
Content-Type: application/octet-stream

hello

(The hash of the file changes at every add because of the random MIME boundary seen above.)

If there is no second file, the daemon gives the following error message (might this be related to #1688?):

% perl -we 'print "a" x 0x2f3ff' | ipfs add                               
ERRO[23:35:00:000] multipart: unexpected line in Next(): "70381cff0868072b9f870d992607c921ce5b1b76f0f5060a3edbe4b2d668--\r\n"  module=commands/http
added Qma4vEvx1iWwmsv5ANXEAMKPyWjMdGVPc33sGCqAzn27Ge Qma4vEvx1iWwmsv5ANXEAMKPyWjMdGVPc33sGCqAzn27Ge

[spikebike]

Pretty sure I'm using the ipfs ubuntu binaries on a currently patched ubuntu-14.04 system.

$ sha256sum ~/pkg/go/bin/ipfs
b32bd6fb80898c65eba93d19062cedcaad045e582aee75541891ae25e5a7bd91
$ ls -al ~/pkg/go/bin/ipfs
24589104 Sep 8 17:51 ~/pkg/go/bin/ipfs

I see the same problem. Note the different object IDs:
$ ipfs add -r test | head -1
added QmdvsA1eYRxaoiASqZq41XD5D2h1jEawrKUix1H21z1JA8 test/a
$ ipfs add test/a
added Qma4vEvx1iWwmsv5ANXEAMKPyWjMdGVPc33sGCqAzn27Ge a

I added a test script test-issue-1722.sh to ipfs with ID QmTEqWS7ttg7V1hsvX9VQBs8wGA3pWmkgovH8S6WsuW9sr

[jbenet]

Versions? 

—
Sent from Mailbox

On Thu, Sep 17, 2015 at 5:07 PM, Bill Broadley notifications@github.com
wrote:

Pretty sure I'm using the ipfs ubuntu binaries on a currently patched ubuntu-14.04 system.
$ sha256sum ~/pkg/go/bin/ipfs
b32bd6fb80898c65eba93d19062cedcaad045e582aee75541891ae25e5a7bd91
$ ls -al ~/pkg/go/bin/ipfs
24589104 Sep 8 17:51 ~/pkg/go/bin/ipfs
I see the same problem. Note the different object IDs:
$ ipfs add -r test | head -1
added QmdvsA1eYRxaoiASqZq41XD5D2h1jEawrKUix1H21z1JA8 test/a
$ ipfs add test/a

added Qma4vEvx1iWwmsv5ANXEAMKPyWjMdGVPc33sGCqAzn27Ge a

Reply to this email directly or view it on GitHub:
#1722 (comment)

[eminence]

The working version of IPFS identifies itself as ipfs version 0.3.8-dev and was built using the following command: go get -u github.com/ipfs/go-ipfs/cmd/ipfs

The non-working version of IPFS identifies itself as ipfs version 0.3.8-dev and was downloaded from https://gobuilder.me/get/github.com/ipfs/go-ipfs/cmd/ipfs/ipfs_master_linux-amd64.zip

[spikebike]

$ ipfs version
ipfs version 0.3.8-dev

[ion1]

It’s not just gobuilder, go-ipfs 0.3.8 I built with go get … also exhibits the bug.

[whyrusleeping]

This has been resolved, it was a bug in the multipart handling in the go http lib.