impossible to upload files to google storage with v0.12.0

[aschmid]

while 0.11.0 works completely fine i am experiencing huge issues uploading files with the latest (0.12.0) version.

a simple file upload takes forever and might even miss a response.

fs.createReadStream('/local/file.txt').pipe(bucket.file('file.txt').createWriteStream());

not sure what changed between the versions but it broke something.

[ryanseys]

Sorry to hear that! Could you try against the latest code in master. We haven't pushed a new version to npm in a while, so you might have better luck with the most up to date code. Let me know how it goes!

[aschmid]

master or 0.12.0 behave the same.
in my package.json i tried:

"gcloud": "0.12.0"

and

"gcloud": "*"

and both have the same issue.

[stephenplusplus]

The * will just pull down the latest tag version which is 0.12.0. To get master, do npm install --save googlecloudplatform/gcloud-node.

[aschmid]

ok so tried again with the latest master branch and it still has the same issue.
tomorrow i might write a package to prove the case.
this is one of the errors i get after a while waiting for the files to be uploaded.

Debug: internal, implementation, error 
    Error: socket hang up
  at SecurePair.error (tls.js:1011:23)
  at EncryptedStream.CryptoStream._done (tls.js:703:22)
  at CleartextStream.read [as _read] (tls.js:499:24)
  at CleartextStream.Readable.read (_stream_readable.js:341:10)
  at EncryptedStream.onCryptoStreamFinish (tls.js:304:47)
  at EncryptedStream.g (events.js:180:16)
  at EncryptedStream.emit (events.js:117:20)
  at finishMaybe (_stream_writable.js:360:12)
  at endWritable (_stream_writable.js:367:3)
  at EncryptedStream.Writable.end (_stream_writable.js:345:5)
  at EncryptedStream.CryptoStream.end (tls.js:641:31)
  at Socket.onend (_stream_readable.js:502:10)
  at Socket.g (events.js:180:16)
  at Socket.emit (events.js:117:20)
  at _stream_readable.js:944:16
  at process._tickDomainCallback (node.js:492:13)

this also:

Debug: internal, implementation, error 
    Error: read ECONNRESET
  at errnoException (net.js:905:11)
  at TCP.onread (net.js:559:19)

[stephenplusplus]

It would be great to see a small sample of code. I'm not sure how this would work in 0.11 and not now, but I'm determined to find the cause!

[ryanseys]

What version of node are you running? The stack trace seems to hint at the fact that the error occurs outside the realms of our library.

[aschmid]

im using node v0.10.38

[aschmid]

ok here it is: https://github.com/aschmid/gcloud.storage.test.git

just copy your credentials into gcs_keyfile.json and insert a bucket name and projectid in the config.js object.
after an npm install start with ./bin/start

this version doesn't work for me, if i change the gcloud version to 0.11.0 in package.json it works as expected.

[stephenplusplus]

After a quick glance, it looks like you're using the end event. Try hooking
on to the complete event.

On Wednesday, April 1, 2015, andreas schmid notifications@github.com
wrote:

ok here it is: https://github.com/aschmid/gcloud.storage.test.git

just copy your credentials into gcs_keyfile.json and insert a bucket name
and projectid in the config.js object.
after an npm install start with ./bin/start

this version doesn't work for me, if i change the gcloud version to 0.11.0
in package.json it works as expected.

—
Reply to this email directly or view it on GitHub
#470 (comment)
.

[aschmid]

i tried the complete too but still same result

[aschmid]

btw the complete doesnt even work with 0.11.0
thats why i used the 'end'

[stephenplusplus]

No problem, I'll take a dive in to the code soon. Thanks for putting the
repo together!

On Wednesday, April 1, 2015, andreas schmid notifications@github.com
wrote:

btw the complete doesnt even work with 0.11.0
thats why i used the 'end'

—
Reply to this email directly or view it on GitHub
#470 (comment)
.

[stephenplusplus]

I changed end to complete and it uploaded the files successfully for me. I'm not sure how we can help, since it's working for us and in our system tests that are run on every change we make to the library.

[ryanseys]

0.12.0 is a thing of the past too :) we just recently launched 0.13.1. That being said, if you still have issues I'd suggest a clean install of node and ensure you're not running behind any proxy server or restrictive firewall that might get in the way of uploading!

[aschmid]

@ryanseys i had to reset my whole computer 2 days ago so my node environment is as clean as it could be :)
to be honeyst i tried 0.13.1 also and i still experience the same issue... it has to be something else.

[ryanseys]

By the looks of the error it's happening at the TLS level so maybe it can't properly negotiate a secure connection. That being said I have no idea why it would work on 0.11 and not 0.13. Funky bug.

[aschmid]

yes it is... in any case i was able to work around it but still very annoying.

[ryanseys]

Do you have a patch or is your workaround to just use 0.11?

[aschmid]

no don't have a patch yet. didn't have time to dig into the actual issue so i used 0.11 for the stuff i needed but obviously i would prefer to be up to date.

[aschmid]

would be interesting to know if it is just me or if other ppl have the same issue

[aschmid]

i just noticed bucket.upload works just fine and works through the 20 files within a few seconds.
the example code i supplied takes around 4-5 minutes ending up in an end event but no files are actually uploaded in the bucket?!
this is very interesting...

[stephenplusplus]

bucket.upload is a wrapper around createWriteStream, with some defaults set: https://github.com/GoogleCloudPlatform/gcloud-node/blob/master/lib/storage/bucket.js#L776

Maybe try to mimic that set up as close as you can.

[stephenplusplus]

@aschmid going to close as it doesn't seem to be an issue with the library, but something on your end. Feel free to re-open if you can replicate on another machine or provide a test case we can reproduce.

[aschmid]

ok good for now. did anybody of you guys try my code? does it work for you?
im strill struggling with this...

[aschmid]

ok guys i found the actual issue.

when creating a write stream for the file its absolutely necessary to pass in {resumable: <boolean>} like so:

bucket.file('myfile.jpg').pipe(newFile.createWriteStream({resumable: false}))

if this option is not passed in the complete event never gets called.
i will see if i can submit a working patch in the next few days

[ryanseys]

Any patch? :)