Add retry logic to file.download because it fails with 504 often

[surjikal]

Is your feature request related to a problem? Please describe.

My file.download calls often fail with 504 gatewayTimeout.
Guesstimating that this happens 1-5% of the time, over 1000s of calls per day.

Describe the solution you'd like

It would be nice if file.download had the same kind of retry handling that file.save has.

Describe alternatives you've considered

I have implemented my own retry logic on top of this library, and it's similar to how the retry logic is implemented for file.save.

---

Here's the relevant part of the error object I'm seeing:

 ApiError {
  code: 504,
  errors: [ { message: '', domain: 'global', reason: 'gatewayTimeout' } ],
  response: PassThrough {
    _readableState: ReadableState {
      objectMode: false,
...

[surjikal]

FYI we're recently seeing 504s in our python gcs calls as well (using an old version of the client lib). Saw it on list blobs, upload and read, couple times per week.

I'm now just wrapping every gcs call with my own retry mechanism, both on node & python.

[danielbankhead]

Hey @surjikal, which version of the library are you using? file#download should be automatically retried in recent releases

[surjikal]

@danielbankhead We are using 5.14.3. It was the latest version when I opened this request.

We're not doing anything too fancy, something like this:

const storage = new Storage()
const data = await storage.bucket('foo').download('bar.txt')

We've always seen a small number of issues in prod, but anecdotally it seems a lot worse under v5 / within the past 2-3 months.

We added a retry wrapper to every single gcs call (before, we only had a wrapper forcreateWritestream({resumable:false})).

We had missed one for download, and started failing a lot with that 504 error, so we opened this feature request.