CORS issue when posting to URI produced by Google.Cloud.Storage.V1.StorageClient.CreateObjectUploader in browser

[shawnshaddock]

@shawnshaddock: Everyone on the team is now on vacation until January, but if you could provide all the parts for me to reproduce this when we're back at work, I'll definitely have a look. Please open a new issue for this rather than using this one, just to keep things clearer.

Originally posted by @jskeet in #2755 (comment)

My scenario is an Angular 10 client and .NET 5 (ASP .NET) API. The angular client calls the API to get a Cloud Storage resumable upload link. The API uses Google.Cloud.Storage.V1.StorageClient.CreateObjectUploader to initiate the upload. I also set UploadObjectOptions.Orgin to "*" in the request.

The API then returns the URI back to the angular client, where it attempts to use angulars HttpClient to do a POST to the URL with the file to upload. This is when I get the error:

Access to XMLHttpRequest at 'https://storage.googleapis.com/upload/storage/v1/b/BUCKET_NAME_REMOVED/o?uploadType=resumable&upload_id=UPLOAD_ID_REMOVED' from origin 'http://localhost:4200' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

[shawnshaddock]

Here is relevant client and server code

Angular client code

const result = await this.storageObjectService.startUpload({
  bucket: 'BUCKET-NAME',
  name: file.name,
  contentType: 'application/pdf'
});

this.httpClient.post(result.uploadUri, file).subscribe(() => {
  console.log(`Uploaded ${file.name}`)
});

ASP.NET Core in .NET 5

var upload = StorageClient.CreateObjectUploader("BUCKET-NAME", "FILE-NAME", "application/pdf", Stream.Null, new UploadObjectOptions
{
    Origin = "*"
});

var uploadUri = await upload.InitiateSessionAsync(cancellationToken);
return uploadUri;

This is a sample of the URI that is returned
https://storage.googleapis.com/upload/storage/v1/b/BUCKET-NAME/o?uploadType=resumable&upload_id=UPLOAD-ID

[jskeet]

Thanks - will look at this when I get a chance. It's just possible it'll be before January, but probably not I'm afraid.

[shawnshaddock]

@jskeet does Google.Cloud.Storage.V1.StorageClient use the Cloud Storage JSON API?

[jskeet]

Yes.

[shawnshaddock]

I should point out that even though the browser is throwing a CORS error the files do get successfully uploaded into the bucket. This is still an issue though because it prevents me from getting the JSON response from the storage api and also prevents me from being able to handle actual upload errors.

[jskeet]

Just as an FYI, I'm now back to work, and trying to get my head round this. I've hardly done anything with CORS, so I don't yet understand all the details... I'll need to plod through it.

Out of interest, are you able to share what the browser request and response look like in terms of headers etc? I don't know enough about what I'm looking at to say whether that's going to be relevant, but it may well be :)

[jskeet]

Okay, I've had a look now... and with Origin="*" there's no Access-Control-Allow-Origin header in the response to the second request (the one that actually performs the upload)... but if you specify an actual origin, that's then returned in the response, which I believe would make the browser happy.

In your application, are you able to specify a particular origin in the UploadObjectOptions, or do you genuinely need it to be *?

[shawnshaddock]

I can provide an actual origin, I was using * as a desperate attempt to fix the issue and also so that I didn't have to manage different origins in development/testing/production. But if that's how to solve it I can do it.

[jskeet]

Right - I don't know whether there is any other way of doing it, but I believe that approach will work, and hopefully isn't too hard.

[shawnshaddock]

I will try it and report back my results. I'm busy with something else ATM so I don't think it will be today, but hopefully sometime this week.

[jskeet]

@shawnshaddock: Great, thanks. I'll leave the issue open for the moment, in the hope that we'll be able to close it when you've been able to try it.

[doubbz]

hi here,

We have the same problem on our project :

• POST request initiating upload (with url for resumable upload in response header Location) OK
• next PUT request is KO because there is no cors header in the response even though the file has been successfully uploaded to the bucket

Fyi CORS configuration on the bucket is ok and matches the origin header for our requests.

Setting the origin header to something particular rather than * does not fix the issue on our side

I hope we can find a solution for this problem together

[jskeet]

@doubbz: I initially thought you were describing a somewhat different issue, but I'm now not sure. Could you give some more details of your situation and show the code you're using?

[doubbz]

@jskeet sure and thank you for your answer. I should maybe open a new issue because, here's the thing, we are working with a ReactJS frontend client and a PHP backend.

IMO the problem might be the same because the scenario is identical as shawnshaddock's one. My js client needs an auth signature in order to perform the first POST request initiating the resumable upload session.

Scenario:
1/ JS client requests a Google JSON API Storage url with the auth signature to my php backend

• the Google PHP client is signing the url with an origin header matching the host of the JS Client and the CORS configuration on the bucket (on which "level access" has been set to uniform )

$bucket->object($path)->signedUploadUrl(new \DateTimeImmutable('+100 minute'), ['Origin' => <host of my JS client>])

2/ JS client performs the "initiating POST request" to the Google JSON API successfully

• Origin header is included in the request (not *)
• responses for both preflight and actual request are OK and includes access-controll-allow-origin - everything's fine, happy browser
• JS client saves the url in the Location header for the next request

3/ JS performs a PUT request to the Google JSON API with the file to upload

• an origin header is also included in the request (with the same value as before)
• Response is surprisingly ok for the preflight request which allows the browser to perform the actual request
• the actual request is performed (even the file is uploaded and available on the bucket) but the response does not include Access-controll-* headers. Since access-controll-allow-origin header is missing, the browser is not happy

I could open an issue on the google-cloud-php repo instead but people is already starting to be impatient around me 😅 asking me to bypass all this and PUT the files from the backend (Pod Kubernetes with low memory) instead :/ 🤷‍♂️

[jskeet]

@doubbz: I don't believe the URL in the original question is a signed one. If it were, I'd expect it to use UrlSigner.

Given that .NET isn't involved at all in your scenario, I think it would be better to either ask in one of the language-neutral Storage support routes or the google-cloud-php repo... as I don't work in either Javascript or PHP, it's going to be hard for me to reproduce the issue (without basically translating that code to .NET, where the issue may not occur). If you were really doing the same requests as in the rest of this issue, it wouldn't be a problem - but the URL signing is a potentially-significant difference here.

[doubbz]

@jskeet I think you're right, it may not be the same use case because of this first step for the signature.

Thank you anyway

[shawnshaddock]

@shawnshaddock: Great, thanks. I'll leave the issue open for the moment, in the hope that we'll be able to close it when you've been able to try it.

I tested setting a specific Origin in the UploadObjectOptions and it does indeed add the access-control-allow-origin header and works without error.

[jskeet]

Great - thanks for getting back to us.