Skip to content

Commit

Permalink
multipart uploads: Hack to support gsutil cp (#1182)
Browse files Browse the repository at this point in the history
* multipart uploads: Hack to support gsutil cp

`gsutil` sends an invalid multipart boundary param, which golang's `mime.ParseMediaType` correctly rejects.
However, the real GCS evidently does not reject this, so in order to make `gsutil` work we need to support it.

In particular, `gsutil` sends a boundary param that is quoted using single-quotes when it should be using double-quotes.
In cases where the param is definitely invalid (so we're guarenteed not to break any valid values), we replace all single-quotes with double-quotes to produce the intended meaning.

Upstream bug: GoogleCloudPlatform/gsutil#1466

* Use ` to avoid having to escape

---------

Co-authored-by: fsouza <108725+fsouza@users.noreply.github.com>
  • Loading branch information
ekimekim and fsouza authored May 27, 2023
1 parent 20e9922 commit 6e130ca
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion fakestorage/upload.go
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,18 @@ func getObjectACL(predefinedACL string) []storage.ACLRule {

func (s *Server) multipartUpload(bucketName string, r *http.Request) jsonResponse {
defer r.Body.Close()
_, params, err := mime.ParseMediaType(r.Header.Get(contentTypeHeader))
requestContentType := r.Header.Get(contentTypeHeader)
// gsutil is observed to submit incorrectly-quoted bounary strings
// like: boundary='===============5900997287163282353=='
// See https://github.com/GoogleCloudPlatform/gsutil/issues/1466
// Having an "=" character in the boundary param requires the value be quoted,
// but ' is not a quote char, " is.
// If we see a string like "boundary='=", which is always invalid anyway,
// attempt to rescue the situation by converting all ' to ".
if strings.Contains(requestContentType, "boundary='=") {
requestContentType = strings.ReplaceAll(requestContentType, "'", `"`)
}
_, params, err := mime.ParseMediaType(requestContentType)
if err != nil {
return jsonResponse{
status: http.StatusBadRequest,
Expand Down

0 comments on commit 6e130ca

Please sign in to comment.