Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3api copy-object with --metadata-directive REPLACE appends the metadata instead of replacing it #787

Closed
tonyipm opened this issue Sep 6, 2024 · 0 comments · Fixed by #789
Assignees
Labels
bug Something isn't working

Comments

@tonyipm
Copy link

tonyipm commented Sep 6, 2024

Describe the bug
s3api copy-object with --metadata-directive REPLACE appends the object metadata instead of replacing it

To Reproduce
Create an s3 object 'a' in bucket 'ttt':
DEV03 [/mmfs1/data/TISRC] :>vgaws s3api put-object --bucket ttt --key a
{
"ETag": "d41d8cd98f00b204e9800998ecf8427e"
}

Use copy-object to add some metadata 'xxx=111':
DEV03 [/mmfs1/data/TISRC] :>vgaws s3api copy-object --bucket ttt --key a --copy-source ttt/a --metadata xxx=111 --metadata-directive REPLACE
{
"CopyObjectResult": {
"ETag": "d41d8cd98f00b204e9800998ecf8427e",
"LastModified": "2024-09-06T14:36:12.484700956+01:00"
}
}

Use head-object to display the metadata:
DEV03 [/mmfs1/data/TISRC] :>vgaws s3api head-object --bucket ttt --key a
{
"Restore": "",
"LastModified": "Fri, 06 Sep 2024 14:36:12 GMT",
"ContentLength": 0,
"ETag": "d41d8cd98f00b204e9800998ecf8427e",
"ContentType": "binary/octet-stream",
"Metadata": {
"xxx": "111"
},
"StorageClass": "STANDARD"
}

Repeat the copy-object with some different metadata yyy=222 using the REPLACE directive:
DEV03 [/mmfs1/data/TISRC] :>vgaws s3api copy-object --bucket ttt --key a --copy-source ttt/a --metadata yyy=222 --metadata-directive REPLACE
{
"CopyObjectResult": {
"ETag": "d41d8cd98f00b204e9800998ecf8427e",
"LastModified": "2024-09-06T14:36:12.484700956+01:00"
}
}

Now repeat the head-object, it shows both metadata keys:
DEV03 [/mmfs1/data/TISRC] :>vgaws s3api head-object --bucket ttt --key a
{
"Restore": "",
"LastModified": "Fri, 06 Sep 2024 14:36:12 GMT",
"ContentLength": 0,
"ETag": "d41d8cd98f00b204e9800998ecf8427e",
"ContentType": "binary/octet-stream",
"Metadata": {
"xxx": "111", ##<<<<<<<<<<<<<<<< WRONG
"yyy": "222"
},
"StorageClass": "STANDARD"
}

If you try this on AWS s3 then the second metadata update overwrites the first and so only the second one is seen (ie yyy=222, not xxx=111).

Expected behavior
copy-object using the metadata-directive REPLACE option should erase/overwrite any existing metadata and show only the latest, ie it should behave the same as AWS s3

Server Version
git clone from 6 sep 24

Additional context

@tonyipm tonyipm added the bug Something isn't working label Sep 6, 2024
benmcclelland added a commit that referenced this issue Sep 7, 2024
When using the REPLACE directive, we were incorrectly removing the
old metadata on the object due to missing the metadata prefix on
the key.  Fix this to remove the correct metadata before setting
new metadata.

Fixes #787
benmcclelland added a commit that referenced this issue Sep 7, 2024
When using the REPLACE directive, we were incorrectly removing the
old metadata on the object due to missing the metadata prefix on
the key.  Fix this to remove the correct metadata before setting
new metadata.

Fixes #787
@benmcclelland benmcclelland self-assigned this Sep 7, 2024
@benmcclelland benmcclelland moved this to In Review in VersityGW Project Sep 7, 2024
benmcclelland added a commit that referenced this issue Sep 7, 2024
When using the REPLACE directive, we were incorrectly removing the
old metadata on the object due to missing the metadata prefix on
the key.  Fix this to remove the correct metadata before setting
new metadata.

Fixes #787
benmcclelland added a commit that referenced this issue Sep 7, 2024
When using the REPLACE directive, we were incorrectly removing the
old metadata on the object due to missing the metadata prefix on
the key.  Fix this to remove the correct metadata before setting
new metadata.

Fixes #787
@github-project-automation github-project-automation bot moved this from In Review to Done in VersityGW Project Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants