Add support for S3 Metadata

[urgelacko]

Amazon S3 provides a way to set several Metadata information such as Server Side Encryption, custom user metadata, etc...

SimpleStorageResource currently handles version numbers in url and object metadata is saved to a private field but since it is not exposed I cannot use it.
Besides this no any metadata can be set through Spring Cloud S3 library cause UploadPartResultCallable call method does not set any ObjectMetada on the request.
Some extension points for setting global metadata for all uploads (like SSE algorithm) would be nice while metadata support on resource level similar to versioning in url (^version) would be awesome.

[jeffellin]

@dsyer is someone working on this? If not it might be something I can take on. I just opened a related issue for not being able to access SimpleStorageResource, but I see that I still won't be able to access the metadata.

[ausov]

Are you sure it's the right pattern to retrieve meta-data? From my understanding, the Resource is a a replacement for URL that is suitable for IO ops. Isn't it better to use a separate service for meta-data retrieval?

[urgelacko]

There should be some mechanism which allows you to manage metadata also since it is part of S3 and it is often needed. This can be implemented with a wrapper or with some service abstraction while the current resource mechanism still would supported.
But currently this is the Spring API for S3 integration and the metadata is part of S3 API. Without metadata support this spring lib is not usable for mature S3 integration.
For example encryption is a functionality which we are using to store sensitive data and it is working with metadata.

[ausov]

Agree regarding metadata service. Although all needed already exists in AWS SDK there can be a simple service bean with convenient ops.

Not so sure about using a Resource class for that. Consider also that Resource should be cacheable so it supposed to be lightweight and not referring to beans, etc. I would suggest to leave it simple with intention to use it in resource resolving chain, IO ops, etc. Like replacement for an URL. Just seems a bit heavy to use it for something more.

[mdedgjonaj]

This is a very valid issue and I am surprised that not the right importance is given to it. In my understanding SimpleStorageResource is a WritableResource and we are supposed to use that to upload new documents as well not just for download. If I want to set Server Side Encryption for the files that I am uploading, I will not be able to do it. If somebody knows a work-around until this feature is added, I would love to hear about it.

[mdedgjonaj]

The solution I would suggest for this is by passing a default initial ObjectMetadata to SimpleStorageResourceLoader which will be passed to SimpleStorageResource when getResource is called.
That ObjectMetadata will be the initial one and all the needed changes will be done on top of that.
That ObjectMetadata should be used only when doing PUT. Nothing changes on getting resources from AWS.

[mdedgjonaj]

@aemruli, @spencergibb, @dsyer : Is anybody working on this? I would love to contribute to solve this issue. I am anyway fixing it for my use case.
I have explained above how I see this feature working.
Please let me know if anybody else has any different suggestion on how to solve it?

[hopeng]

I can't use spring cloud for S3 upload because it cannot set SSE. Would be great to see a solution to this problem.

[lukaszkrawiec]

huh, like others I'm surprised this is not a feature. Can't use spring-cloud-aws because of that

[raviguptasmarsh]

Cannot use spring-cloud-aws because of SSE requirement. Is this in pipeline ?