Skip to content

Commit

Permalink
Add Get- / Put- ObjectAcl
Browse files Browse the repository at this point in the history
Implement GetObjectACL / PutObjectACL
Return / accept String instead of a POJO. We need to use JAX-B
annotations instead of Jackson annotations because AWS decided to use
xsi:type annotations in the XML representation, which are not supported
by Jackson. It doesn't seem to be possible to use bot JAX-B and Jackson
for (de-)serialization in parallel.

fixes #213 / #290
  • Loading branch information
afranken committed Sep 20, 2022
1 parent cba5b33 commit 73d166c
Show file tree
Hide file tree
Showing 31 changed files with 1,095 additions and 60 deletions.
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,17 @@ Version 2.x is JDK8 LTS bytecode compatible, with Docker and JUnit / direct Java
* Version updates
* TBD

## 2.7.0
2.x will support JDK8 LTS until LTS support is cancelled, with Docker and JUnit integrations as-is.

* Features and fixes
* Add support for ACL APIs (fixes #213 / #290)
* Implement GetObjectACL / PutObjectACL
* Return / accept String instead of a POJO. We need to use JAX-B annotations instead of Jackson annotations
because AWS decided to use xsi:type annotations in the XML representation, which are not supported
by Jackson. It doesn't seem to be possible to use bot JAX-B and Jackson for (de-)serialization in parallel.


## 2.6.1
2.x is JDK8 LTS bytecode compatible, with Docker and JUnit / direct Java integration.

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ The following [actions are supported by Amazon S3](https://docs.aws.amazon.com/A
| [GetBucketVersioning](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketVersioning.html) | :x: |
| [GetBucketWebsite](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketWebsite.html) | :x: |
| [GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html) | :white_check_mark: |
| [GetObjectAcl](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAcl.html) | :x: |
| [GetObjectAcl](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAcl.html) | :white_check_mark: |
| [GetObjectAttributes](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAttributes.html) | :x: |
| [GetObjectLegalHold](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLegalHold.html) | :white_check_mark: |
| [GetObjectLockConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLockConfiguration.html) | :white_check_mark: |
Expand Down Expand Up @@ -130,7 +130,7 @@ The following [actions are supported by Amazon S3](https://docs.aws.amazon.com/A
| [PutBucketVersioning](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketVersioning.html) | :x: |
| [PutBucketWebsite](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketWebsite.html) | :x: |
| [PutObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html) | :white_check_mark: |
| [PutObjectAcl](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectAcl.html) | :x: |
| [PutObjectAcl](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectAcl.html) | :white_check_mark: |
| [PutObjectLegalHold](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLegalHold.html) | :white_check_mark: |
| [PutObjectLockConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLockConfiguration.html) | :white_check_mark: |
| [PutObjectRetention](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectRetention.html) | :white_check_mark: |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
/*
* Copyright 2017-2022 Adobe.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.adobe.testing.s3mock.its

import com.adobe.testing.s3mock.dto.Owner.DEFAULT_OWNER
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.TestInfo
import software.amazon.awssdk.core.sync.RequestBody
import software.amazon.awssdk.services.s3.model.AccessControlPolicy
import software.amazon.awssdk.services.s3.model.CreateBucketRequest
import software.amazon.awssdk.services.s3.model.GetObjectAclRequest
import software.amazon.awssdk.services.s3.model.Grant
import software.amazon.awssdk.services.s3.model.Grantee
import software.amazon.awssdk.services.s3.model.Owner
import software.amazon.awssdk.services.s3.model.Permission.FULL_CONTROL
import software.amazon.awssdk.services.s3.model.PutObjectAclRequest
import software.amazon.awssdk.services.s3.model.PutObjectRequest
import software.amazon.awssdk.services.s3.model.Type.CANONICAL_USER
import java.io.File

class AclIT : S3TestBase() {

@Test
fun testGetAcl_noAcl(testInfo: TestInfo) {
val uploadFile = File(UPLOAD_FILE_NAME)
val sourceKey = UPLOAD_FILE_NAME
val bucketName = bucketName(testInfo)
s3ClientV2!!.createBucket(CreateBucketRequest.builder().bucket(bucketName).build())
s3ClientV2!!.putObject(
PutObjectRequest.builder().bucket(bucketName).key(sourceKey).build(),
RequestBody.fromFile(uploadFile)
)

val acl = s3ClientV2!!.getObjectAcl(
GetObjectAclRequest
.builder()
.bucket(bucketName)
.key(sourceKey)
.build()
)

val owner = acl.owner()
assertThat(owner.id()).isEqualTo(DEFAULT_OWNER.id)
assertThat(owner.displayName()).isEqualTo(DEFAULT_OWNER.displayName)
val grants = acl.grants()
assertThat(grants).hasSize(1)
val grant = grants[0]
assertThat(grant.permission()).isEqualTo(FULL_CONTROL)
val grantee = grant.grantee()
assertThat(grantee).isNotNull
assertThat(grantee.id()).isEqualTo(DEFAULT_OWNER.id)
assertThat(grantee.displayName()).isEqualTo(DEFAULT_OWNER.displayName)
assertThat(grantee.type()).isEqualTo(CANONICAL_USER)
}


@Test
fun testPutAndGetAcl(testInfo: TestInfo) {
val uploadFile = File(UPLOAD_FILE_NAME)
val key = UPLOAD_FILE_NAME
val bucketName = bucketName(testInfo)
s3ClientV2!!.createBucket(
CreateBucketRequest
.builder()
.bucket(bucketName)
.objectLockEnabledForBucket(true)
.build()
)
s3ClientV2!!.putObject(
PutObjectRequest.builder().bucket(bucketName).key(key).build(),
RequestBody.fromFile(uploadFile)
)

val userId = "79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2ab"
val userName = "John Doe"
val granteeId = "79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2ef"
val granteeName = "Jane Doe"
val granteeEmail = "jane@doe.com"
s3ClientV2!!.putObjectAcl(
PutObjectAclRequest
.builder()
.bucket(bucketName)
.key(key)
.accessControlPolicy(
AccessControlPolicy
.builder()
.owner(Owner.builder().id(userId).displayName(userName).build())
.grants(
Grant.builder()
.permission(FULL_CONTROL)
.grantee(
Grantee.builder().id(granteeId).displayName(granteeName)
.type(CANONICAL_USER).build()
).build()
).build()
)
.build()
)

val acl = s3ClientV2!!.getObjectAcl(
GetObjectAclRequest
.builder()
.bucket(bucketName)
.key(key)
.build()
)
val owner = acl.owner()
assertThat(owner).isNotNull
assertThat(owner.id()).isEqualTo(userId)
assertThat(owner.displayName()).isEqualTo(userName)

assertThat(acl.grants()).hasSize(1)

val grant = acl.grants()[0]
assertThat(grant.permission()).isEqualTo(FULL_CONTROL)

val grantee = grant.grantee()
assertThat(grantee).isNotNull
assertThat(grantee.id()).isEqualTo(granteeId)
assertThat(grantee.displayName()).isEqualTo(granteeName)
assertThat(grantee.type()).isEqualTo(CANONICAL_USER)
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ class BucketTestsV1IT : S3TestBase() {
assertThat(createdBucket.creationDate).isAfterOrEqualTo(creationDate)
val bucketOwner = createdBucket.owner
assertThat(bucketOwner.displayName).isEqualTo("s3-mock-file-store")
assertThat(bucketOwner.id).isEqualTo("123")
assertThat(bucketOwner.id)
.isEqualTo("79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be")
}

@Test
Expand Down
3 changes: 1 addition & 2 deletions server/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
</properties>

<dependencies>

<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
Expand Down Expand Up @@ -74,7 +73,7 @@
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-runtime</artifactId>
<scope>test</scope>
<!-- <scope>test</scope>-->
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,10 @@
import static com.adobe.testing.s3mock.util.AwsHttpHeaders.X_AMZ_SERVER_SIDE_ENCRYPTION;
import static com.adobe.testing.s3mock.util.AwsHttpHeaders.X_AMZ_SERVER_SIDE_ENCRYPTION_AWS_KMS_KEY_ID;
import static com.adobe.testing.s3mock.util.AwsHttpHeaders.X_AMZ_TAGGING;
import static com.adobe.testing.s3mock.util.AwsHttpParameters.ACL;
import static com.adobe.testing.s3mock.util.AwsHttpParameters.DELETE;
import static com.adobe.testing.s3mock.util.AwsHttpParameters.LEGAL_HOLD;
import static com.adobe.testing.s3mock.util.AwsHttpParameters.NOT_ACL;
import static com.adobe.testing.s3mock.util.AwsHttpParameters.NOT_LEGAL_HOLD;
import static com.adobe.testing.s3mock.util.AwsHttpParameters.NOT_RETENTION;
import static com.adobe.testing.s3mock.util.AwsHttpParameters.NOT_TAGGING;
Expand All @@ -51,12 +53,14 @@
import static org.springframework.http.HttpStatus.REQUESTED_RANGE_NOT_SATISFIABLE;
import static org.springframework.http.MediaType.APPLICATION_XML_VALUE;

import com.adobe.testing.s3mock.dto.AccessControlPolicy;
import com.adobe.testing.s3mock.dto.CopyObjectResult;
import com.adobe.testing.s3mock.dto.CopySource;
import com.adobe.testing.s3mock.dto.Delete;
import com.adobe.testing.s3mock.dto.DeleteResult;
import com.adobe.testing.s3mock.dto.LegalHold;
import com.adobe.testing.s3mock.dto.ObjectKey;
import com.adobe.testing.s3mock.dto.Owner;
import com.adobe.testing.s3mock.dto.Range;
import com.adobe.testing.s3mock.dto.Retention;
import com.adobe.testing.s3mock.dto.Tag;
Expand All @@ -65,12 +69,15 @@
import com.adobe.testing.s3mock.service.ObjectService;
import com.adobe.testing.s3mock.store.S3ObjectMetadata;
import com.adobe.testing.s3mock.util.AwsHttpHeaders.MetadataDirective;
import com.adobe.testing.s3mock.util.XmlUtil;
import java.io.InputStream;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import javax.xml.bind.JAXBException;
import javax.xml.stream.XMLStreamException;
import org.apache.commons.io.IOUtils;
import org.apache.commons.io.input.BoundedInputStream;
import org.springframework.http.HttpHeaders;
Expand Down Expand Up @@ -204,7 +211,8 @@ public ResponseEntity<Void> deleteObject(@PathVariable String bucketName,
NOT_UPLOAD_ID,
NOT_TAGGING,
NOT_LEGAL_HOLD,
NOT_RETENTION
NOT_RETENTION,
NOT_ACL
},
method = RequestMethod.GET,
produces = {
Expand Down Expand Up @@ -240,6 +248,70 @@ public ResponseEntity<StreamingResponseBody> getObject(@PathVariable String buck
.body(outputStream -> Files.copy(s3ObjectMetadata.getDataPath(), outputStream));
}

/**
* Adds an ACL to an object.
* This method accepts a String instead of the POJO. We need to use JAX-B annotations
* instead of Jackson annotations because AWS decided to use xsi:type annotations in the XML
* representation, which are not supported by Jackson.
* It doesn't seem to be possible to use bot JAX-B and Jackson for (de-)serialization in parallel.
* :-(
* <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectAcl.html">API Reference</a>
*
* @param bucketName the Bucket in which to store the file in.
*
* @return {@link ResponseEntity} with Status Code and empty ETag.
*/
@RequestMapping(
value = "/{bucketName:[a-z0-9.-]+}/{*key}",
params = {
ACL,
},
method = RequestMethod.PUT,
consumes = APPLICATION_XML_VALUE
)
public ResponseEntity<Void> putObjectAcl(@PathVariable final String bucketName,
@PathVariable ObjectKey key,
@RequestBody String body) throws XMLStreamException, JAXBException {
bucketService.verifyBucketExists(bucketName);
objectService.verifyObjectExists(bucketName, key.getKey());
AccessControlPolicy policy = XmlUtil.deserializeJaxb(body);
objectService.setAcl(bucketName, key.getKey(), policy);
return ResponseEntity
.ok()
.build();
}

/**
* Gets ACL of an object.
* This method returns a String instead of the POJO. We need to use JAX-B annotations
* instead of Jackson annotations because AWS decided to use xsi:type annotations in the XML
* representation, which are not supported by Jackson.
* It doesn't seem to be possible to use bot JAX-B and Jackson for (de-)serialization in parallel.
* :-(
* <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAcl.html">API Reference</a>
*
* @param bucketName the Bucket in which to store the file in.
*
* @return {@link ResponseEntity} with Status Code and empty ETag.
*/
@RequestMapping(
value = "/{bucketName:[a-z0-9.-]+}/{*key}",
params = {
ACL,
},
method = RequestMethod.GET,
produces = {
APPLICATION_XML_VALUE
}
)
public ResponseEntity<String> getObjectAcl(@PathVariable final String bucketName,
@PathVariable ObjectKey key) throws JAXBException {
bucketService.verifyBucketExists(bucketName);
objectService.verifyObjectExists(bucketName, key.getKey());
AccessControlPolicy acl = objectService.getAcl(bucketName, key.getKey());
return ResponseEntity.ok(XmlUtil.serializeJaxb(acl));
}

/**
* Returns the tags identified by bucketName and fileName.
* <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html">API Reference</a>
Expand Down Expand Up @@ -423,7 +495,8 @@ public ResponseEntity<String> putObjectRetention(@PathVariable String bucketName
NOT_UPLOAD_ID,
NOT_TAGGING,
NOT_LEGAL_HOLD,
NOT_RETENTION
NOT_RETENTION,
NOT_ACL
},
headers = {
NOT_X_AMZ_COPY_SOURCE
Expand All @@ -447,6 +520,8 @@ public ResponseEntity<String> putObject(@PathVariable String bucketName,
bucketService.verifyBucketExists(bucketName);

InputStream stream = objectService.verifyMd5(inputStream, contentMd5, sha256Header);
//TODO: need to extract owner from headers
Owner owner = Owner.DEFAULT_OWNER;
Map<String, String> userMetadata = getUserMetadata(headers);
S3ObjectMetadata s3ObjectMetadata =
objectService.putS3Object(bucketName,
Expand All @@ -458,7 +533,8 @@ public ResponseEntity<String> putObject(@PathVariable String bucketName,
userMetadata,
encryption,
kmsKeyId,
tags);
tags,
owner);

return ResponseEntity
.ok()
Expand Down Expand Up @@ -486,7 +562,11 @@ public ResponseEntity<String> putObject(@PathVariable String bucketName,
X_AMZ_COPY_SOURCE
},
params = {
NOT_UPLOAD_ID
NOT_UPLOAD_ID,
NOT_TAGGING,
NOT_LEGAL_HOLD,
NOT_RETENTION,
NOT_ACL
},
method = RequestMethod.PUT,
produces = {
Expand Down
Loading

0 comments on commit 73d166c

Please sign in to comment.