feat: put object with instruction file configured #466

akareddy04 · 2025-05-30T18:47:35Z

Issue #, if available: #355

Description of changes:
*Adds ability to add instruction file configuration for putObject

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Check any applicable:

Were any files moved? Moving files changes their URL, which breaks all hyperlinks to the files.

…a into inst-file-put

…r MPU + instructionFile

…tructionFileConfig

…equest to PutObjectRequest

…nabled

…uctionFile

… variable

kessplas · 2025-06-02T17:43:51Z

src/main/java/software/amazon/encryption/s3/internal/ContentMetadataEncodingStrategy.java

-// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
-// SPDX-License-Identifier: Apache-2.0


We need to include these lines in every file, they should not be deleted here.

kessplas · 2025-06-02T17:54:07Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

+                                    "To work around this issue you can disable multi part upload," +
+                                    "use the Async client, or not set this value on PutObject." +
+                                    "You may be able to update this value after the PutObject request completes."


Suggested change

"To work around this issue you can disable multi part upload," +

"use the Async client, or not set this value on PutObject." +

"You may be able to update this value after the PutObject request completes."

"To work around this issue you can disable Instruction File on PutObject or disable " +

"multi part upload, or use the Async client, or not set this value on PutObject." +

"You may be able to update this value after the request completes."

The text should be slightly different as it only applies to cases where InstructionFile puts are enabled.

kessplas · 2025-06-02T17:54:44Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

+                    // Rather than silently dropping the value,
+                    // we loudly signal that we don't know how to handle this field.
+                    throw new IllegalArgumentException(
+                            f.locationName() + " is an unknown field. " +


Suggested change

f.locationName() + " is an unknown field. " +

f.memberName() + " is an unknown field. " +

Please fix this in the other convertRequest method, we missed that originally.

kessplas · 2025-06-02T17:55:34Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java


 public class ConvertSDKRequests {

+  public static PutObjectRequest convertRequest(CreateMultipartUploadRequest request) {


nit: We should add some comments here, i.e. that this is used to set the optional fields for Instruction File puts when the request is a multipart upload, because the instruction file is Put with an ordinary PutObject request.

And likewise for the other convertRequest method which is used to set the optional fields on high-level Multipart PutObject requests.

src/main/java/software/amazon/encryption/s3/internal/MultipartUploadObjectPipeline.java

kessplas · 2025-06-02T18:24:27Z

src/test/java/software/amazon/encryption/s3/S3EncryptionClientInstructionFileTest.java

+        final InputStream objectStreamForResult = new BoundedInputStream(fileSizeLimit);
+
+        S3Client wrappedClient = S3Client.create();
+        S3Client s3Client = S3EncryptionClient.builder()


We need to set .enableMultipartPutObject(true) to enable multipart put.

kessplas · 2025-06-02T18:26:11Z

src/test/java/software/amazon/encryption/s3/S3EncryptionClientInstructionFileTest.java

+                .bucket(BUCKET)
+                .key(object_key + ".instruction")
+                .build());
+        assertTrue(directInstGetResponse.response().metadata().containsKey("x-amz-crypto-instr-file"));


Let's also set StorageClass in the original put request and check for it here.

kessplas · 2025-06-02T18:26:59Z

src/test/java/software/amazon/encryption/s3/S3EncryptionClientInstructionFileTest.java

+
+
+        CreateMultipartUploadResponse initiateResult = v3Client.createMultipartUpload(builder ->
+                builder.bucket(BUCKET).key(object_key));


Let's also set StorageClass here and check for it in the directInstGetResponse later on.

kessplas · 2025-06-02T18:27:54Z

src/test/java/software/amazon/encryption/s3/internal/ConvertSDKRequestsTest.java

 import software.amazon.awssdk.awscore.AwsRequestOverrideConfiguration;
 import software.amazon.awssdk.services.s3.model.*;

+import java.awt.event.ComponentListener;


This looks unused

kessplas · 2025-06-02T18:29:00Z

src/test/java/software/amazon/encryption/s3/internal/ConvertSDKRequestsTest.java

+  }
+
+  @Test
+  public void testConversionAllFields() {


nit: Specify that this is for CreateMPUToPutObject

kessplas

Just a few minor things to fix

kessplas · 2025-06-04T23:19:42Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

+    /*Converts a CreateMultipartUploadRequest into a PutObjectRequest by setting optional fields needed for
+    putInstructionFile operation.
+     */


This should be a Javadoc comment. Meaning, it comes before the method and it has this syntax (most importantly, the two *s in the first block):

/** * Comment here */ public whatever methodName() {

There are many examples throughout the codebase. Your IDE should also render them as green rather than grey.

In terms of content, it is not really accurate. We should specify that the conversion is used when Instruction File PutObject is enabled and the customer performs a multipart upload. putInstructionFile doesn't exist in v3.

kessplas · 2025-06-04T23:20:20Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java


  public static CreateMultipartUploadRequest convertRequest(PutObjectRequest request) {
-
+     /*Converts a PutObjectRequest into a CreateMultipartUploadRequest by setting optional fields needed for high-level


kessplas · 2025-06-04T23:21:53Z

src/test/java/software/amazon/encryption/s3/S3AsyncEncryptionClientTest.java

 import software.amazon.awssdk.services.s3.model.PutObjectResponse;
 import software.amazon.awssdk.services.s3.model.S3Exception;
 import software.amazon.awssdk.services.s3.model.StorageClass;
+import software.amazon.awssdk.services.s3.model.StorageClassAnalysisDataExport;


I don't think this is used

kessplas

LGTM, lets get another review for this though.

lucasmcdonald3

Mostly looking for confirmation on whether we should include some fields in the ConvertSDKRequests, but the shape of this looks good

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

lucasmcdonald3 · 2025-06-09T16:46:45Z

src/test/java/software/amazon/encryption/s3/internal/ConvertSDKRequestsTest.java

+            .bucket(value)
+            .build();


nit: Indentation; it looks like this file was using 2 spaces per indentation but you've changed it to 6 here. Was that intentional?

That was an error on my part, will fix that now!

lucasmcdonald3

LGTM!

ashu027 · 2025-06-19T12:52:20Z

We have a use case that requires the addition of an instruction file. However, we haven't seen these changes included in the latest release. Could you please let us know when we can expect a new release that incorporates the changes from this PR?

GSudeendra · 2025-07-18T06:39:50Z

@kessplas We rely on separate instruction files for encrypted uploads (legacy setup), but they haven’t been showing up in S3 when using S3AsyncEncryptionClient. Looks like this PR fixes that.

Any idea when the next release with this fix will be out?

Thank You

kessplas · 2025-07-18T16:54:03Z

Hey @ashu027 and @GSudeendra ,
Sorry for the late reply. This feature, along with functionality to re-encrypt/rotate keys using instruction files will be released in the coming weeks. Thanks.

## [3.4.0](v3.3.5...v3.4.0) (2025-07-30) ### Features * put object with instruction file configured ([#466](#466)) ([99077dc](99077dc)) * reEncryptInstructionFile Implementation ([#475](#475)) ([ff66e72](ff66e72)) * reEncryptInstructionFile Implementation ([#478](#478)) ([f7e6fa5](f7e6fa5)) ### Fixes * Revert "feat: reEncryptInstructionFile Implementation ([#475](#475))" ([#477](#477)) ([6d45ec5](6d45ec5)) ### Maintenance * guard against properties conflicts ([#479](#479)) ([793c73b](793c73b)) * **pom:** fix scm url ([#469](#469)) ([1bc2ca3](1bc2ca3)) * **release:** Migrate release to Central Portal ([#468](#468)) ([da71231](da71231)) * validate against legacy wrapping on client but customer passes keyring with no legacy wrapping ([#473](#473)) ([bb898d1](bb898d1))

kessplas · 2025-08-01T23:19:35Z

Hey @ashu027 and @GSudeendra ,
This feature, along with functionality to re-encrypt/rotate keys using instruction files has been released in v3.4.0 and is available in Maven. Thanks!

GSudeendra · 2025-08-13T06:31:24Z

@kessplas thanks for the update

kessplas and others added 20 commits November 25, 2024 14:05

step 1 - refactoring

8e51328

wip - instruction file puts

898c55e

cleanup

fd64f1c

Merge branch 'main' of github.com:aws/amazon-s3-encryption-client-jav…

95b3b5a

…a into inst-file-put

Merge branch 'main' of github.com:aws/amazon-s3-encryption-client-jav…

3546538

…a into inst-file-put

more tests in new class

ed761d9

Merge branch 'main' of github.com:aws/amazon-s3-encryption-client-jav…

a34b347

…a into inst-file-put

v3 to v2 tests for default client

d6da2cf

Merge branch 'main' of github.com:aws/amazon-s3-encryption-client-jav…

60feafd

…a into inst-file-put

Fixed nullptr exception in code

288ee65

testing whether v2 does or doesn't translate additional parameters fo…

9c1c50d

…r MPU + instructionFile

Fixed nullptr errors for MultipartUploadObjectPipeline regarding _ins…

ae6b6b4

…tructionFileConfig

Added another convertRequest method to convert CreateMultipartUploadR…

c9378cf

…equest to PutObjectRequest

Added test cases for converting MultipartUploadRequests

ef353d7

Added tests for S3EncryptonClient with Instruction File + Multipart e…

fbdbe92

…nabled

added .instructionFileConfig for PutEncryptedObjectPipeline builder

9e25d8f

cleaned up ContentMetadataEncodingStrategy class

7bf259c

Added test cases to S3AsyncEncryptionClientTest regarging MPU + instr…

ff5e99c

…uctionFile

fixed import issues

a396a7c

removed debugging lines

b1c0898

akareddy04 requested a review from a team as a code owner May 30, 2025 18:47

Anirav Kareddy added 2 commits May 30, 2025 12:47

Replaced passing in ARN of my personal testing bucket with the BUCKET…

b4b2047

… variable

removed unused import

b9ae158

kessplas requested changes Jun 2, 2025

View reviewed changes

Anirav Kareddy added 2 commits June 3, 2025 16:25

fixed all changes requested from PR

4f51c42

Removed HelloWorldProgramExample.java

bf2be6e

kessplas requested changes Jun 4, 2025

View reviewed changes

Anirav Kareddy added 2 commits June 6, 2025 09:21

Fixed the second PR round of reviews

02f6f48

testMultipartPutWithInstructionFile() should be working now

c319d44

kessplas previously approved these changes Jun 6, 2025

View reviewed changes

lucasmcdonald3 reviewed Jun 9, 2025

View reviewed changes

fixed second round of PR

0ed41a5

akareddy04 dismissed kessplas’s stale review via 0ed41a5 June 12, 2025 22:04

removed unused import

0349dec

lucasmcdonald3 approved these changes Jun 13, 2025

View reviewed changes

akareddy04 merged commit 99077dc into main Jun 13, 2025
45 of 56 checks passed

		// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
		// SPDX-License-Identifier: Apache-2.0

	f.locationName() + " is an unknown field. " +
	f.memberName() + " is an unknown field. " +


		public class ConvertSDKRequests {

		public static PutObjectRequest convertRequest(CreateMultipartUploadRequest request) {



		CreateMultipartUploadResponse initiateResult = v3Client.createMultipartUpload(builder ->
		builder.bucket(BUCKET).key(object_key));


		public static CreateMultipartUploadRequest convertRequest(PutObjectRequest request) {

		/*Converts a PutObjectRequest into a CreateMultipartUploadRequest by setting optional fields needed for high-level

feat: put object with instruction file configured #466

feat: put object with instruction file configured #466

Uh oh!

Conversation

akareddy04 commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Check any applicable:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kessplas left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kessplas left a comment

Choose a reason for hiding this comment

Uh oh!

lucasmcdonald3 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucasmcdonald3 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ashu027 commented Jun 19, 2025

Uh oh!

GSudeendra commented Jul 18, 2025

Uh oh!

kessplas commented Jul 18, 2025

Uh oh!

kessplas commented Aug 1, 2025

Uh oh!

GSudeendra commented Aug 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

akareddy04 commented May 30, 2025 •

edited

Loading