Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): add signed url #4065

Merged
merged 32 commits into from
Oct 3, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
4bd6e50
chore(docs): remove double word from tests heading (#4063)
Lancear Sep 3, 2023
da7df59
Update signedUrl.w
kavinpanneer Sep 18, 2023
8706506
Update signedUrl.w
kavinpanneer Sep 18, 2023
50e7667
Update bucket.ts
kavinpanneer Sep 18, 2023
415fc2d
Update bucket.inflight.ts
kavinpanneer Sep 18, 2023
4b77f66
Update signedUrl.w
kavinpanneer Sep 18, 2023
2128540
Update bucket.inflight.test.ts
kavinpanneer Sep 18, 2023
a016d51
Update permissions.ts
kavinpanneer Sep 18, 2023
9e5cb42
Update permissions.ts
kavinpanneer Sep 18, 2023
34f0f27
Update bucket.inflight.ts
kavinpanneer Sep 18, 2023
006652f
Update bucket.test.ts
kavinpanneer Sep 18, 2023
e0ef5ce
Update signedUrl.w
kavinpanneer Sep 19, 2023
fc62e44
Update bucket.inflight.test.ts
kavinpanneer Sep 19, 2023
5ec4934
Update bucket.ts
kavinpanneer Sep 23, 2023
2fe963e
Proper Error message to the failure condition
kavinpanneer Sep 23, 2023
1d21506
Aadded comment for the reduced feature
kavinpanneer Sep 23, 2023
d21e33e
Update signedUrl.w
kavinpanneer Sep 23, 2023
d54d278
Update libs/wingsdk/src/shared-aws/bucket.inflight.ts
Sep 26, 2023
069468e
Update libs/wingsdk/src/shared-aws/bucket.inflight.ts
Sep 26, 2023
82999af
Merge branch 'main' into feat-sdk-signedurl
tsuf239 Oct 1, 2023
5330455
Update libs/wingsdk/test/target-sim/bucket.test.ts
kavinpanneer Oct 2, 2023
77df9a8
updating lockfile
tsuf239 Oct 2, 2023
a30691b
Merge remote-tracking branch 'upstream/main' into feat-sdk-signedurl
tsuf239 Oct 2, 2023
79756ca
Revert "updating lockfile"
tsuf239 Oct 2, 2023
5759919
fixing versions
tsuf239 Oct 2, 2023
19d9a6f
fixing tests
tsuf239 Oct 2, 2023
99eef86
Merge branch 'main' into feat-sdk-signedurl
monadabot Oct 2, 2023
22bd31c
chore: self mutation (e2e-1of2.diff)
monadabot Oct 2, 2023
48e9255
Merge remote-tracking branch 'upstream/main' into feat-sdk-signedurl
tsuf239 Oct 3, 2023
7e8f4bd
fix tests
tsuf239 Oct 3, 2023
6101cba
Merge branch 'main' into feat-sdk-signedurl
monadabot Oct 3, 2023
8481d47
chore: self mutation (build.diff)
monadabot Oct 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/02-concepts/04-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ test "bucket starts empty" {

In the first test (`bucket list should include created file`), a file is created in the bucket. The second test (`bucket starts empty`) verifies that the bucket is initialized without any file.

### Running tests in a the cloud
### Running tests in the cloud

Consider the following example:

Expand Down
14 changes: 14 additions & 0 deletions examples/tests/sdk_tests/bucket/signedUrl.w
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
bring cloud;
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
bring http;
bring util;

let testBucket = new cloud.Bucket(public: true) as "testBucket";

test "signedUrl" {
let var error = "";
testBucket.put("file1.txt", "Foo");

let signedUrl = testBucket.signedUrl("file1.txt");
assert(signedUrl != "");
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved

}
1 change: 1 addition & 0 deletions libs/wingsdk/.projenrc.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ const project = new cdk.JsiiProject({
// shared client dependencies
"ioredis",
"jsonschema",
"@aws-sdk/s3-request-presigner",
],
devDeps: [
`@cdktf/provider-aws@^15.0.0`, // only for testing Wing plugins
Expand Down
10 changes: 10 additions & 0 deletions libs/wingsdk/src/cloud/bucket.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { fqnForType } from "../constants";
import { App } from "../core";
import { convertBetweenHandlers } from "../shared/convert";
import { Json, IResource, Node, Resource } from "../std";
import { Duration } from "../std";
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved

/**
* Global identifier for `Bucket`.
Expand Down Expand Up @@ -345,6 +346,13 @@ export interface IBucketClient {
* @inflight
*/
publicUrl(key: string): Promise<string>;

/**
* Returns a signed url to the given file.
* @Throws if object does not exist.
* @inflight
kavinpanneer marked this conversation as resolved.
Show resolved Hide resolved
*/
signedUrl(key: string, duration?: Duration): Promise<string>;
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved
}

/**
Expand Down Expand Up @@ -447,4 +455,6 @@ export enum BucketInflightMethods {
TRY_GET_JSON = "tryGetJson",
/** `Bucket.tryDelete` */
TRY_DELETE = "tryDelete",

SIGNED_URL = "signedUrl"
kavinpanneer marked this conversation as resolved.
Show resolved Hide resolved
}
34 changes: 23 additions & 11 deletions libs/wingsdk/src/shared-aws/bucket.inflight.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,11 @@ import {
} from "@aws-sdk/client-s3";
import { BucketDeleteOptions, IBucketClient } from "../cloud";
import { Duration, Json } from "../std";

import { getSignedUrl ,S3RequestPresigner} from "@aws-sdk/s3-request-presigner";
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved
export class BucketClient implements IBucketClient {
constructor(
private readonly bucketName: string,
private readonly s3Client = new S3Client({})
private readonly s3Client :any = new S3Client({}),
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
) {}

/**
Expand Down Expand Up @@ -263,17 +263,29 @@ export class BucketClient implements IBucketClient {
);
}

/**
* Returns a signed url to the given file. This URL can be used by anyone to
* access the file until the link expires (defaults to 24 hours).

/**
* Returns a signed url to the given file.
* @Throws if object does not exist.
* @inflight
* @param key The key to reach
* @param duration Time until expires
* @param duration Time until expires
*/
public async signed_url(key: string, duration?: Duration): Promise<string> {
// for signed_url take a look here: https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/s3-example-creating-buckets.html#s3-create-presigendurl-get
throw new Error(
`signed_url is not implemented yet (key=${key}, duration=${duration})`
);


public async signedUrl(key: string, duration?: Duration): Promise<string> {
if (!(await this.exists(key))) {
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
throw new Error(
`Cannot provide signed url for a non-existent key (key=${key})`
);
}
const expiryTimeInSeconds:number= duration?.seconds|| 86400;
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
const command: any = new GetObjectCommand({
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
Bucket: this.bucketName,
Key: key
});
return await getSignedUrl(this.s3Client,command,{expiresIn:expiryTimeInSeconds});
kavinpanneer marked this conversation as resolved.
Show resolved Hide resolved

}

private async getLocation(): Promise<string> {
Expand Down
4 changes: 3 additions & 1 deletion libs/wingsdk/src/shared-aws/permissions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ export function calculateBucketPermissions(
// contains a check if an object exists/list
if (
ops.includes(cloud.BucketInflightMethods.PUBLIC_URL) ||
ops.includes(cloud.BucketInflightMethods.SIGNED_URL) ||
ops.includes(cloud.BucketInflightMethods.LIST) ||
ops.includes(cloud.BucketInflightMethods.EXISTS) ||
ops.includes(cloud.BucketInflightMethods.TRY_GET) ||
Expand Down Expand Up @@ -120,13 +121,14 @@ export function calculateBucketPermissions(
ops.includes(cloud.BucketInflightMethods.TRY_GET_JSON) ||
ops.includes(cloud.BucketInflightMethods.TRY_DELETE) ||
ops.includes(cloud.BucketInflightMethods.PUBLIC_URL) ||
ops.includes(cloud.BucketInflightMethods.SIGNED_URL) ||
ops.includes(cloud.BucketInflightMethods.EXISTS)
) {
actions.push("s3:GetObject*", "s3:GetBucket*");
}

// accessing the publicAccessBlock
if (ops.includes(cloud.BucketInflightMethods.PUBLIC_URL)) {
if (ops.includes(cloud.BucketInflightMethods.PUBLIC_URL || ops.includes(cloud.BucketInflightMethods.SIGNED_URL))) {
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved
actions.push("s3:GetBucketPublicAccessBlock");
}

Expand Down
24 changes: 23 additions & 1 deletion libs/wingsdk/src/target-sim/bucket.inflight.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
IBucketClient,
ITopicClient,
} from "../cloud";
import { Json } from "../std";
import { Json , Duration } from "../std";
import {
ISimulatorContext,
ISimulatorResourceInstance,
Expand Down Expand Up @@ -208,6 +208,28 @@ export class Bucket implements IBucketClient, ISimulatorResourceInstance {
});
}

public async signedUrl(key: string, duration?: Duration){
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
const expiryTimeInSeconds:string= String((duration?.seconds|| 86400));
return this.context.withTrace({
message: `Signed URL (key=${key})`,
activity: async () =>{
const filePath = join(this._fileDir, key);
if(!this.objectKeys.has(key)){
throw new Error(
`Cannot provide signed url for an non-existent key (key=${key})`
);
}
/**
* Generating a mock signed url response
*/

return url.pathToFileURL(filePath).searchParams.append("Expires",expiryTimeInSeconds);
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved

}
});
}


private async addFile(key: string, value: string): Promise<void> {
const actionType: BucketEventType = this.objectKeys.has(key)
? BucketEventType.UPDATE
Expand Down
72 changes: 72 additions & 0 deletions libs/wingsdk/test/shared-aws/bucket.inflight.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -535,3 +535,75 @@ test("tryDelete a non-existent object from the bucket", async () => {
// THEN
expect(objectTryDelete).toEqual(false);
});

test("Given a bucket when reaching to a non existent key, signed url it should throw an error", async () => {
// GIVEN
let error;
const BUCKET_NAME = "BUCKET_NAME";
const KEY = "KEY";

s3Mock.on(GetPublicAccessBlockCommand, { Bucket: BUCKET_NAME }).resolves({
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved
PublicAccessBlockConfiguration: {
BlockPublicAcls: false,
BlockPublicPolicy: false,
RestrictPublicBuckets: false,
IgnorePublicAcls: false,
},
});
s3Mock
.on(GetBucketLocationCommand, { Bucket: BUCKET_NAME })
.resolves({ LocationConstraint: "us-east-2" });
s3Mock
.on(HeadObjectCommand, { Bucket: BUCKET_NAME, Key: KEY })
.rejects({ name: "NotFound" });

//WHEN
const client = new BucketClient(BUCKET_NAME);
try {
await client.signedUrl(KEY);
} catch (err) {
error = err;
}
// THEN
expect(error?.message).toBe(
"Cannot provide signed url for a non-existent key (key=KEY)"
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
);
});
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved

test("Given a bucket, when giving one of its keys, we should get its signed url", async () => {
// GIVEN
const BUCKET_NAME = "BUCKET_NAME";
const KEY = "KEY";
const REGION = "us-east-2";

s3Mock.on(GetPublicAccessBlockCommand, { Bucket: BUCKET_NAME }).resolves({
PublicAccessBlockConfiguration: {
BlockPublicAcls: false,
BlockPublicPolicy: false,
RestrictPublicBuckets: false,
IgnorePublicAcls: false,
},
});
s3Mock
.on(GetBucketLocationCommand, { Bucket: BUCKET_NAME })
.resolves({ LocationConstraint: REGION });
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved
s3Mock.on(HeadObjectCommand, { Bucket: BUCKET_NAME, Key: KEY }).resolves({
AcceptRanges: "bytes",
ContentLength: 3191,
ContentType: "image/jpeg",
ETag: "6805f2cfc46c0f04559748bb039d69ae",
LastModified: new Date("Thu, 15 Dec 2016 01:19:41 GMT"),
Metadata: {},
VersionId: "null",
});

// WHEN
const client = new BucketClient(BUCKET_NAME);
const response = await client.signedUrl(KEY);

// THEN
expect(response).contains(
'https://s3.amazonaws.com'
);
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved

});
47 changes: 47 additions & 0 deletions libs/wingsdk/test/target-sim/bucket.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -705,3 +705,50 @@ test("tryDelete objects from bucket", async () => {
expect(existingObject2TryDelete).toEqual(true);
expect(nonExistentObjectTryDelete).toEqual(false);
});

test("Given a bucket when reaching to a non existent key, signed url it should throw an error", async () => {
//GIVEN
let error;
const app = new SimApp();
cloud.Bucket._newBucket(app, "my_bucket", { public: true });

const s = await app.startSimulator();
const client = s.getResource("/my_bucket") as cloud.IBucketClient;

const KEY = "KEY";

// WHEN
try {
await client.signedUrl(KEY);
} catch (err) {
error = err;
}

expect(error?.message).toBe(
"Cannot provide signed url for an non-existent key (key=KEY)"
revitalbarletz marked this conversation as resolved.
Show resolved Hide resolved
);
// THEN
await s.stop();
});
tsuf239 marked this conversation as resolved.
Show resolved Hide resolved


test("Given a bucket, when giving one of its keys, we should get it's signed url", async () => {
// GIVEN
const app = new SimApp();
cloud.Bucket._newBucket(app, "my_bucket", { public: true });

const s = await app.startSimulator();
const client = s.getResource("/my_bucket") as cloud.IBucketClient;

const KEY = "KEY";
const VALUE = "VALUE";

// WHEN
await client.put(KEY, VALUE);
const response = await client.signedUrl(KEY);

// THEN
await s.stop();
const filePath = `${client.fileDir}/${KEY}`;
expect(response).toEqual(url.pathToFileURL(filePath).searchParams.append("Expires","86400"));
});
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,8 @@
"@aws-sdk/util-utf8-node@3.259.0": "patches/@aws-sdk__util-utf8-node@3.259.0.patch",
"@aws-sdk/util-utf8-browser@3.259.0": "patches/@aws-sdk__util-utf8-browser@3.259.0.patch"
}
},
"dependencies": {
"@aws-sdk/s3-request-presigner": "^3.395.0"
}
}