Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: sync main with storage browser integrity branch #13928

Merged
merged 17 commits into from
Oct 16, 2024

Conversation

Samaritan1011001
Copy link
Member

Description of changes

Issue #, if available

Description of how you validated changes

Checklist

  • PR description included
  • yarn test passes
  • Unit Tests are changed or added
  • Relevant documentation is changed or added (and PR referenced)

Checklist for repo maintainers

  • Verify E2E tests for existing workflows are working as expected or add E2E tests for newly added workflows
  • New source file paths included in this PR have been added to CODEOWNERS, if appropriate

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

erinleigh90 and others added 14 commits October 4, 2024 09:56
 - @aws-amplify/adapter-nextjs@1.2.21
 - @aws-amplify/analytics@7.0.51
 - @aws-amplify/api@6.0.53
 - @aws-amplify/api-graphql@4.4.0
 - @aws-amplify/api-rest@4.0.51
 - @aws-amplify/auth@6.5.1
 - aws-amplify@6.6.4
 - @aws-amplify/core@6.4.4
 - @aws-amplify/datastore@5.0.53
 - @aws-amplify/datastore-storage-adapter@2.1.53
 - @aws-amplify/geo@3.0.51
 - @aws-amplify/interactions@6.0.50
 - @aws-amplify/notifications@2.0.51
 - @aws-amplify/predictions@6.1.26
 - @aws-amplify/pubsub@6.1.26
 - @aws-amplify/storage@6.6.9
 - tsc-compliance-test@0.1.56
* docs: update contributing guide vite troubleshooting

* Update CONTRIBUTING.md

---------

Co-authored-by: AllanZhengYP <zheallan@amazon.com>
 - @aws-amplify/adapter-nextjs@1.2.22
 - @aws-amplify/analytics@7.0.52
 - @aws-amplify/api@6.0.54
 - @aws-amplify/api-graphql@4.4.1
 - @aws-amplify/api-rest@4.0.52
 - @aws-amplify/auth@6.5.2
 - aws-amplify@6.6.5
 - @aws-amplify/core@6.4.5
 - @aws-amplify/datastore@5.0.54
 - @aws-amplify/datastore-storage-adapter@2.1.54
 - @aws-amplify/geo@3.0.52
 - @aws-amplify/interactions@6.0.51
 - @aws-amplify/notifications@2.0.52
 - @aws-amplify/predictions@6.1.27
 - @aws-amplify/pubsub@6.1.27
 - @aws-amplify/storage@6.6.10
 - tsc-compliance-test@0.1.57
jimblanc
jimblanc previously approved these changes Oct 16, 2024
@Samaritan1011001 Samaritan1011001 changed the base branch from storage-browser/main to storage-browser/integrity October 16, 2024 16:03
@Samaritan1011001 Samaritan1011001 dismissed jimblanc’s stale review October 16, 2024 16:03

The base branch was changed.

@Samaritan1011001 Samaritan1011001 changed the title chore: sync main with storage browser main chore: sync main with storage browser integrity branch Oct 16, 2024
@@ -24,7 +24,7 @@ export const uploadData = (
StorageValidationErrorCode.ObjectIsTooLarge,
);

if (dataByteLength && dataByteLength <= DEFAULT_PART_SIZE) {
if (dataByteLength !== undefined && dataByteLength <= DEFAULT_PART_SIZE) {
Copy link
Member Author

@Samaritan1011001 Samaritan1011001 Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manually added this here since we moved this logic inside and we made this exact change in uploadData.ts earlier according to this PR.
#13927

HuiSF
HuiSF previously approved these changes Oct 16, 2024
jimblanc
jimblanc previously approved these changes Oct 16, 2024
@Samaritan1011001 Samaritan1011001 dismissed stale reviews from jimblanc and HuiSF via bac2f00 October 16, 2024 18:05
jimblanc
jimblanc previously approved these changes Oct 16, 2024
@Samaritan1011001 Samaritan1011001 merged commit 9563334 into storage-browser/integrity Oct 16, 2024
28 checks passed
@Samaritan1011001 Samaritan1011001 deleted the merge/to-sb-main branch October 16, 2024 19:31
ashika112 added a commit that referenced this pull request Nov 20, 2024
* chore: enable storage-browser preid release

* chore: sync main (#13478)

* release(required): Parsing custom oAuth in amplify_outputs (#13474)

* update parseAmplify logic

* revert custom oAuth from gen1 config

* update bundle size

* chore(release): Publish [skip release]

 - @aws-amplify/adapter-nextjs@1.2.3
 - @aws-amplify/analytics@7.0.35
 - @aws-amplify/api@6.0.37
 - @aws-amplify/api-graphql@4.1.6
 - @aws-amplify/api-rest@4.0.35
 - @aws-amplify/auth@6.3.5
 - aws-amplify@6.3.6
 - @aws-amplify/core@6.3.2
 - @aws-amplify/datastore@5.0.37
 - @aws-amplify/datastore-storage-adapter@2.1.37
 - @aws-amplify/geo@3.0.35
 - @aws-amplify/interactions@6.0.34
 - @aws-amplify/notifications@2.0.35
 - @aws-amplify/predictions@6.1.10
 - @aws-amplify/pubsub@6.1.9
 - @aws-amplify/storage@6.4.6
 - tsc-compliance-test@0.1.39

* chore(release): update API docs [skip release]

---------

Co-authored-by: ashika112 <155593080+ashika112@users.noreply.github.com>
Co-authored-by: aws-amplify-bot <aws@amazon.com>
Co-authored-by: Jim Blanchard <jim.l.blanchard@gmail.com>

* feat(storage): add delimiter support (#13480)

* feat: add delimiter input/output types

* feat: pass Delimiter parameter to ListObjectsV2 API

* chore: add unit tests

* chore: bump bunde size

* chore: address feedback

* chore: fix build

* chore: address feedback

* chore: address feedback

* chore: address feedback

* chore: enable storage-browser preid release (#13524)

chore: fix npm dist tag to be storage-browser

* feat(storage): add base types for storage browser (#13528)

* feat(storage): add base types

* fix(storage): address feedbacks

Co-authored-by: israx <70438514+israx@users.noreply.github.com>

* feat(storage): add creds store scaffolding and update types (#13558)

* feat(storage): add cred store lru implementation (#13561)

* refactor(storage): decouple utils from Amplify singleton (#13562)

* feat: add config constructor

* refactor: remove singleton reference from storage utils

* refactor: update storage utils

* chore: update upload api

* chore: address feedback

* chore: fix unit tests

* chore: remove singleton reference

* chore: add license headers

* chore: address feedback

* chore: update bundle size

* chore: address feedback

* chore: update bundle size

* feat(storage): add cred store creation implementation (#13575)

Co-authored-by: israx <70438514+israx@users.noreply.github.com>

* feat(storage): use WeakMap for store registry (#13586)

* feat(storage): use WeakMap for store registry

* chore(storage): export storage browser utils from @aws-amplify/storage/storage-browser

* doc(storage): add disclaimer

* feat(storage): Added getDataAccess & listCallerAccessGrant clients (#13582)

* feat(storage): add an adapter interface for storage browser (#13576)


---------

Co-authored-by: Jim Blanchard <jim.l.blanchard@gmail.com>

* Revert "refactor(storage): decouple utils from Amplify singleton (#13562)" (#13597)

This reverts commit 1079e7f.

* feat(storage): simplify the location cred provider option input (#13601)

Remove the un-essetnial validation of per-API's location credentials provider
input scope and permission for now.

* feat: Implement getLocationCredentials handler & integrate with adapter (#13600)

* feat(storage): implement listLocations API and creation handler (#13602)

* chore: expose path storage-browser from scoped package (#13611)

chore: expose path storage-browser from scoped package

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* feat(storage): enables location credentials provider (#13605)

* feat: add location credentials provider

* chore: add unit tests

* chore: address feedback

* chore: add locationCredentialsOption to copy

* chore: remove casting types

* chore: assert idenitity id

* chore: avoid export common options interface

* chore: address feedback

* chore: fix test

* chore: address feedback

* address feedback

* chore: clean-up types

* chore: add test

* chore: update api bundlesize

* feat(storage): resolve merge issue with multibucket

* chore: update bundle size for config change and s3 multibucket

* chore: address feedbacks

* feat(storage): introduce preventOverwrite option to uploadData via HeadObject (#13640)

* feat(storage): introduce preventOverwrite operation to uploadData via HeadObject

* fix: add missing license and remove dependency on core in preventOverwrite validator

* chore: update storage:uploadData bundle size

* feat: move existing object validation to before completeMultipartUpload

* fix: increase storage:uploadData bundle size

* fix(storage): export storage-browser types for TS v4.2+ (#13647)

* chore(storage-browser): export store and credentials related types, update createListLocationsHandler (#13660)

* feat(storage): support force refresh location credentials (#13589)

* feat(storage): require temporary creds for storage browser interfaces (#13664)

* feat: introduce CRC32 checksums to storage:uploadData API (#13649)

Co-authored-by: Donny Wu <wuuxi@amazon.com>

* fix(storage-browser): listCallerAccessGrantsDeserializer not parsing multiple AccessGrant instances (#13671)

* fix(storage-browser): listCallerAccessGrantsDeserializernot parsing multiple AccessGrant instances

* chore: add unit tests for single and multiple grants

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore(storage-browser): expose additional input output types (#13682)

* chore(storage-browser): expose additional internal types

* address feedback

* remove 'applicationArn' from listCallerAccessGrant unit test

* Update packages/storage/src/storageBrowser/apis/listCallerAccessGrants.ts

Co-authored-by: Caleb Pollman <cpollman1@gmail.com>

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>
Co-authored-by: Caleb Pollman <cpollman1@gmail.com>

* chore: add ui to storage browser co-owner

* chore: update bundle size

* chore: enable storage-browser integ test (#13698)

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore(storage): update s3 control model (#13705)

* chore(storage): update s3 control model

* fix: move permission validation to custom client

* chore: formatting code

* chore(storage-browser): pin crc-32 dep at 1.2.2 (#13752)

* chore(storage-browser): pin crc-32 dep at 1.2.2

* chore: update lock file

* chore(storage): add durability check for urls in put object (#13746)

chore: add durability check for presigned urls in put object

* fix(storage-browser): missing error wrapping for s3 control responses (#13779)

* feat: validate corroborating elements in response, and compare echoed elements between request/response (#13764)

* feat: add list call response validation

* fix: add input key to test mock

* feat: align list response validations with design

* fix: increment bundle size for storage:list

---------

Co-authored-by: Donny Wu <wuuxi@amazon.com>

* feat(storage): add support for conditional headers to copy, and validate serialization (#13772)

* feat: add notModifiedSince and eTag to copy

* missing tests for durability helpers

* add tests for integrity helpers

* feat: add URL validation and tests for copyObject

* chore: increase bundle size of storage:copy

* feat: clean-up copy header validation logic

* fix: revert copy option interface name changes

---------

Co-authored-by: Donny Wu <wuuxi@amazon.com>

* feat: adding object url checks (#13810)

* chore: durability checks for create & complete multipart (#13809)

* chore: durability checks for create & complete multipart

* fix: parsePayload mock path

* chore: enable e2e and tagged release (#13848)

* chore: enable e2e and tagged release

* core: update tag name

* Revert "core: update tag name"

This reverts commit c994f51.

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore: Setup storage internals route (#13858)

* chore: increse bundle size

* chore: Refactor contents of `storage-browser/` path into `internals/` (#13859)

* feat(storage): export Default Part Size constant (#13851)

* chore: adds a flag to uploadData api output

* chore: update comments for the flag

* chore: revert some naming changes

* chore: revert and make internal paramter optional with a default value

* chore: update bundle size

* chore: update to expose constant instead of bool in return type

* chore: update export tests

* chore: increase bundle size

* chore: Setup storage internals route (#13858)

* chore: Refactor contents of `storage-browser/` path into `internals/` (#13859)

* fix: multipart upload storage crc32 integrity (#13878)

* fix: multipart upload storage crc32 integrity

* address comments

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* fix linter

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>
Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* feat(storage): internal GetProperties API (#13869)


---------

Co-authored-by: Jim Blanchard <jim.l.blanchard@gmail.com>

* feat(storage): add new internal remove api (#13880)

feat(internal-remove): add new internal remove api

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore: Add internal getUrl API (#13882)

* feat(storage): internals list API  (#13874)

* feat(storage): add new internal downloadData api (#13887)

* feat(internal-remove): add new internal downloadData api

* code cleanup

* code cleanup

* chore: fix ts doc

---------

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* feat: optional checksum algorithm for upload (#13849)

* feat: opt in checksum

* fix: revert local prettier suggestion

* fix: up size limit for storage upload data

* feat: react native crc32

* fix: up bundle size limit and fix typo

* feat: add documentation for checksumAlgorithm

* fix: update bundle size limit

* fix: update bundle size limit

* fix: address pr feedbacks

* fix: bundle-size limit

---------

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* feat(storage): add remaining copy changes for internals (#13889)

* chore: add remaining copy changes for internals

* chore; explicitly call internal API with params

* chore: copy return type

* chore: feedback changes

* feat(storage): internal uploadData implementation (#13888)

* feat(storage): internal uploadData implementation

* chore: update bundle size

* fix: address feedbacks

* revert: optional checksum algorithm for upload (#13849) (#13910)

This reverts commit 02cb08a.

* fix(storage): internals list function not able to decide which output types to use (#13915)

chore: add function overloading to list

* feat(storage): Add API support for Expected Bucket Owner  (#13914)

* Update Top/Internal API for expected bucket owner feat

---------

Co-authored-by: JoonWon Choi <joonwonc@amazon.com>

* chore: sync main with storage browser integrity branch (#13928)

* chore(release): Set core metadata [skip release]

* chore(release): Publish [skip release]

 - @aws-amplify/adapter-nextjs@1.2.21
 - @aws-amplify/analytics@7.0.51
 - @aws-amplify/api@6.0.53
 - @aws-amplify/api-graphql@4.4.0
 - @aws-amplify/api-rest@4.0.51
 - @aws-amplify/auth@6.5.1
 - aws-amplify@6.6.4
 - @aws-amplify/core@6.4.4
 - @aws-amplify/datastore@5.0.53
 - @aws-amplify/datastore-storage-adapter@2.1.53
 - @aws-amplify/geo@3.0.51
 - @aws-amplify/interactions@6.0.50
 - @aws-amplify/notifications@2.0.51
 - @aws-amplify/predictions@6.1.26
 - @aws-amplify/pubsub@6.1.26
 - @aws-amplify/storage@6.6.9
 - tsc-compliance-test@0.1.56

* chore(release): Update API docs [skip release]

* chore: update contributing guide vite troubleshooting (#13826)

* docs: update contributing guide vite troubleshooting

* Update CONTRIBUTING.md

---------

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* chore(ai): add UpdateConversation enum (#13920)

* chore(release): Set core metadata [skip release]

* chore(release): Publish [skip release]

 - @aws-amplify/adapter-nextjs@1.2.22
 - @aws-amplify/analytics@7.0.52
 - @aws-amplify/api@6.0.54
 - @aws-amplify/api-graphql@4.4.1
 - @aws-amplify/api-rest@4.0.52
 - @aws-amplify/auth@6.5.2
 - aws-amplify@6.6.5
 - @aws-amplify/core@6.4.5
 - @aws-amplify/datastore@5.0.54
 - @aws-amplify/datastore-storage-adapter@2.1.54
 - @aws-amplify/geo@3.0.52
 - @aws-amplify/interactions@6.0.51
 - @aws-amplify/notifications@2.0.52
 - @aws-amplify/predictions@6.1.27
 - @aws-amplify/pubsub@6.1.27
 - @aws-amplify/storage@6.6.10
 - tsc-compliance-test@0.1.57

* chore(release): Update API docs [skip release]

* fix(storage): multipart upload is firing on 0 bytes data (#13927)

* chore: update bundle size

---------

Co-authored-by: erinleigh90 <106691284+erinleigh90@users.noreply.github.com>
Co-authored-by: aws-amplify-bot <aws@amazon.com>
Co-authored-by: Parker Scanlon <69879391+scanlonp@users.noreply.github.com>
Co-authored-by: AllanZhengYP <zheallan@amazon.com>
Co-authored-by: Danny Banks <djb@amazon.com>
Co-authored-by: ashika112 <155593080+ashika112@users.noreply.github.com>
Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com>

* feat: Add expectedBucketOwner to remaining internal APIs (#13932)

feat: Add expectedBucketOwner to remaining internal APIs.

* feat(storage): add advanced option to disable upload cache (#13931)

* feat(storage): set allowedByApp to listCallerAccessGrants (#13934)

* Storage Browser Default Auth (#13866)

* first draft poc

* upadtes

* add listPaths API

* update new file structure

* fix types

* refactor types and utils

* update tests

* fix test

* fix bundle size test

* update the listLocation handler

* rename util

* update Path type

* fix missed type

* chore(ci): add new storage-gen2-internal e2e test (#13916)

Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* chore(storage): remove credential store and managed auth config factory (#13944)

* feat: optional checksum algorithm for upload (#13939)

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* feat: preventOverwrite with if-none-match header (#13954)

Co-authored-by: AllanZhengYP <zheallan@amazon.com>

* Feat: Add pagination to Amplify Default Auth storage Browser (#13897)

* update the listLocation handler
* implement memoization
* add pagination logic
* update usergroup logic & test
* update getPaginated Locations
* fix failing test

* test(storage): refactor unit tests for public & internal facade (#13955)

* Chore: Remove createAmplifyAdapter & refactor (#13974)

remove createAmplifyAdapter & refactor

* update bundle size

* fix bundle size & test

* Fix: retry failure in storage retryDecider (#13977)

fix retry failure

* feat: validate uploaded parts before completing upload (#13763)

Co-authored-by: Donny Wu <wuuxi@amazon.com>
Co-authored-by: Allan Zheng <zheallan@amazon.com>

* feat(storage): add customEndpoint to internal apis in advanced options (#13961)

* feat: add baseEndpoint to advanced options

* feat: add baseEndpoint to customEndpoint

* feat: thread baseEndpoint through resolved config to endpoint resolver

* add customEndpoint advanced option to internals storage data-plane apis

* add customEndpoint advanced option to internals storage control-plane apis

* fix unit test

* code cleanup

* increase bundle size

* wire up customEndpoint on copy API

* increase the bundle size

* add customEndpoint unit tests for all data and control apis

* increase bundle size

* update ts docs

* add additional error unit tests for endpointResolver

* add unit tests for internals/ apis

* code cleanup

* address feedback

* add comment for ForcePathStyleEndpointNotSupported ErrorCode

* increase bundle size

* remove docs links from error recovery message

---------

Co-authored-by: Erin Beal <erinleig@amazon.com>
Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>

* fix(core): support endpoint resolver accepting both input and config(#13985)

* feat(storage): allow checksum algo for internal upload API (#14002)

* feat: allow setting encoding type for list calls

* refactor: move to list internal

* fix: add license

* fix(storage): bug in copy unmodified since to use UTC (#14025)

fix to use UTC and test

* Revert "chore: enable e2e and tagged release (#13848)"

This reverts commit 3dfde04.

* revert tagged release

---------

Co-authored-by: Allan Zheng <zheallan@amazon.com>
Co-authored-by: israx <70438514+israx@users.noreply.github.com>
Co-authored-by: ashika112 <155593080+ashika112@users.noreply.github.com>
Co-authored-by: aws-amplify-bot <aws@amazon.com>
Co-authored-by: Jim Blanchard <jim.l.blanchard@gmail.com>
Co-authored-by: Ashwin Kumar <ashwinkumar2468@gmail.com>
Co-authored-by: Ashwin Kumar <ashwsrir@amazon.com>
Co-authored-by: Jamie Epp <168486127+eppjame@users.noreply.github.com>
Co-authored-by: Jamie Epp <eppjame@amazon.com>
Co-authored-by: Caleb Pollman <cpollman@amazon.com>
Co-authored-by: Donny Wu <wuuxi@amazon.com>
Co-authored-by: Caleb Pollman <cpollman1@gmail.com>
Co-authored-by: Sergio Castillo Yrizales <yriz@amazon.com>
Co-authored-by: Joon Choi <requiemdeciel@gmail.com>
Co-authored-by: JoonWon Choi <joonwonc@amazon.com>
Co-authored-by: erinleigh90 <106691284+erinleigh90@users.noreply.github.com>
Co-authored-by: Parker Scanlon <69879391+scanlonp@users.noreply.github.com>
Co-authored-by: Danny Banks <djb@amazon.com>
Co-authored-by: Hui Zhao <10602282+HuiSF@users.noreply.github.com>
Co-authored-by: ashika112 <akasivis@amazon.com>
Co-authored-by: Erin Beal <erinleig@amazon.com>
Co-authored-by: Pranav Malewadkar <pranavml@amazon.com>
Co-authored-by: Pranav Malewadkar <malewadkar.1@osu.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants