Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amplify v6 Storage - ExpiredToken After 1 Hour Upload #13307

Closed
3 tasks done
minatmalek opened this issue Apr 29, 2024 · 6 comments
Closed
3 tasks done

Amplify v6 Storage - ExpiredToken After 1 Hour Upload #13307

minatmalek opened this issue Apr 29, 2024 · 6 comments
Labels
bug Something isn't working Storage Related to Storage components/category

Comments

@minatmalek
Copy link

minatmalek commented Apr 29, 2024

Before opening, please confirm:

JavaScript Framework

React

Amplify APIs

Storage

Amplify Version

v6

Amplify Categories

auth, storage

Backend

Amplify CLI

Environment information

# Put output below this line


Describe the bug

I am encountering an error message "ExpiredToken: The provided token has expired" when uploading files to Amplify Storage using Amplify v6 (I tried with v5 as well). This error occurs specifically after approximately 1 hour of upload time.

Expected behavior

The file upload should complete successfully regardless of upload duration.

Reproduction steps

1- Configure Amplify project with Storage functionality using Amplify v6.
2- Initiate a file upload process to Amplify Storage.
3- Observe the upload progress for approximately 1 hour.

Code Snippet

// Put your code below this line.
  // Function to upload multiple files in the 'files' folder
  const uploadFiles = useCallback(
    async (filesToUpload) => {
      try {
        setUploading({ state: true, progress: 0 }); // Start the upload process

        const totalSize = filesToUpload.reduce((acc, file) => acc + file.size, 0); // Calculate total size of all files

        const fileProgress = {}; // Track progress of individual files

        const uploadPromises = filesToUpload.map(async (file) => {
          const fileName = file.name;
          const key = `files/${fileName}`;

          const uploadOperation = uploadData({
            key,
            data: file,
            options: {
              accessLevel: 'private',
              contentType: file.type,
              onProgress: ({ transferredBytes }) => {
                // Progress implementation
                fileProgress[fileName] = transferredBytes; // Calculate individual file progress
                const totalFileProgress = Object.values(fileProgress).reduce(
                  (acc, val) => acc + val,
                  0
                );
                const totalProgress = Math.round((totalFileProgress / totalSize) * 100); // Calculate total uploaded bytes based on total progress
                if (totalProgress > 0 && uploading.state) {
                  setUploading({ state: true, progress: totalProgress }); // Update overall progress in state
                }
              },
            },
          });

          return uploadOperation.result;
        });

        await Promise.all(uploadPromises); // Wait for all files to be uploaded

        await listFiles();

        setUploading({ state: false, progress: 100 }); // Set upload state to completed
        enqueueSnackbar('Files uploaded successfully');
      } catch (error) {
        setUploading({ state: false, progress: 0 }); // Reset upload state
        enqueueSnackbar('Error uploading files', { variant: 'error' });
        console.error('Error uploading files:', error);
      }
    },
    [listFiles, enqueueSnackbar, uploading.state]
  );

Log output

// Put your logs below this line
[DEBUG] 52:20.175 xhr-http-handler ProgressEvent {isTrusted: true, lengthComputable: true, loaded: 2424832, total: 5242880, type: 'progress', …}
aws-amplify_storage.js?v=38da3be5:360 PUT https://tutix-storage91744-staging.s3.us-east-1.amazonaws.com/private/us-east-1%3A5510f7b1-11a5-4bf9-aa39-c8b5144b755e/files/Chr20.fastq?partNumber=945&uploadId=LScDnDcb1YDPuU1RuKDeUSjfJfb7PdZSb5EV9J9S565iQ1LT7uE6xA2k6v.9vRz5PxT1Fzqki9BIsSkOMioFTSkmDRZenLnCJktznkJiLUAJ_5dePsLDGEwQnsSpMBu2Pbk5kULuo08cM6NWyXZ6oeIGN6Xhr1vyMsQuGTbjnkk- 400 (Bad Request)
(anonymous) @ aws-amplify_storage.js?v=38da3be5:360
xhrTransferHandler @ aws-amplify_storage.js?v=38da3be5:276
composedHandler @ chunk-RLA2GRRQ.js?v=38da3be5:1400
signingMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:3444
await in signingMiddleware (async)
retryMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:1330
userAgentMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:1391
contentSha256Middleware @ aws-amplify_storage.js?v=38da3be5:244
await in contentSha256Middleware (async)
(anonymous) @ chunk-RLA2GRRQ.js?v=38da3be5:1406
(anonymous) @ chunk-RLA2GRRQ.js?v=38da3be5:1267
await in (anonymous) (async)
uploadPartExecutor @ aws-amplify_storage.js?v=38da3be5:1499
Show 10 more frames
Show less
storage-provider.jsx:150 Error uploading files: ExpiredToken: The provided token has expired.
    at buildStorageServiceError (http://127.0.0.1:3031/node_modules/.vite/deps/aws-amplify_storage.js?v=38da3be5:851:24)
    at uploadPartDeserializer (http://127.0.0.1:3031/node_modules/.vite/deps/aws-amplify_storage.js?v=38da3be5:1169:11)
    at async uploadPartExecutor (http://127.0.0.1:3031/node_modules/.vite/deps/aws-amplify_storage.js?v=38da3be5:1499:30)
    at async Promise.all (index 2)
    at async startUpload (http://127.0.0.1:3031/node_modules/.vite/deps/aws-amplify_storage.js?v=38da3be5:1798:5)
    at async http://127.0.0.1:3031/node_modules/.vite/deps/aws-amplify_storage.js?v=38da3be5:504:22
    at async Promise.all (index 0)
    at async http://127.0.0.1:3031/src/providers/storage/context/storage-provider.jsx:149:13
    at async http://127.0.0.1:3031/src/sections/common/file-manager/view/file-manager-view.jsx:104:9
overrideMethod @ console.js:273
(anonymous) @ storage-provider.jsx:150
setTimeout (async)
(anonymous) @ aws-amplify_storage.js?v=38da3be5:340
XMLHttpRequest.send (async)
(anonymous) @ aws-amplify_storage.js?v=38da3be5:360
xhrTransferHandler @ aws-amplify_storage.js?v=38da3be5:276
composedHandler @ chunk-RLA2GRRQ.js?v=38da3be5:1400
signingMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:3444
await in signingMiddleware (async)
retryMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:1330
userAgentMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:1391
contentSha256Middleware @ aws-amplify_storage.js?v=38da3be5:244
await in contentSha256Middleware (async)
(anonymous) @ chunk-RLA2GRRQ.js?v=38da3be5:1406
(anonymous) @ chunk-RLA2GRRQ.js?v=38da3be5:1267
await in (anonymous) (async)
uploadPartExecutor @ aws-amplify_storage.js?v=38da3be5:1499
Show 12 more frames
Show less
chunk-RLA2GRRQ.js?v=38da3be5:258 [DEBUG] 52:20.675 CognitoCredentialsProvider - Clearing out in-memory credentials
aws-amplify_storage.js?v=38da3be5:360 
       PUT https://tutix-storage91744-staging.s3.us-east-1.amazonaws.com/private/us-east-1%3A5510f7b1-11a5-4bf9-aa39-c8b5144b755e/files/Chr20.fastq?partNumber=914&uploadId=LScDnDcb1YDPuU1RuKDeUSjfJfb7PdZSb5EV9J9S565iQ1LT7uE6xA2k6v.9vRz5PxT1Fzqki9BIsSkOMioFTSkmDRZenLnCJktznkJiLUAJ_5dePsLDGEwQnsSpMBu2Pbk5kULuo08cM6NWyXZ6oeIGN6Xhr1vyMsQuGTbjnkk- 400 (Bad Request)
(anonymous) @ aws-amplify_storage.js?v=38da3be5:360
xhrTransferHandler @ aws-amplify_storage.js?v=38da3be5:276
composedHandler @ chunk-RLA2GRRQ.js?v=38da3be5:1400
signingMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:3444
await in signingMiddleware (async)
retryMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:1330
userAgentMiddleware @ chunk-RLA2GRRQ.js?v=38da3be5:1391
contentSha256Middleware @ aws-amplify_storage.js?v=38da3be5:244
await in contentSha256Middleware (async)
(anonymous) @ chunk-RLA2GRRQ.js?v=38da3be5:1406
(anonymous) @ chunk-RLA2GRRQ.js?v=38da3be5:1267
await in (anonymous) (async)
uploadPartExecutor @ aws-amplify_storage.js?v=38da3be5:1499
Show 10 more frames
Show less
chunk-RLA2GRRQ.js?v=38da3be5:261 [DEBUG] 52:24.952 xhr-http-handler ProgressEvent {isTrusted: true, lengthComputable: true, loaded: 704512, total: 5242880, type: 'progress', …}
chunk-RLA2GRRQ.js?v=38da3be5:261 [DEBUG] 52:25.52 xhr-http-handler ProgressEvent {isTrusted: true, lengthComputable: true, loaded: 753664, total: 5242880, type: 'progress', …}
chunk-RLA2GRRQ.js?v=38da3be5:261 [DEBUG] 52:25.169 xhr-http-handler ProgressEvent {isTrusted: true, lengthComputable: true, loaded: 802816, total: 5242880, type: 'progress', …}
chunk-RLA2GRRQ.js?v=38da3be5:258 [DEBUG] 52:25.213 CognitoCredentialsProvider - returning stored credentials as they neither past TTL nor expired.

aws-exports.js

{
"aws_project_region": "us-east-1",
"aws_cognito_identity_pool_id": "us-east-1:e21c4017-5126-4feb-ac6a-XXXXXXXX",
"aws_cognito_region": "us-east-1",
"aws_user_pools_id": "us-east-1_XXXXXXXX",
"aws_user_pools_web_client_id": "XXXXXXXX",
"oauth": {},
"aws_cognito_username_attributes": [
"EMAIL"
],
"aws_cognito_social_providers": [],
"aws_cognito_signup_attributes": [
"GIVEN_NAME",
"FAMILY_NAME",
"EMAIL"
],
"aws_cognito_mfa_configuration": "OFF",
"aws_cognito_mfa_types": [
"SMS"
],
"aws_cognito_password_protection_settings": {
"passwordPolicyMinLength": 8,
"passwordPolicyCharacters": [
"REQUIRES_LOWERCASE",
"REQUIRES_NUMBERS",
"REQUIRES_SYMBOLS",
"REQUIRES_UPPERCASE"
]
},
"aws_cognito_verification_mechanisms": [
"EMAIL"
],
"aws_user_files_s3_bucket": "XXXXXXXX-staging",
"aws_user_files_s3_bucket_region": "us-east-1"
}

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

image

@minatmalek minatmalek added the pending-triage Issue is pending triage label Apr 29, 2024
@cwomack cwomack self-assigned this Apr 29, 2024
@cwomack cwomack added the Storage Related to Storage components/category label Apr 29, 2024
@cwomack
Copy link
Member

cwomack commented Apr 29, 2024

Hello, @minatmalek 👋. Can you clarify if the upload is stopping completely when this occurs (or if the file still uploads)? Also, do you see this error happening repeatedly or is this more intermittent? It may also be helpful to know what the file size is and the amount of time it's taking to upload the file if you can share that.

@cwomack cwomack added pending-response and removed pending-triage Issue is pending triage labels Apr 29, 2024
@minatmalek
Copy link
Author

minatmalek commented Apr 29, 2024

Hi @cwomack , thank you for your quick reply.
File size is around 10GB, it always fails at the 1 hour mark (around 45% of the upload)

I notice from the logs that the progress callback fires a couple of times after the ExpiredToken error occurred then it stops completely with a 400 error.

What I also noticed from the logs is that it clears the credentials after they expire (not right before they do), I'm not sure if thats the expected behavior.

@AllanZhengYP
Copy link
Member

Hi @minatmalek I can reproduce this issue thus confirm this is a bug. The team is prioritizing fixing it. We will keep you post with our progress.

@cwomack cwomack added the bug Something isn't working label Apr 29, 2024
@AllanZhengYP AllanZhengYP added the bug Something isn't working label Apr 29, 2024
@cwomack cwomack removed their assignment Apr 29, 2024
@cwomack cwomack added question General question pending-response bug Something isn't working and removed bug Something isn't working question General question pending-response labels Apr 30, 2024
@vinu-ablabs
Copy link

I am having the same issue in production. In my case, the user leaves the screen idle or even interacts with it. After an hour from the last refresh, the upload fails with the credentials expiring. Right now, when this happens, we force the user to logout. But that isn't right and very annoying.

I am using v5 of sdk. Is there a way to work around the issue with the approach in the PR. I have been trying to refresh the credentials before the upload but not been able to do so.

@ashwinkumar6
Copy link
Member

Hi @minatmalek, thanks for reporting the issue, it has been released in aws-amplify@6.3.3.
Please feel free to let us know if it's resolved.

@minatmalek
Copy link
Author

Hi @ashwinkumar6, thank you for the support. I can confirm that the issue has been resolved in aws-amplify@6.3.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Storage Related to Storage components/category
Projects
None yet
Development

No branches or pull requests

5 participants