Does S3 sync executes copy in parallel?

[yudho]

Does S3 Sync command execute necessary operations (e.g. cp) in parallel fashion?

[justnance]

@yudho - Thanks for asking. Yes, the S3 Sync command can execute in parallel. There is more information in our docs and a few related issues: #907, #1122, #905, and #658.

Let us know if this helps.

[yudho]

Awesome. Thank you

[arya-coding]

Hi,

I cannot execute two aws sync operation in parallel. I'm trying to upload two folders to the same S3 Bucket that is in a different region than the EC2 with the files.
I tried with cp aswell.

Two case scenarios :

• The first sync start and upload correctly. The second one is on hold and waiting for the first one to finish. They are both on different ssh session.
• The first sync works, and the second, if I use the cp command, return the following error :

s3 upload failed need to rewind .... the stream but stream is not seekable

The version of AWS CLI

aws-cli/2.0.31 Python/3.7.3 Linux/5.3.0-1030-aws botocore/2.0.0dev35

The version of the OS

Distributor ID: Ubuntu
Description: Ubuntu 18.04.4 LTS
Release: 18.04
Codename: bionic