Paradox on sam deploy and s3 bucket creation

[xpdable]

Description:

I am to automate all serverless CICD process. So am to do sam build then sam deploy
Local PoC of deployment with first sam deploy --guided will create the aws-sam-cli-managed-default stack which create the coresponding S3
However, the process will be non-interactive mode in my CICD projects amon 40+ env/aws account.
Therefore we go directly sam build & sam deploy or sam deploy --config-file samconfig.toml which configure file is written previously with or without a S3 bucket definition.
Then the deploy will failed to find the S3 bucket

S3 Bucket does not exist.

Steps to reproduce:

1. Ensure your env is clear, no aws-sam-cli-managed-default bucket, not coresponding S3, no sam stack.
2. Execute sam build, then sam deploy
3. Or execute sam buid then sam deploy --config-file samconfig.toml with below snippet of samconfig.toml

version = 0.1

[dev]
[dev.deploy]
[dev.deploy.parameters]
stack_name = "SharedService-Dev-Serverless-OAP3-Stack"
# s3_bucket = "aws-sam-cli-managed-default-samclisourcebucket-1e6nti2oditbe"
s3_prefix = "SharedService-Dev-Serverless-OAP3-Stack"
region = "cn-north-1"
profile = "share-non-prod"
capabilities = "CAPABILITY_IAM"
parameter_overrides = "Environment=\"dev\""

Observed result:

 sam deploy
Error: Unable to upload artifact xxx referenced by CodeUri parameter of xxxx resource.

S3 Bucket does not exist.

Expected result:

I thought the stack aws-sam-cli-managed-default will also be create without --guided.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: MacOS
2. sam --version: SAM CLI, version 1.24.0
3. AWS region: cn-north-1

Add --debug flag to command you are running

sam deploy --config-file samconfig.toml --config-env dev --debug
2021-06-22 13:16:26,985 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2021-06-22 13:16:26,986 | Using config file: samconfig.toml, config environment: dev
2021-06-22 13:16:26,986 | Expand command line arguments to:
2021-06-22 13:16:26,986 | --template_file=/Users/xiaopeng/workspace/serverless/itt-jira-service-desk-dispatcher/.aws-sam/build/template.yaml --stack_name=SharedService-Dev-Serverless-OAP3-Stack --s3_prefix=SharedService-Dev-Serverless-OAP3-Stack --fail_on_empty_changeset --parameter_overrides={'Environment': 'dev'} --capabilities=['CAPABILITY_IAM']
2021-06-22 13:16:28,604 | Unable to export
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/samcli/lib/package/packageable_resources.py", line 121, in export
    self.do_export(resource_id, resource_dict, parent_dir)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/samcli/lib/package/packageable_resources.py", line 143, in do_export
    uploaded_url = upload_local_artifacts(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/samcli/lib/package/utils.py", line 172, in upload_local_artifacts
    return zip_and_upload(local_path, uploader, extension)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/samcli/lib/package/utils.py", line 190, in zip_and_upload
    return uploader.upload_with_dedup(zip_file, precomputed_md5=md5_hash, extension=extension)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/samcli/lib/package/s3_uploader.py", line 145, in upload_with_dedup
    return self.upload(file_name, remote_path)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/samcli/lib/package/s3_uploader.py", line 87, in upload
    if not self.force_upload and self.file_exists(remote_path):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/samcli/lib/package/s3_uploader.py", line 158, in file_exists
    raise BucketNotSpecifiedError()
samcli.commands.package.exceptions.BucketNotSpecifiedError:
S3 Bucket not specified, use --s3-bucket to specify a bucket name or run sam deploy --guided
2021-06-22 13:16:28,608 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': 'f7eb530c-7610-4727-8a5b-5570f20f3c8c', 'installationId': 'ead60707-ccc7-42dc-961d-cc6123afbb9d', 'sessionId': '90e43424-444f-4c81-8ef6-5d29374d620b', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.9', 'samcliVersion': '1.24.0', 'awsProfileProvided': True, 'debugFlagProvided': True, 'region': 'cn-north-1', 'commandName': 'sam deploy', 'duration': 1621, 'exitReason': 'ExportFailedError', 'exitCode': 1}}]}
2021-06-22 13:16:30,240 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: Unable to upload artifact xxx referenced by CodeUri parameter of xxx resource.

[qingchm]

Hey @xpdable actually the use case you are referring to is very well doable. The aws-sam-cli-managed-default bucket is not only supported in guided deployment. Actually you can use this bucket simply by using the --resolve-s3 flag. For example, sam deploy --resolve-s3, this way it won't error even if you don't provide any s3 locations in samconfig or command line ;)

[xpdable]

Hey @xpdable actually the use case you are referring to is very well doable. The aws-sam-cli-managed-default bucket is not only supported in guided deployment. Actually you can use this bucket simply by using the --resolve-s3 flag. For example, sam deploy --resolve-s3, this way it won't error even if you don't provide any s3 locations in samconfig or command line ;)

tried with

sam deploy --resolve-s3 --config-file samconfig.toml --config-env dev

And confirmed working correctly.
Thanks bro @qingchm

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.