Error putting S3 ACL: NoSuchBucket: The specified bucket does not exist #10121

abguy · 2016-11-15T05:40:02Z

My test deployment periodically failed with Error putting S3 ACL: NoSuchBucket: The specified bucket does not exist message. It looks like there is an issue with dependencies.
I actually had to add an additional attempt to my deployment script, but it fails sometimes anyway.

Some parts of my deployment script with workaround:

PLAN="/tmp/deployment.tfplan"
ARGS="-var build_number=237 -var branch=dc2ff0f3e435da28181c79d6fb556701f47f2e7c -var-file=mode/migration-demo.tfvars -out=$PLAN"
#...
    echo "Recreate S3 buckets..."
    if /opt/terraform/terraform show | grep -q "aws_s3_bucket.private"; then
        /opt/terraform/terraform taint aws_s3_bucket.private
    fi
    if /opt/terraform/terraform show | grep -q "aws_s3_bucket.public"; then
        /opt/terraform/terraform taint aws_s3_bucket.public
    fi
    /opt/terraform/terraform plan $ARGS
    /opt/terraform/terraform apply $PLAN || (sleep 5 && /opt/terraform/terraform apply $PLAN)

Despite the workaround it fails sometimes. I see this failure several times per a week. Let me know if you need additional details and I will help to reproduce this issue.

The part of my configuration is main.tf
My Jenkins build console output with debug enabled console.log

ubuntu@bastion:~$ terraform --version
Terraform v0.7.10

The text was updated successfully, but these errors were encountered:

ctindel · 2016-12-27T20:53:00Z

I am also seeing this error consistently, except mine is when trying to set the versioning:

aws_s3_bucket.s3_sa_bucket: Error putting S3 versioning: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: CEF088D52B17ABA8

abguy · 2017-01-03T10:44:22Z

Are there any ideas? I still see this issue periodically. It is really annoying.

ozbillwang · 2017-03-08T04:39:31Z

In my case, I DO manually delete some buckets, I saw the similar issue. But the log output is not clear enough, I turn on debug option, I still don't know which bucket is missed.

I used to re-create the bucket with same name to avoid this error, but now there are many buckets I am not sure which bucket I need restore.

stack72 · 2017-03-28T19:46:54Z

Hi Friends

Is this still an issue? I see we are handling the 404 on buckets now and give better error messages:

if err != nil {
		if awsError, ok := err.(awserr.RequestFailure); ok && awsError.StatusCode() == 404 {
			log.Printf("[WARN] S3 Bucket (%s) not found, error code (404)", d.Id())
			d.SetId("")
			return nil
		} else {
			// some of the AWS SDK's errors can be empty strings, so let's add
			// some additional context.
			return fmt.Errorf("error reading S3 bucket \"%s\": %s", d.Id(), err)
		}
	}

I believe this is all ok now

Paul

ozbillwang · 2017-03-28T22:17:25Z

@stack72

Thanks, after upgrade to v0.9.1, I saw the bucket name within the error message. From my view, this issue can be closed

abguy · 2017-03-29T02:23:36Z

I'll upgrade soon and check. We still have these issues with v0.8.8.
Hope it is really fixed now!

ozbillwang · 2017-03-29T05:48:39Z

@abguy

I can confirm. This issue is not fixed with version 0.8.8

abguy · 2017-03-29T07:29:07Z

@SydOps
Yes, I am going to check how it works with v0.9.2

abguy · 2017-03-29T23:24:29Z

It still doesn't work. v0.9.2:

The Terraform execution plan has been generated and is shown below.
...
-/+ aws_s3_bucket.public (tainted)
...
aws_s3_bucket.public: Destruction complete
aws_s3_bucket.public: Creating...
...
Error applying plan:

1 error(s) occurred:

* aws_s3_bucket.public: 1 error(s) occurred:

* aws_s3_bucket.public: NoSuchBucket: The specified bucket does not exist
	status code: 404, request id: 1FEBE90552364E40

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.
Failed to load backend: This plan was created against an older state than is current. Please create
a new plan file against the latest state and try again.

Terraform doesn't allow you to run plans that were created from older
states since it doesn't properly represent the latest changes Terraform
may have made, and can result in unsafe behavior.

Plan Serial:    4335
Current Serial: 4337

davidhesson · 2017-04-24T20:16:54Z

I'm seeing this issue in 0.9.3 (tries to set bucket versioning and gets a 404)

Running terraform again works, because the bucket was made.

cemo · 2017-05-25T11:45:48Z

@stack72, this issue is still valid.

cemo · 2017-05-26T06:56:53Z

@stack72 My problem was related to my mistake. My s3 module was using like this

variable "bucket" {}

output "bucket" {
  value = "${ var.bucket }"
}

This was, of course, causing early return without waiting s3 bucket to be created. I replaced it with this:

output "bucket" {
  value = "${ aws_s3_bucket.tls.bucket }"
}

Now my problem has resolved.

prasaddabbiru · 2018-05-12T11:53:37Z

Error inspecting states in the "s3" backend:
NoSuchBucket: The specified bucket does not exist
status code: 404, request id: CFC0880652E0C7B0,

I have got he same error while setting s3 bucket in the backend.
Terraform v0.11.7 - this is the terraform version. Can someone tell me what could be the issue?

batmanhit · 2018-06-13T13:45:14Z

I also have this issue with Terraform v0.11.7.
Bucket exists in the s3 but I am getting
Error inspecting states in the "s3" backend: NoSuchBucket: The specified bucket does not exist status code: 404, request id: C9220D8E2F881CCA, host id: 6QA9xH6QMgNql/vNnQy3TWToEj2neOob0V/yyeBjd9cewTPl5LkGPzdkjmc9EkwDn+m012ulKw0=

ozbillwang · 2018-06-14T02:31:24Z

I am curious for anyone who still have this issue, did you run with the latest terraform provider aws releases?

upgrade it, then check again.

shashanktomar · 2018-07-11T05:31:32Z

I tried it with the latest terraform provider aws releases and it still didn't work. Deleting the .terraform folder and running init again worked for me.

ozbillwang · 2018-07-11T05:42:38Z

@shashanktomar

I run with a wrapper script and always delete .terraform before any terraform commands.

batmanhit · 2018-07-11T05:43:54Z

Removing .terraform folder solved my issue as well.

oonisim · 2018-08-25T13:42:14Z

It looks this has not been fixed. Please consider reopening.

Removing .terraform did not work. Run destroy everytime before apply to be sure.

Terraform v0.11.7
+ provider.archive v1.1.0
+ provider.aws v1.33.0
+ provider.local v1.1.0
+ provider.null v1.0.0
+ provider.template v1.0.0

Error messages

Sporadic and not consistent.

* aws_s3_bucket.transcoded: 1 error(s) occurred:
* aws_s3_bucket.transcoded: Error putting S3 lifecycle: NoSuchBucket: The specified bucket does not exist
	status code: 404, request id: A0432B3628100A69, host id: bxydwJW36F9NQDUN/27R7xgE6EhH4U52mAgws6gAqus6Vz9MAEwPph/5TS7gqJitbbxCk3+lfPk=

Or

1 error(s) occurred:

* aws_s3_bucket.package: 1 error(s) occurred:

* aws_s3_bucket.package: Error putting S3 lifecycle: NoSuchBucket: The specified bucket does not exist
	status code: 404, request id: A74931D58CF315C2, host id: IeJI6yOCywGttxxpQ0LRmPwSShLcHOLWH9DL6bJFenUCIyQWJO9mtNxRqR6Hj9+fyRQQSUQ4RLM=

Or

* aws_s3_bucket.transcoded: 1 error(s) occurred:

* aws_s3_bucket.transcoded: Error putting S3 lifecycle: NoSuchBucket: The specified bucket does not exist
	status code: 404, request id: F88C267818FC2D85, host id: B/N+Zw5ENN0AUCThbB4OtDWlNcM9BtUo4hPVZ4x1hHceyWCd07yGfhuerf1cV5JFP/25si9Na+c=
* aws_s3_bucket.package: 1 error(s) occurred:

* aws_s3_bucket.package: Error putting S3 lifecycle: NoSuchBucket: The specified bucket does not exist
	status code: 404, request id: 6E0EDA4BB0B6DF52, host id: npaZMz5ml5otfwAPhBlojj9EyXTyZr7HFTyQoubx8ZNWlri+XLLkDr0KzxFbOuicirRA21+HPK8=

Terraform to create S3 buckets

Running s3 bucket creation alone does not cause the issue. I have other resources depends on the S3 buckets e.g. lambda, S3 to SNS publish, etc.

#--------------------------------------------------------------------------------
# S3 bucket to upload video files to be transcoded
#--------------------------------------------------------------------------------
resource "aws_s3_bucket" "uploaded" {
  region = "${var.aws_region}"
  bucket = "${var.bucket_uploaded_name}"
  acl    = "private"
  force_destroy = true
  tags {
    Project = "${var.project}"
  }
  lifecycle_rule {
    id      = "uploaded"
    enabled = true
    tags {
      Project = "${var.project}"
    }
    expiration {
      days = 1
    }
  }
}
#--------------------------------------------------------------------------------
# S3 bucket for Elastic Transcoder to place transcoded video files.
#--------------------------------------------------------------------------------
resource "aws_s3_bucket" "transcoded" {
  region = "${var.aws_region}"
  bucket = "${var.bucket_transcoded_name}"
  force_destroy = true
  tags {
    Project = "${var.project}"
  }
  lifecycle_rule {
    id      = "transcoded"
    enabled = true
    tags {
      Project = "${var.project}"
    }
    transition {
      days          = 1
      storage_class = "GLACIER"
    }
  }
}
#--------------------------------------------------------------------------------
# S3 bucket to upload lambda packages
#--------------------------------------------------------------------------------
resource "aws_s3_bucket" "package" {
  region = "${var.aws_region}"
  bucket = "${var.bucket_package_name}"
  acl    = "private"
  force_destroy = true
  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        sse_algorithm = "AES256"
      }
    }
  }
  versioning {
    enabled = true
  }
  tags {
    Project = "${var.project}"
  }
  lifecycle_rule {
    id      = "package"
    enabled = true
    tags {
      Project = "${var.project}"
    }
    noncurrent_version_transition {
      days          = 1
      storage_class = "GLACIER"
    }
  }
  /*
  provisioner "local-exec" {
    command = "aws s3api put-object --bucket ${aws_s3_bucket.package.bucket} --key ${var.lambda_metadata_archive} --body ${data.archive_file.lambda_metadata.output_path}"
  }
*/
}

elgordino · 2018-12-30T10:42:52Z

I am also intermittently seeing the same error

Error putting S3 lifecycle: NoSuchBucket: The specified bucket does not exist

Can this issue be reopened?

This is my config

resource "aws_s3_bucket" "xxxxx" {
  bucket = "bucket-name-xxxxx"
  acl    = "private"

  force_destroy = true

  lifecycle_rule {
    enabled = true
    abort_incomplete_multipart_upload_days = 14
    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }
    expiration {
      days = 365
    }
  }

  server_side_encryption_configuration {
    rule {
      apply_server_side_encryption_by_default {
        sse_algorithm = "AES256"
      }
    }
  }

  tags {
    cost = "${var.cost}"
    role = "logs"
    branch = "${var.environment}"
    service = "xxxxx"
  }
}

apparentlymart · 2018-12-31T00:23:54Z

The AWS provider is no longer developed in this repository. Please report issues with it in its own repository.

abguy mentioned this issue Nov 15, 2016

Dependencies Don't Wait for Provisioners to Complete #5595

Closed

catsby added bug provider/aws labels Nov 15, 2016

hashibot closed this as completed Jun 13, 2017

hashibot mentioned this issue Jun 13, 2017

Error putting S3 ACL: NoSuchBucket: The specified bucket does not exist hashicorp/terraform-provider-aws#372

Closed

hashicorp locked as off-topic and limited conversation to collaborators Dec 31, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error putting S3 ACL: NoSuchBucket: The specified bucket does not exist #10121

Error putting S3 ACL: NoSuchBucket: The specified bucket does not exist #10121

abguy commented Nov 15, 2016

ctindel commented Dec 27, 2016

abguy commented Jan 3, 2017

ozbillwang commented Mar 8, 2017 •

edited

Loading

stack72 commented Mar 28, 2017

ozbillwang commented Mar 28, 2017

abguy commented Mar 29, 2017

ozbillwang commented Mar 29, 2017

abguy commented Mar 29, 2017

abguy commented Mar 29, 2017

davidhesson commented Apr 24, 2017 •

edited

Loading

cemo commented May 25, 2017

cemo commented May 26, 2017 •

edited

Loading

prasaddabbiru commented May 12, 2018

batmanhit commented Jun 13, 2018

ozbillwang commented Jun 14, 2018 •

edited

Loading

shashanktomar commented Jul 11, 2018

ozbillwang commented Jul 11, 2018

batmanhit commented Jul 11, 2018

oonisim commented Aug 25, 2018 •

edited

Loading

elgordino commented Dec 30, 2018

apparentlymart commented Dec 31, 2018

Error putting S3 ACL: NoSuchBucket: The specified bucket does not exist #10121

Error putting S3 ACL: NoSuchBucket: The specified bucket does not exist #10121

Comments

abguy commented Nov 15, 2016

ctindel commented Dec 27, 2016

abguy commented Jan 3, 2017

ozbillwang commented Mar 8, 2017 • edited Loading

stack72 commented Mar 28, 2017

ozbillwang commented Mar 28, 2017

abguy commented Mar 29, 2017

ozbillwang commented Mar 29, 2017

abguy commented Mar 29, 2017

abguy commented Mar 29, 2017

davidhesson commented Apr 24, 2017 • edited Loading

cemo commented May 25, 2017

cemo commented May 26, 2017 • edited Loading

prasaddabbiru commented May 12, 2018

batmanhit commented Jun 13, 2018

ozbillwang commented Jun 14, 2018 • edited Loading

shashanktomar commented Jul 11, 2018

ozbillwang commented Jul 11, 2018

batmanhit commented Jul 11, 2018

oonisim commented Aug 25, 2018 • edited Loading

Error messages

Terraform to create S3 buckets

elgordino commented Dec 30, 2018

apparentlymart commented Dec 31, 2018

ozbillwang commented Mar 8, 2017 •

edited

Loading

davidhesson commented Apr 24, 2017 •

edited

Loading

cemo commented May 26, 2017 •

edited

Loading

ozbillwang commented Jun 14, 2018 •

edited

Loading

oonisim commented Aug 25, 2018 •

edited

Loading