Skip to content
This repository has been archived by the owner on Aug 11, 2021. It is now read-only.

Provider produced inconsistent result after apply #90

Closed
blucas opened this issue Jul 23, 2020 · 8 comments
Closed

Provider produced inconsistent result after apply #90

blucas opened this issue Jul 23, 2020 · 8 comments
Labels
acknowledged bug Something isn't working

Comments

@blucas
Copy link

blucas commented Jul 23, 2020

Terraform Version and Provider Version

➜ terraform -v
Terraform v0.13.0-rc1
+ provider local.example.com/local/kubernetes-alpha v0.1.0

Kubernetes Version

➜ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.5", GitCommit:"20c265fef0741dd71a66480e35bd69f18351daea", GitTreeState:"clean", BuildDate:"2019-10-15T19:16:51Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17+", GitVersion:"v1.17.7-gke.15", GitCommit:"7c750a9c53b41ac5ebf1806473bf9d8cebb9c2bf", GitTreeState:"clean", BuildDate:"2020-07-08T23:13:47Z", GoVersion:"go1.13.9b4", Compiler:"gc", Platform:"linux/amd64"}

Affected Resource(s)

  • resource "kubernetes_manifest" "service-productpage"

Terraform Configuration Files

provider "kubernetes-alpha" {
  config_path = "~/.kube/config" // path to kubeconfig
  version     = "0.1.0"
}

resource "kubernetes_manifest" "service-productpage" {
  provider = kubernetes-alpha

  manifest = {
    "apiVersion" = "v1"
    "kind"       = "Service"
    "metadata" = {
      "labels" = {
        "app"     = "productpage"
        "service" = "productpage"
      }
      "name" = "productpage"
      "namespace" = "default"
    }
    "spec" = {
      "ports" = [
        {
          "name" = "http"
          "port" = 9080
          "protocol" = "TCP"
          "targetPort" = 9080
        },
      ]
      "selector" = {
        "app" = "productpage"
      }
    }
  }
}

Debug Output

Panic Output

Expected Behavior

Apply should have succeeded without error

Actual Behavior

Resource was created, but apply failed with error. Attempting a second apply fails as well with another error (see "Steps to Reproduce")

Steps to Reproduce

➜ terraform apply
kubernetes_manifest.test-namespace: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # kubernetes_manifest.service-productpage will be created
  + resource "kubernetes_manifest" "service-productpage" {
      + manifest = {
          + apiVersion = "v1"
          + kind       = "Service"
          + metadata   = {
              + labels    = {
                  + app     = "productpage"
                  + service = "productpage"
                }
              + name      = "productpage"
              + namespace = "default"
            }
          + spec       = {
              + ports    = [
                  + {
                      + name       = "http"
                      + port       = 9080
                      + protocol   = "TCP"
                      + targetPort = 9080
                    },
                ]
              + selector = {
                  + app = "productpage"
                }
            }
        }
      + object   = {
          + apiVersion = "v1"
          + kind       = "Service"
          + metadata   = {
              + annotations = {
                  + cloud.google.com/neg = jsonencode(
                        {
                          + ingress = true
                        }
                    )
                }
              + labels      = {
                  + app     = "productpage"
                  + service = "productpage"
                }
              + name        = "productpage"
              + namespace   = "default"
            }
          + spec       = {
              + ports           = [
                  + {
                      + name       = "http"
                      + port       = 9080
                      + protocol   = "TCP"
                      + targetPort = 9080
                    },
                ]
              + selector        = {
                  + app = "productpage"
                }
              + sessionAffinity = "None"
              + type            = "ClusterIP"
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

kubernetes_manifest.service-productpage: Creating...

Error: Provider produced inconsistent result after apply

When applying changes to kubernetes_manifest.service-productpage, provider
"local.example.com/local/kubernetes-alpha" produced an unexpected new value:
.object: wrong final value type: attribute "apiVersion": string required.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Subsequent apply fails:

➜ terraform apply
kubernetes_manifest.test-namespace: Refreshing state...
kubernetes_manifest.service-productpage: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # kubernetes_manifest.service-productpage is tainted, so must be replaced
-/+ resource "kubernetes_manifest" "service-productpage" {
        manifest = {
            apiVersion = "v1"
            kind       = "Service"
            metadata   = {
                labels    = {
                    app     = "productpage"
                    service = "productpage"
                }
                name      = "productpage"
                namespace = "default"
            }
            spec       = {
                ports    = [
                    {
                        name       = "http"
                        port       = 9080
                        protocol   = "TCP"
                        targetPort = 9080
                    },
                ]
                selector = {
                    app = "productpage"
                }
            }
        }
      ~ object   = {
            apiVersion = "v1"
            kind       = "Service"
          ~ metadata   = {
              - annotations = {
                  - cloud.google.com/neg = jsonencode(
                        {
                          - ingress = true
                        }
                    )
                } -> null
                labels      = {
                    app     = "productpage"
                    service = "productpage"
                }
                name        = "productpage"
                namespace   = "default"
            }
            spec       = {
                clusterIP       = "10.26.47.117"
                ports           = [
                    {
                        name       = "http"
                        port       = 9080
                        protocol   = "TCP"
                        targetPort = 9080
                    },
                ]
                selector        = {
                    app = "productpage"
                }
                sessionAffinity = "None"
                type            = "ClusterIP"
            }
        }
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

kubernetes_manifest.service-productpage: Destroying...
kubernetes_manifest.service-productpage: Destruction complete after 0s

Error: Provider produced inconsistent final plan

When expanding the plan for kubernetes_manifest.service-productpage to include
new values learned so far during apply, provider
"local.example.com/local/kubernetes-alpha" produced an invalid new value for
.object: wrong final value type: attribute "apiVersion": string required.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Important Factoids

References

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment
@blucas blucas added the bug Something isn't working label Jul 23, 2020
@blucas
Copy link
Author

blucas commented Jul 23, 2020

I've also tried using the example service. It has the same issue. I've also tried terraform 0.12.19 and the same error occurs.

➜ terraform apply
kubernetes_manifest.test-namespace: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # kubernetes_manifest.service-injector will be created
  + resource "kubernetes_manifest" "service-injector" {
      + manifest = {
          + apiVersion = "v1"
          + kind       = "Service"
          + metadata   = {
              + labels    = {
                  + app.kubernetes.io/instance   = "test-service"
                  + app.kubernetes.io/managed-by = "Terraform"
                  + app.kubernetes.io/name       = "vault-agent-injector"
                }
              + name      = "test-service-vault-agent-injector-svc"
              + namespace = "default"
            }
          + spec       = {
              + ports    = [
                  + {
                      + port       = 443
                      + protocol   = "TCP"
                      + targetPort = "http"
                    },
                ]
              + selector = {
                  + app.kubernetes.io/instance = "test-service"
                  + app.kubernetes.io/name     = "vault-agent-injector"
                  + component                  = "webhook"
                }
            }
        }
      + object   = {
          + apiVersion = "v1"
          + kind       = "Service"
          + metadata   = {
              + annotations = {
                  + cloud.google.com/neg = jsonencode(
                        {
                          + ingress = true
                        }
                    )
                }
              + labels      = {
                  + app.kubernetes.io/instance   = "test-service"
                  + app.kubernetes.io/managed-by = "Terraform"
                  + app.kubernetes.io/name       = "vault-agent-injector"
                }
              + name        = "test-service-vault-agent-injector-svc"
              + namespace   = "default"
            }
          + spec       = {
              + ports           = [
                  + {
                      + port       = 443
                      + protocol   = "TCP"
                      + targetPort = "http"
                    },
                ]
              + selector        = {
                  + app.kubernetes.io/instance = "test-service"
                  + app.kubernetes.io/name     = "vault-agent-injector"
                  + component                  = "webhook"
                }
              + sessionAffinity = "None"
              + type            = "ClusterIP"
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

kubernetes_manifest.service-injector: Creating...

Error: Provider produced inconsistent result after apply

When applying changes to kubernetes_manifest.service-injector, provider
"registry.terraform.io/-/kubernetes-alpha" produced an unexpected new value
for .object: wrong final value type: attribute "apiVersion": string required.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

@alexsomesan
Copy link
Member

Thanks for reporting this! I'm looking into it.

@alexsomesan
Copy link
Member

This should start working once #41 is merged. I tested it already in that branch and it works correcly.

@blucas
Copy link
Author

blucas commented Jul 24, 2020

@alexsomesan Thanks for the update and the work so far on this provider. Do you guys have any ETA on #41 or general roadmap for the provider? It seems the current version of this provider is pretty much unusable at this stage without that change. I'm not trying to be negative, but rather trying to identify whether or not it is feasible to use, or plan to use, this provider any time soon.

@aareet
Copy link
Member

aareet commented Aug 27, 2020

Hi @blucas, our desire is to have this provider reach stability for production usage as soon as we are able, however #41 is proving to be quite complex to implement. We are doing our best to resolve the issues preventing us from merging that PR but unfortunately cannot share an ETA currently since investigation is still underway. We will be sure to publish any updates as we have them. Thank you for your patience.

@Kun-Kun
Copy link

Kun-Kun commented Aug 31, 2020

I've got a problem at the generation execution plan for the Service:

Terraform v0.12.29
Configuring remote state backend...
Initializing Terraform configuration...
2020/08/31 22:09:56 [DEBUG] Using modified User-Agent: Terraform/0.12.29 TFC/a6c47d4486
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

kubernetes_manifest.config-nfs-server-service: Refreshing state...
kubernetes_manifest.test-configmap: Refreshing state...
kubernetes_manifest.config-persistent-volume-nfs: Refreshing state...
kubernetes_manifest.config-nfs-server: Refreshing state...

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

panic: interface conversion: interface {} is *cty.unknownType, not bool

goroutine 42 [running]:
github.com/zclconf/go-cty/cty.Value.True(0x2469f40, 0xc00009c391, 0x1bebc20, 0x367cb20, 0x246a040)
	/go/pkg/mod/github.com/zclconf/go-cty@v1.2.1/cty/value_ops.go:355 +0x12c
github.com/hashicorp/terraform/command/format.ctyEqualWithUnknown(0x246a040, 0xc0000a8e88, 0x1bd0d80, 0xc0008412f0, 0x246a040, 0xc0000a9200, 0x1bd0d80, 0xc0007c6630, 0x246a040)
	/home/circleci/project/project/command/format/diff.go:1123 +0x10b
github.com/hashicorp/terraform/command/format.(*blockBodyDiffPrinter).writeAttrDiff(0xc000647868, 0xc000206428, 0x6, 0xc0006bf680, 0x246a040, 0xc0000a8e88, 0x1bd0d80, 0xc0008412f0, 0x246a040, 0xc0000a9200, ...)
	/home/circleci/project/project/command/format/diff.go:220 +0x5d0
github.com/hashicorp/terraform/command/format.(*blockBodyDiffPrinter).writeBlockBodyDiff(0xc000647868, 0xc00023e760, 0x246a040, 0xc0000a9210, 0x1bd0d80, 0xc0007c66f0, 0x246a040, 0xc0000a9218, 0x1bd0d80, 0xc0007c6780, ...)
	/home/circleci/project/project/command/format/diff.go:181 +0x693
github.com/hashicorp/terraform/command/format.ResourceChange(0xc00084e5a0, 0x0, 0xc00023e760, 0xc0004f3f30, 0x4d, 0xc00003eec0)
	/home/circleci/project/project/command/format/diff.go:125 +0x626
github.com/hashicorp/terraform/backend/local.RenderPlan(0xc000595f80, 0xc0000a83d0, 0xc00051bc20, 0x247dee0, 0xc00000d5c0, 0xc0004f3f30)
	/home/circleci/project/project/backend/local/backend_plan.go:290 +0x6d9
github.com/hashicorp/terraform/backend/local.(*Local).renderPlan(0xc0005ef110, 0xc000595f80, 0xc0000a83d0, 0xc00051bc20)
	/home/circleci/project/project/backend/local/backend_plan.go:198 +0x68
github.com/hashicorp/terraform/backend/local.(*Local).opPlan(0xc0005ef110, 0x2468440, 0xc000682e80, 0x2468440, 0xc000682ec0, 0xc0004b7c00, 0xc000682e40)
	/home/circleci/project/project/backend/local/backend_plan.go:170 +0xc6f
github.com/hashicorp/terraform/backend/local.(*Local).Operation.func1(0xc0005018b0, 0xc0005018c0, 0xc0005018d0, 0xc0004b7c00, 0xc0005ef110, 0xc000682e40, 0xc0005018a0, 0x2468440, 0xc000682e80, 0x2468440, ...)
	/home/circleci/project/project/backend/local/backend.go:350 +0x135
created by github.com/hashicorp/terraform/backend/local.(*Local).Operation
	/home/circleci/project/project/backend/local/backend.go:334 +0x36b



!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.

SECURITY WARNING: the "crash.log" file that was created may contain 
sensitive information that must be redacted before it is safe to share 
on the issue tracker.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

My code:

resource "kubernetes_manifest" "config-nfs-server-service" {
  provider = kubernetes-alpha

  manifest = {
    "kind" = "Service"
    "apiVersion" = "v1"
    "metadata" = {
      "name" = "config-nfs-server"
      "namespace" = "default"
    }
    "spec" = {
      "ports" = [
        {
          "name" = "nfs"
          "port" = 2049
          "targetPort" = 2049
          "protocol" = "TCP"
        },
        {
          "name" = "mountd"
          "port" = 20048
          "targetPort" = 20048
          "protocol" = "UDP"
        },
        {
          "name" = "rpcbind"
          "port" = 111
          "targetPort" = 111
          "protocol" = "UDP"
        }
      ]
      "selector" = {
        "role" = "nfs-server"
      }
    }
  }
}

@dak1n1
Copy link
Contributor

dak1n1 commented Oct 12, 2020

It looks like the above crash may have been fixed. I tried applying the config mentioned in the previous comment and got this error instead (which is expected):

kubernetes_manifest.config-nfs-server-service: Creating...

Error: Provider produced inconsistent result after apply

When applying changes to kubernetes_manifest.config-nfs-server-service,
provider "registry.terraform.io/hashicorp/kubernetes-alpha" produced an
unexpected new value: .object: wrong final value type: attribute "kind":
string required.

Since the root cause of this problem is being tracked in issue #34, I'm going to close this one as a duplicate.

@dak1n1 dak1n1 closed this as completed Oct 12, 2020
@ghost
Copy link

ghost commented Nov 12, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

@ghost ghost locked as resolved and limited conversation to collaborators Nov 12, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
acknowledged bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants