Unable to find kustomization.yaml file

[msnelling]

I have the following error

Error: buildKustomizeOverlay: Kustomizer Run for path '.' failed: unable to find one of 'kustomization.yaml', 'kustomization.yml' or 'Kustomization' in directory '/mnt/c/Code/terraform/aws-infra'
   with module.kafka.data.kustomization_overlay.schema_registry,
   on modules/kafka/kustomize_schema_registry.tf line 1, in data "kustomization_overlay" "schema_registry":
    1: data "kustomization_overlay" "schema_registry" {

I am trying to deploy the Kafka schema registry using terraform. My codebase has the following layout

/
+-- main.tf
+-- ...
+-- modules/
    +-- kafka/
        +-- main.tf
        +-- ...
        +-- kustomize/
            +-- schema-registry/
                +-- base/
                    +-- kustomization.yaml
                    +-- deployment.yaml
                    +-- ...
                +-- overlays/
                    +-- dev/
                        +-- schema-registry.env
                        +-- ingress.yaml
                    +-- uat/
                        ...

My kafka module main.tf
has the following kustomization_overlay

data "kustomization_overlay" "schema_registry" {
  namespace          = var.namespace
  common_annotations = local.standard_annotations
  resources = [
    "${path.module}/kustomize/schema-registry/base/"
  ]

  patches {
    path = "${path.module}/kustomize/schema-registry/overlays/${terraform.workspace}/ingress.yaml"
  }

  config_map_generator {
    name     = "schema-registry-config"
    behavior = "create"
    envs = [
      "${path.module}/kustomize/schema-registry/overlays/${terraform.workspace}/schema-registry.env"
    ]
  }
}

[pst]

According to your code snippet it should probably be kustomize/base/schema-registry not kustomize/schema-registry/base

[msnelling]

@pst Sorry, that's a mistake in my hand-crafted dir listing! I'll edit that now.

[pst]

I see, that was the only thing that came to mind after reading the issue.

If it's not that, you can try and run the tests in your (I'm assuming) Windows/WSL environment and see if they fail. There's some code in the provider to fake a path for the kustomization_overlay and I can't of course rule out that there may be a bug there.

[msnelling]

I'm running the make test from my WSL environment and already seeing some tests fail. Would you like me to post the test output here or somewhere else?

[pst]

If the output is very verbose, it may be easier to put it in a Gist and just link that here.

[msnelling]

Gist of test output https://gist.github.com/msnelling/7841d8af55957298942f66103e3d0b0b

[pst]

The root cause is probably this: https://gist.github.com/msnelling/7841d8af55957298942f66103e3d0b0b#file-output-log-L43-L58 Not sure why though. I have a busy launch coming up. So I won't be able to jump on this. If you want to take a stab at it, I can review PRs after Monday (assuming all goes well). Just trying to play with open cards here.

[msnelling]

I'm not a go developer I'm afraid (more C++ and C#).
The test was run from a directory that was symlinked to a dir outside of WSL. When I moved the dir to sit within WSL the tests run fine.

[msnelling]

To add a bit more info, I ran the following and the tests failed

ln -s terraform-provider-kustomization test
cd test
make test

So it seems it doesn't handle symlinks well, it's probably not a WSL specific issue.

[flyte]

I've hit this same error message after removing the .terraform directory and running init and plan again.

EDIT: I should add that I'm using remote Terraform state.

I'm using the data resource to modify a set of remote manifests - there was never a kustomization.yaml file anywhere, but it was working fine until I removed the .terraform dir.

data "kustomization_overlay" "argocd" {
  resources = [
    "https://raw.githubusercontent.com/argoproj/argo-cd/v${local.argocd_version}/manifests/install.yaml"
  ]
  namespace = local.argocd_namespace

  patches {
    target {
      kind = "ConfigMap"
      name = "argocd-cmd-params-cm"
    }
    patch = <<-EOF
      kind: ConfigMap
      metadata:
        name: argocd-cmd-params-cm
      data:
        server.insecure: "true"
    EOF
  }
}

╷
│ Error: buildKustomizeOverlay: Kustomizer Run for path '.' failed: unable to find one of 'kustomization.yaml', 'kustomization.yml' or 'Kustomization' in directory '/home/flyte/dev/mycompany/clients/myclient/theirclient/infra/terraform-30-infra/prod'
│ 
│   with data.kustomization_overlay.argocd,
│   on argo-cd.tf line 19, in data "kustomization_overlay" "argocd":
│   19: data "kustomization_overlay" "argocd" {
│ 
╵

If I add a blank kustomization.yml file in that directory, I get the following instead:

Error: buildKustomizeOverlay: Can not build dynamic overlay, found "kustomization.yml" in working directory.

[flyte]

So it seems it doesn't handle symlinks well

This is actually what's causing my issue as well.

The full path to this terraform project on my local filesystem is:

/home/flyte/dev/mycompany/clients/myclient/theirclient/infra/terraform-30-infra/prod

But I've shortened it as follows, using a symlink:

ln -s /home/flyte/dev/mycompany/clients/myclient/theirclient /home/flyte/dev/theirclient

And so:

# This works
cd /home/flyte/dev/mycompany/clients/myclient/theirclient/infra/terraform-30-infra/prod
terraform plan

# This does not work
cd /home/flyte/dev/theirclient/infra/terraform-30-infra/prod
terraform plan

it's probably not a WSL specific issue

I'm on Ubuntu 20.04

[pst]

It's probably an issue with this code or the upstream code it relies on. If somebody wants to take a stab at sending a PR to fix this.