Using helm with skaffold -f path/to/skaffold.yaml throws error expecting skaffold.yaml in cwd

[j-windsor]

From GoogleCloudPlatform/cloud-code-vscode#379
Creating on behalf of Cloud Code user @dsaydon90

Expected behavior

When running skaffold -f path/to/skaffold.yaml, the command knows where to find the skaffold yaml file because it is specified in the args.

Actual behavior

User gets error searching for skaffold in the cwd: skaffold config file skaffold.yaml not found - check your current working directory, or try running skaffold init

Information

• Skaffold version: 1.19.0
• Operating system: Ubuntu 20 WSL
• Contents of skaffold.yaml:

apiVersion: skaffold/v2beta10
kind: Config
build:
  tagPolicy:
    sha256: {}
  local:
    useDockerCLI: true
    useBuildkit: true
    push: false
  artifacts:
  - image: project
    context: "../Application"
    docker:
      dockerfile: "Dockerfile"
      buildArgs:
        FOO: "{{.BAR}}"
    sync:
      infer:
      - '**/*'
deploy:
  kubeContext: minikube
  helm:
    flags:
      upgrade: ['--install']  
    releases:
    - name: project
      chartPath: project
      namespace: local
      valuesFiles:
      - "project/dev-values.yaml"
      wait: true
      upgradeOnChange: true
      artifactOverrides:
        image: project
      setValues:
        image.pullPolicy: "IfNotPresent"

[MarlonGamez]

Thanks for opening this @j-windsor! Does this fail from the start? I tried locally and it seems to work, but in the linked issue it looks like it works initially and breaks once it gets to helm related logic. I think this might be an issue with our helm debug related code

[briandealwis]

Should update exiting dev mode to use the run-mode so it is properly exiting debug mode.

[j-windsor]

@dsaydon90 might be able to answer your question. It looks like it fails when it gets to the helm logic.

[devopsmash]

In my case, I had to run skaffold from the root of the project and adding -f Kubernetes/skaffold.yaml since skaffold is not respecting relative path from the skaffold.yaml and from some reason it uses the root project instead.
more info: GoogleCloudPlatform/cloud-code-vscode#377 (comment)

[briandealwis]

I tried to reproduce this as best I could given the descriptions here and in GoogleCloudPlatform/cloud-code-vscode#379. The underlying issue is that Skaffold resolves files and directories relative to the current working directory (which in this case is the VSCode workspace location), not to the location of the skaffold.yaml, with one exception: files and directories referenced in a build artifact are resolved relative to the artifact's context.

So because the Helm chartPath and valuesFiles need to be relative to Skaffold's current working directory, which is the VS Code workspace. It all works if I update the skaffold.yaml to reference files and directories relative to the VS Code workspace:

--- Kuerbenetes/skaffold.yaml.orig	2021-02-26 17:35:37.000000000 -0500
+++ Kuerbenetes/skaffold.yaml	2021-02-26 14:37:16.000000000 -0500
@@ -9,7 +9,7 @@
     push: false
   artifacts:
   - image: project
-    context: "../Application"
+    context: "Application"
     docker:
       dockerfile: "Dockerfile"
       buildArgs:
@@ -24,10 +24,10 @@
       upgrade: ['--install']  
     releases:
     - name: project
-      chartPath: project
+      chartPath: Kuerbenetes/project
       namespace: local
       valuesFiles:
-      - "project/dev-values.yaml"
+      - "Kuerbenetes/project/dev-values.yaml"
       wait: true
       upgradeOnChange: true
       artifactOverrides:

I'll update the Skaffold docs to clarify how files are resolved.

[briandealwis]

One thing to note: we did have a bug that was fixed in #5138 with Skaffold v1.18.0 that would have affected use of skaffold debug with Helm when the skaffold.yaml was located outside of the current directory.

[j-windsor]

I see in issue GoogleCloudPlatform/cloud-code-vscode#376 that you were actually using our Cloud Code managed dependencies at the time running skaffold v1.17.2, so #5138 might be the fix that you are looking for.

[devopsmash]

Hi @j-windsor , I have used the latest version of skaffold on that time v1.19.0 by running skaffold from the command line manually.
You right that cloud code installed for me an old version under the cloud code cache directory (it's not part of PATH) which not relevant when running skaffold manually.
The issue here related to the path of the skaffold.yaml and the context of the dockerfile.
Please have a look of my project directory structure and the skaffold.yaml (please ignore cloud code stuff) here: GoogleCloudPlatform/cloud-code-vscode#377 (comment)

[briandealwis]

@dsaydon90 where are you launching Skaffold from (what is the cwd)?

Your Helm snippet references a directory named project in the chartPath and valuesFiles:

chartPath: project
namespace: local
valuesFiles:
   - "project/dev-values.yaml"

but I don't see that directory mentioned in your directory structure:

I have the following project structure under my home dir (/home/user/project)

project
|_.vscode
  |_launch.json
 |_ Application
 |_Dockerfile
  |_ <node js code files..>
|_Kuerbenetes
  |_skaffold.yaml
  |_ <helm chart of the project>

[devopsmash]

Hi @briandealwis , my cwd in this case is Kuerbenetes so the command to run skaffold is

skaffold dev -f skaffold.yaml

[briandealwis]

To close this out: Skaffold generally resolves files from the current working directory, the directory from which it was invoked. This is a common idiom amongst command-line tools, and is fundamental to how PATH works:

• make -f path/Makefile executes from the cwd
• node src/index.js executes from the cwd
• sh /path/to/script executes from the cwd (and this is what happens with an executable script with a #!/bin/sh header)

Skaffold makes some exceptions: file references with an artifact are relative to the artifact's context (aka workspace in the code), though that context is relative to the current directory.

Cloud Code for VSC executes Skaffold from the root of the workspace. So running skaffold dev from a directory is not doing to resolve files in the same way.