/api/plan does not create workspace if it does not exist

[jamengual]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

when using the new /api/plan API and sending a request I get a 500 Internal server error due to the PR+workspace does not exist.

Reproduction Steps

send a request to the altantis server with this payload:

{"Repository":"jamengual/atlantmos","Ref":"main","Type":"Github","PR":0,"Projects":null,"Paths":[{"Directory":"components/terraform/null-resource","Workspace":"pepe-ue2-lab"}]}

Logs

Server Logs

caller":"events/instrumented_project_command_builder.go:66","msg":"Error building plan commands: checking if workspace exists: stat /Users/pepe.amengual/.atlantis/repos/jamengual/atlantmos/1/default: no such file or directory","json":{},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).buildAndEmitStats\n\t/Users/pepe.amengual/github/atlantis/server/events/instrumented_project_command_builder.go:66\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).BuildPlanCommands\n\t/Users/pepe.amengual/github/atlantis/server/events/instrumented_project_command_builder.go:35\ngithub.com/runatlantis/atlantis/server/controllers.(*APIRequest).getCommands\n\t/Users/pepe.amengual/github/atlantis/server/controllers/api_controller.go:64\ngithub.com/runatlantis/atlantis/server/controllers.(*APIController).apiPlan\n\t/Users/pepe.amengual/github/atlantis/server/controllers/api_controller.go:145\ngithub.com/runatlantis/atlantis/server/controllers.(*APIController).Plan\n\t/Users/pepe.amengual/github/atlantis/server/controllers/api_controller.go:90\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/Cellar/go/1.19.3/libexec/src/net/http/server.go:2109\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/Users/pepe.amengual/go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210\ngithub.com/urfave/negroni/v3.Wrap.func1\n\t/Users/pepe.amengual/go/pkg/mod/github.com/urfave/negroni/v3@v3.0.0/negroni.go:59\ngithub.com/urfave/negroni/v3.HandlerFunc.ServeHTTP\n\t/Users/pepe.amengual/go/pkg/mod/github.com/urfave/negroni/v3@v3.0.0/negroni.go:33\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/Users/pepe.amengual/go/pkg/mod/github.com/urfave/negroni/v3@v3.0.0/negroni.go:51\ngithub.com/runatlantis/atlantis/server.(*RequestLogger).ServeHTTP\n\t/Users/pepe.amengual/github/atlantis/server/middleware.go:70\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/Users/pepe.amengual/go/pkg/mod/github.com/urfave/negroni/v3@v3.0.0/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Recovery).ServeHTTP\n\t/Users/pepe.amengual/go/pkg/mod/github.com/urfave/negroni/v3@v3.0.0/recovery.go:210\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/Users/pepe.amengual/go/pkg/mod/github.com/urfave/negroni/v3@v3.0.0/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Negroni).ServeHTTP\n\t/Users/pepe.amengual/go/pkg/mod/github.com/urfave/negroni/v3@v3.0.0/negroni.go:111\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/Cellar/go/1.19.3/libexec/src/net/http/server.go:2947\nnet/http.(*conn).serve\n\t/usr/local/Cellar/go/1.19.3/libexec/src/net/http/server.go:1991"}
{"level":"debug","ts":"2023-01-07T17:56:18.244-0800","caller":"metrics/debug.go:52","msg":"timer","json":{"name":"atlantis.builder.execution_time","value":0.001017071,"tags":{},"type":"timer"}}

Environment details

• Atlantis version: 0.22.2
• Atlantis flags:

Atlantis server-side config file:

#atlantis-url: "localhost"
gh-user: jamengual
gh-token: "TOKEN"
repo-allowlist: "*"
gh-webhook-secret: "SECRET"
repo-config: atlantis-repo-config.yaml
enable-diff-markdown-format: true
silence-vcs-status-no-plans: true
hide-prev-plan-comments: true 
log-level: debug
atlantis-url: "http://localhost:4141"
api-secret: "SECRET"

Repo atlantis.yaml file:

version: 3
automerge: true
delete_source_branch_on_merge: true
parallel_plan: true
parallel_apply: true
allowed_regexp_prefixes:
- lab/
- staging/
- prod/
projects:
- name: pepe-ue2-lab-null-resource
  workspace: pepe-ue2-lab
  workflow: workflow-1
  dir: components/terraform/null-resource
  terraform_version: v1.2.9
  delete_source_branch_on_merge: false
  autoplan:
    enabled: true
    when_modified:
    - '**/*.tf'
    - $PROJECT_NAME.tfvars.json
  apply_requirements:
  - approved
workflows:
  workflow-1:
    apply:
      steps:
      - run: terraform apply $PLANFILE
    plan:
      steps:
      - run: terraform init -input=false --reconfigure
      - run: terraform workspace select $WORKSPACE || terraform workspace new $WORKSPACE
      - run: terraform plan -input=false -refresh -out $PLANFILE -var-file $PROJECT_NAME.tfvars.json

Additional Context

Trying to implement drift detection

[nitrocode]

For reference, the pr that implemented the /plan endpoint is PR #997 by @remilapeyre and then by @lilincmu.

It's also quite possible that if the /plan route doesn't work for workspaces, that the /apply route may not either.

From the error, it looks like it's looking in the wrong directory for the file.

stat /Users/pepe.amengual/.atlantis/repos/jamengual/atlantmos/1/default: no such file

[Sickafant]

Hi,

I'm seeing this problem as well in 0.22.2, for me it occurs when a directory targeted plan is run:

atlantis plan -d devops/jenkins

result:

Plan Error
checking if workspace exists: stat /home/atlantis/.atlantis/repos/CICD-devops Applications/terraform/397/default: no such file or directory

logs:

{"level":"debug","ts":"2023-01-17T00:13:44.182Z","caller":"server/middleware.go:45","msg":"POST /events – from 10.75.20.246:63654","json":{}}
{"level":"debug","ts":"2023-01-17T00:13:44.182Z","caller":"events/events_controller.go:125","msg":"handling Bitbucket Server post","json":{}}
{"level":"debug","ts":"2023-01-17T00:13:44.182Z","caller":"events/events_controller.go:251","msg":"handling as comment created event","json":{}}
{"level":"info","ts":"2023-01-17T00:13:44.182Z","caller":"events/events_controller.go:542","msg":"parsed comment as command=\"plan\" verbose=false dir=\"devops/jenkins\" workspace=\"\" project=\"\" flags=\"\"","json":{}}
{"level":"debug","ts":"2023-01-17T00:13:44.182Z","caller":"events/events_controller.go:574","msg":"executing command","json":{}}
{"level":"debug","ts":"2023-01-17T00:13:44.182Z","caller":"logging/simple_logger.go:157","msg":"Processing...","json":{}}
{"level":"debug","ts":"2023-01-17T00:13:44.182Z","caller":"server/middleware.go:72","msg":"POST /events – respond HTTP 200","json":{}}
{"level":"error","ts":"2023-01-17T00:13:44.275Z","caller":"events/instrumented_project_command_builder.go:66","msg":"Error building plan commands: checking if workspace exists: stat /home/atlantis/.atlantis/repos/CICD-devops Applications/terraform/397/default: no such file or directory","json":{},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).buildAndEmitStats\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:66\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).BuildPlanCommands\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:35\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:181\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:258\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:296"}
{"level":"debug","ts":"2023-01-17T00:13:44.275Z","caller":"metrics/debug.go:52","msg":"timer","json":{"name":"atlantis.builder.execution_time","value":0.000104064,"tags":{},"type":"timer"}}
{"level":"error","ts":"2023-01-17T00:13:44.369Z","caller":"events/pull_updater.go:17","msg":"checking if workspace exists: stat /home/atlantis/.atlantis/repos/CICD-devops Applications/terraform/397/default: no such file or directory","json":{"repo":"CICD-devops Applications/terraform","pull":"397"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*PullUpdater).updatePull\n\tgithub.com/runatlantis/atlantis/server/events/pull_updater.go:17\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:186\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:258\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunCommentCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:296"}

Workaround:

• Run a non-directory plan atlantis plan.
• subsequent -d plans in run in the same PR are able to run ok.
• If plans are unlocked, or PR is reopened, the error can be reproduced again until a regular atlantis plan

[mralanlee]

Will this be addressed in the latest build?

[jamengual]

if someone created a PR with the fix, yes

…

On Wed, Feb 15, 2023, 9:25 a.m. Alan Lee ***@***.***> wrote: Will this be addressed in the latest build? — Reply to this email directly, view it on GitHub <#2949 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQ3ERHGXO5TMQH53TLVFCDWXUGQNANCNFSM6AAAAAATUJGTIE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[jukie]

Did you figure out a workaround for this? Looking to replicate drift detection for gitlab or add gitlab support directly and hitting this when using the API. In some cases even with an existing PR so wondering if I'm just using it wrong.

curl --request POST 'https://atlantis-endpoint.com/api/plan' --header 'X-Atlantis-Token: mysecret' --header 'Content-Type: application/json' --data-raw '{
    "Repository": "repo/name",
    "Ref": "some-branch-from-MR-or-main",
    "Type": "Gitlab",
    "Paths": [{
      "Directory": "./tf-dir",
      "Workspace": "MR-number, empty, or default"
    }]
}'

[GenPage]

I will be reviewing this week.

[mralanlee]

@GenPage any updates?

[GenPage]

See #3584 (comment)

I'm hesitant to merge until we understand the problem with locks as detailed in ADR 0002 #3345

The last thing we want to do is call to clone/workspace in its current regressed state.

[mralanlee]

See #3584 (comment)

I'm hesitant to merge until we understand the problem with locks as detailed in ADR 0002 #3345

The last thing we want to do is call to clone/workspace in its current regressed state.

Got it, thanks for the clarity.

Unfortunately, this is blocking us from upgrading to newer versions of Atlantis because it keeps hitting the stat error.

[jamengual]

I think this will not get merged, it is just a matter of time before we fix the locks problem that has caused a lot of issues.

[peikk0]

I'm running into this issue as well, I was trying to use api/plan for drift detection from the main branch.