Support remaining atlantis terraform commands

[nitrocode]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

• I'd be willing to implement this feature (contributing guide)

Describe the user story

There are a lot of missing terraform commands and I do not like running them locally. I'd like to run them all from github.

Describe the solution you'd like

remaining (in order of highest value)

• terraform refresh
  • this would be helpful to clean up attributes changing without hcl changes (this is not drift, this is api related changes that do not affect the apply but are shown in the plan)
  • workaround is to either do this locally OR do an atlantis refresh-only plan with an apply with disable-automerge, which is cumbersome.
• taint
  • this would helpful but there is a workaround for it using -replace
  • workaround: atlantis plan -- -replace=aws_s3_bucket.example
  • terraform taint
  • terraform untaint

Lower value

• terraform state list
  • this would be handy, but does not change state so it can be run locally
• terraform validate
  • does not change state so can be run locally
  • also i believe this runs when doing an init
• terraform providers
  • does not change state so can be run locally

done

• terraform import as of v0.22.x
• terraform state rm as of 0.23.x (unreleased, use dev tag image)

won't do

• terraform destroy
  • workaround: atlantis plan -- -destroy=aws_s3_bucket.example
• terraform state mv
  • use hcl move blocks in terraform 1.x

Option 1: implement each subcommand

This is what we currently do.

Write golang code for each terraform subcommand that exists today and future subcommands that are created...

Option 2: Pass through command

There are many current issues for each terraform subcommand and it seems difficult to scale up.

It would be nice to create a pass through command so all terraform commands can be supported.

Spacelift, for instance, supports only plan and apply out of the box. However, they have a separate section in the UI that allows running any arbitrary commands, even non terraform commands. Of course to get to their UI, the user has to be authenticated through sso/saml and things are gated using rego policies which we do not currently support.

Pros:

• much easier to implement all the subcommands
• much easier to implement new commands

Cons:

• The terraform destroy command would become available unless we wrote the code so select subcommands of terraform could be disabled using a config/flag.

References

• Support atlantis validate subcommand #2766
• Support atlantis destroy subcommand #941
• Support atlantis state rm subcommand #273
• Support atlantis import subcommand #217
• Support and document -replace (since terraform taint subcommand is deprecated) #527
• Support atlantis test subcommand #4409
• Support atlantis untaint subcommand #4464
• Ability to cancel a plan/apply using atlantis cancel #187

[jamengual]

@nitrocode as I mentioned in slack, doing a pass thorough it could make the code pretty difficult to manage once a feature like this is released and people start sending PR to add logic to deny access to certain commands and such, we could end up having a huge amount of policy code everywhere for every command etc.

[nitrocode]

I imagined it more like this in the server config

atlantis_allowed_terraform_subcommands:
- validate
- state
- import
- providers

Then it would only allow select commands. I figured it would be less work to maintain a pass thru command then adding each subcommand separately in golang which is our current method. And if terraform releases a new subcommand then it's one more subcommand to support.

If the community would prefer to be explicitly create each subcommand in golang, then someone would need to write the code to support each one I suppose.

[nitrocode]

I changed this ticket around so it has multiple options for supporting the remaining terraform commands. Option 2 is pass through and Option 1 is what we're currently doing.

This ticket will at least contain each of the secondary tickets for supporting each terraform subcommand.

[marcinswigon]

@nitrocode @krrrr38 How are this commands supposed to work with workflow hooks? I was trying to get import to work but in my case it did not run pre-workflow hook (it is supposed to set correct workspace for the project, something similar to #982 (comment)). Also with multi-atlantis setup for single repo (branch allowlist) it seems that multiple instances are responding to import comment even when not in their branch allowlist (for sure that is the case if command is not in allowed_commands).

[krrrr38]

@marcinswigon

How are this commands supposed to work with workflow hooks? I was trying to get import to work but in my case it did not run pre-workflow hook

I guess it should works, could you put reproduce steps and create a ticket?

Also with multi-atlantis setup for single repo

Please check this docs. --executable-name may help you.

[marcinswigon]

I went through my configuration and the first issue was actually missing extra_args for init step under import command in my custom workflow. Would it be possible to have generic configuration for init so that extra_args is added automatically for all commands?

The second issue I think is the order of checks that atlantis performs when parsing comments: it checks if the command is allowed before checking if repo/branch is in allowlist.

===edit
plan command also automatically adds -var-file flag if it finds env/${workspace}.tfvar file:

atlantis/server/core/runtime/plan_step_runner.go

Line 110 in ee7a5ca

// Check if env/{workspace}.tfvars exist and include it. This is a use-case

Can this also be included in other commands?

[nitrocode]

Please create a separate issue for this.

[dupuy26]

Having the ability to run terraform state replace-provider ... commands through Atlantis would also be very useful, especially when migrating old pre-0.13 code (but also in more modern contexts).

[entscheidungsproblem]

Could we add another issue for the terraform test command that was added in v1.6.0?
https://developer.hashicorp.com/terraform/language/tests

This ticket would likely be relevant as additional tests (such as terratest) could be added to the steps in the test workflow.

[Gianluca755]

Hello everybody,
as a side note I would specify that the terraform commands classified as lower value can be considered so only if the state is freely accessible by the various developers.
If for security reasons the only interactions possible with the state is with Atlantis, those commands became useful.

[tfhartmann]

I'd love to see terraform test added as a feature that gets run by default if any test files are present and adjacent to other terraform manifests as part of the default workflow. Basically, something like : If terraform version >= 1.7 then check for any .tftest.hcl or .tftest.json files, if exist run terraform test before a plan gets run.

[SamuelMolling]

Do we currently have any way to remove a remote backend from state?

Terraform's state rm removes the state resource, but from what I understand, Atlantis' state rm removes the plan file.

This command discards the result of the terraforming plan.

I need to remove a resource and recreate it via atlantis and I don't know how to do it.
I tried using plan's -replace option, but without success either.

The idea is to recreate the random_password resource to force a password change for another user resource.

[GMartinez-Sisti]

The idea is to recreate the random_password resource to force a password change for another user resource.

@SamuelMolling you don't need to remove the resources from the state. Use the keepers configuration instead.

As a side note, removing resources from state should only be required when you are migrating resources from a state file to another or something went wrong and you need to cleanup, not as a regular process.

[SamuelMolling]

@GMartinez-Sisti The documentation says that to use keepers, I need to run taint and this command is not available in Atlantis, right ?

[GMartinez-Sisti]

@GMartinez-Sisti The documentation says that to use keepers, I need to run taint and this command is not available in Atlantis, right ?

Not as a regular command, but you can add it if you want using a Custom Workflow, however if you implement keepers the correct way it should trigger a change on plan when the linked resources change.

[SamuelMolling]

In my case, it thinks that for some reason we need to change the password, but we don't have access to the console or anything related. We would need a way to change using the pipeline.

[GMartinez-Sisti]

In my case, it thinks that for some reason we need to change the password, but we don't have access to the console or anything related. We would need a way to change using the pipeline.

As a suggestion, you can create a terraform variable of type string and link it to the keepers config. Whenever that variable changes the password will change, you can use whatever value you want in the variable: a date, a ticket, a reason, the content doesn't matter since the goal is to just change the password.

Taint should be used as a last resort, avoid using it as a daily operation since you might mess up your terraform state file while doing it.