Support variable files (*.tfvars)

[radeksimko]

Depends on:

• Support for type declarations (for variable type) #490
• Support for references to variables and locals #491

---

Current Version

0.16.1

Use-cases

Many users declare variables in a dedicated file, i.e. *.tfvars as per https://www.terraform.io/docs/language/values/variables.html#variable-definitions-tfvars-files

Users would benefit from the same level of support they get for Terraform configurations (*.tf) for *.tfvars files too.

Completion

Assuming there is a *.tf file with the following declarations

variable "image_id" {
  type = string
  description = "Amazon AMI ID"
}

variable "availability_zone_names" {
  type    = list(string)
  default = ["us-west-1a"]
  description = "Names of availability zones"
}

then in an empty configuration, completion would bring up names of these declared variables with their type & description, i.e.

• image_id string - Amazon AMI ID
• availability_zone_names - list(string) - Names of availability zones

Then further completion would be available for each variable value, depending on its type, e.g. for a more complex variable type, such as object

variable "docker_ports" {
  type = list(object({
    internal = number
    external = number
    protocol = string
  }))
}

the internal attributes of the object would be completed

docker_ports = [{
  # HERE -> internal, external, protocol
}]

Hover

variable "availability_zone_names" {
  type    = list(string)
  description = "Names of availability zones"
}
variable "docker_ports" {
  type = list(object({
    internal = number
    external = number
    protocol = string
  }))
}

If user was to hover over a variable name, e.g. availability_zone_names in the above config they would be presented with a popup with details about the field - i.e. its type and description

_list(string)_
Names of availability zones

Semantic Tokens

All valid (known) variable names which are declared via variable blocks would be reported as attribute tokens and compatible expressions would be highlighted if present, for example:

docker_ports = [{
  internal = 80
}]

i.e. docker_ports and internal would be reported as attribute, and 80 as number.

Diagnostics Reporting

Any duplicates or mismatch variable types would be reported as diagnostics, similar to how HCL syntax errors are reported today. For example the following config

test = "something"
test = "blah"

would result in the following diagnostic being published:

The argument "test" was already set at terraform.tfvars:1,1-5. Each argument may be set only once.

Duplicate variables will be reported automatically upon saving of any file within a module for auto-loaded files, i.e. terraform.tfvars & *.auto.tfvars, and duplicates will also be reported within the context of a single *.tfvars file upon saving that file.

Proposal

1. Introduce module.Variable{ Type cty.Type; Description string } to terraform-schema/module: https://github.com/hashicorp/terraform-schema/blob/main/module/meta.go
2. Modify logic in hashicorp/terraform-schema -> earlydecoder to also collect and return variables as map[string]module.Variable as part of *module.Meta: https://github.com/hashicorp/terraform-schema/blob/main/earlydecoder/decoder.go
3. Add Variables field to state.ModuleMetadata:

   terraform-ls/internal/state/module.go

   Lines 15 to 19 in 92a2a2c

   	type ModuleMetadata struct {
   	CoreRequirements version.Constraints
   	ProviderReferences map[tfmod.ProviderRef]tfaddr.Provider
   	ProviderRequirements map[tfaddr.Provider]version.Constraints
   	}

4. Make sure these Variables are also persisted to state by modifying ModuleStore.UpdateMetadata()
5. Implement new module operation ParseVariables for parsing terraform.tfvars or *.auto.tfvars (similar to ParseConfiguration). It may also be helpful to rename ParseConfiguration to ParseModuleConfiguration.
   • we can start collecting these variables as map[string]cty.Value initially as that will be helpful for Show interpolated expression values on hover #495 but we don't really need to do persist anything at all
   • diagnostics can be stored in state.Module - e.g. as VariableDiagnostics map[string]hcl.Diagnostics. We could also consider separating diagnostics for the auto-loaded tfvars files from "standalone" ones as the auto-loaded diags should be always presented for any open file within a module, but standalone only when that standalone file is open.
6. Introduce SchemaForVariables(map[string]module.Variable) *schema.BodySchema to terraform-schema/schema
7. Recognize new LanguageID in all relevant RPC methods and utilize SchemaForVariables where needed, e.g.
   • textDocument/didOpen
   • textDocument/didSave
   • textDocument/didChange
   • textDocument/completion
   • textDocument/hover
   • textDocument/semanticTokens
   • textDocument/documentLink (basically to be ignored as there's typically no systematically linkable content)
8. Implement formatting for this type of file in textDocument/formatting the same way as it is implemented for *.tf (via terraform fmt)
9. Report duplicates, mismatching types or any similar diagnostics via textDocument/publishDiagnostics

[radeksimko]

Many clients treat plain HCL, tf and tfvars as equal, which will need to change. The language matching often happens via a dedicated Terraform plugin and generic LSP plugin then uses that information.

Resolving this will therefore require changes in the affected plugins, so that they send the appropriate languageId which the generic LSP plugin can then pass to the LS.

Before requesting changes in these clients though we should define and document some standard language IDs - and ideally publish them in the LSP spec too.

Examples in the wild:

• https://github.com/alexlouden/Terraform.tmLanguage
  • languageId terraform matches tf, tfvars and hcl
  • this affects the LSP plugin as it doesn't have any other way to match file syntax
• https://github.com/hashivim/vim-terraform
  • languageId terraform matches tf and tfvars
  • this affects the coc.nvim plugin, but the plugin is also capable of simple extension-based matching, in which case the languageId comes through as the extension (e.g. tf)
• https://github.com/emacsorphanage/terraform-mode
  • matches tf and tfvars
  • this affects lsp-mode as it doesn't seem to have any other way to match file syntax

---

Then assuming that a client sends languageId: "tfvars" we can track the language ID alongside a file:

terraform-ls/internal/filesystem/types.go

Lines 8 to 12 in 906c368

	type File interface {
	FileHandler
	Text() []byte
	Lines() source.Lines
	}

and then start treating tfvars differently e.g. here based on that language ID

terraform-ls/langserver/handlers/complete.go

Lines 39 to 49 in 906c368

	file, err := fs.GetFile(ilsp.FileHandler(params.TextDocument.URI))
	if err != nil {
	return list, err
	}
	hclFile := hcl.NewFile(file)
	fPos, err := ilsp.FilePositionFromDocumentPosition(params.TextDocumentPositionParams, file)
	if err != nil {
	return list, err
	}
	hclBlock, hclPos, err := hclFile.BlockAtPosition(fPos)

[radeksimko]

For any subscribers and future readers of this (closed) issue: The server will support variable files from the upcoming release. However it is still up to each individual language client to make updates to send such files to the server and do so with the correct LanguageID (terraform-vars) as per docs.

Relatedly the Terraform VS Code extension is yet to gain that functionality.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.