Allow version constraints in plugin version

[Vlaaaaaaad]

Hi!

I just saw the warning regarding the aws bundled plugin deprecation and I started my migration to the .tflint.hcl way of defining plugins. I was surprised to find versioning implemented in a very strict way.

There is no support for setting latest or using something like >= 0.3. Dependabot does not support tflint, so this will lead to a lot of manual version bumping. I, like a lot of other users, want to always run with the latest and full ruleset available.

Any chance tflint can get full versioning support, just like Terraform has? If not, any chance for a latest version as a workaround?

[bendrucker]

TFLint plugins are sourced from GitHub releases. So this is definitely possible by paginating through releases until a match is found if a range is provided, rather than getting a specific tag.

https://github.com/terraform-linters/tflint/blob/master/plugin/install.go

[bendrucker]

Support for latest will come at the same time as ranges I think. We don't want to be steering the bulk of users towards latest while TFLint is still very much an 0.x product, especially the plug-in system.

[wata727]

Thank you for your feedback. I understand the desire to always use the latest plugin version.

The reason why I didn't adopt version constraints is that I couldn't be sure if the TFLint plugin ecosystem would require constraints. In Terraform, each module can declare its own constraints and must select a version that takes them into consideration, so version constraints are important. However, TFLint does not have such a situation.

Also, if we adopt version constraints or the latest keyword, there is a problem that the installed version is not fixed. This causes problems due to external factors. To avoid this, we need to generate a lockfile like .terraform.lock.hcl, but this will not meet the requirement to keep using the latest version.

Due to such issues, I started offering the tflint-bundle image. Does this help solve the problem?
If that doesn't work, can you solve it by providing commands or tools that automatically update the plugin to the latest version, for example?

[bendrucker]

Dependabot does not support tflint, so this will lead to a lot of manual version bumping.

Renovate might:

https://docs.renovatebot.com/modules/datasource/

We might be able to write a Renovate preset that handles this for TFLint plugins. Constraints allow some flexibility without turning everyone to latest but the best workflow I've found is exact versions with automated PRs.

[wata727]

I didn't think there was a reasonable reason to allow version constraints on the plugin block, but there may be cases like #1217.

If you share a config file with many workspaces, you may want to use different plugin versions for each workspace. In this case, it makes sense to allow version constraints in the config file. However, this may be an advanced use case, and I'm not sure if there really is such a requirement.

Either way, support for this requires hacks to avoid rate limiting and lock file design.

[bendrucker]

This seems like the most promising direction for automated version updates:

https://docs.renovatebot.com/getting-started/use-cases/#custom-dependency-extraction

Renovate can use GitHub tags/releases as a data source. It can be told how to parse the config file. Assuming this can be done with just config, it could be made into a preset and shared.

[bendrucker]

To the original question, here's my updated thoughts:

• Semver constraints require a lot of work, doubly so w/ GitHub as the backing API, and we're unlikely to ship that anytime soon.
• I don't like latest (invalid semver) but we could allow version to be omitted.

Omitting version is not equivalent to latest, it means any. Only if no match is found locally will the latest be downloaded to fulfill the requirement. If you run TFLint on disposable machines, you would always get the latest plugin version. If you run on a computer with permanent storage, you will never get new versions after the initial init, as your existing installation will always satisfy the requirement. This means your configuration is non-deterministic and may return different results depending on the execution environment.

This is fairly close to how Terraform works.

[PatMyron]

there is a problem that the installed version is not fixed

don't think that's a problem since that's the requested behavior

Semver constraints require a lot of work, doubly so w/ GitHub as the backing API, and we're unlikely to ship that anytime soon.

agreed, let's just support grabbing the latest plugin version with:
https://pkg.go.dev/github.com/google/go-github/v41/github#RepositoriesService.GetLatestRelease

[bendrucker]

Using the latest version is inherently unpredictable, but Terraform allows it anyway. It's not necessarily a good idea but at most the software will print a warning.

If you are able to explicitly specify latest, every init is a cache miss and requires hitting the GitHub API. If instead version is empty, the hit rate approaches 1 as you run more inits on the same machine.

Again there's a clear example to follow from Terraform, where the default behavior is the latter (cache heavy) but you can -upgrade if you want.

[bendrucker]

Ok so I don't mind working on this a bit. As described above this would work like Terraform, minus constraints.

• version is optional
• When omitted, any installed version is allowed
• If none is installed or --upgrade is set, install latest
• A warning will be printed for each un-versioned plugin

Upgrades will ignore all specific versions.

[PatMyron]

If you are able to explicitly specify latest, every init is a cache miss and requires hitting the GitHub API

Are tflint plugins initialized often enough in the same environment for that to matter? Seems fine

[wata727]

Another issue when the latest always gets the latest plugin is the compatibility of the plugin API version.
The TFLint host process and the plugin process each have an API version, depending on the version of the SDK built, and if the versions are not matched, it will fail to run. This means that as soon as we release a new API version of the plugin, your workflow will be broken.

In my experience, this non-deterministic behavior is fatal and I believe it is not useful in many environments, but if there are use cases where this is useful, I think it's not a bad idea to implement a feature like #1181 (comment)

Are tflint plugins initialized often enough in the same environment for that to matter? Seems fine to me

I'm interested in this question. Is your use case like this?

• Run TFLint primarily in your local machine.
• Don't run init often and usually refer to plugins that were installed in the past.
• Want to run init once in a while to get the latest version at once.

If so, Could you resolve it by providing a command like tflint --init --upgrade?

[wata727]

Renovate now supports TFLint plugins.
https://docs.renovatebot.com/modules/manager/tflint-plugin/