Feature Request: Module cache dir à la plugins

[sysadmiral]

References

• Module Versioning #15613
• cli: Opt-in local cache directory for plugins #16000

Similar to the update in 0.10.7 that cached plugins in a shared dir it would also be nice to be able to keep modules in a shared location to stop duplication on things like CI/Build servers.

To prevent updates to a module from breaking things in two separate terraform runs that reference the same shared module this shouldn't happen until we have module versioning.

My go fu is weak but I would be happy to volunteer as a terraform tester and/or documentation contributor on any of this.

[apparentlymart]

Hi @sysadmiral! Thanks for this suggestion.

There is still lots of work going on with module versioning and the module registry so we'd need to hold on this for the moment to avoid conflicting with that other work, but in the long run some mechanism to allow modules to be installed from a local source (cache, local registry, or something else) does seem reasonable. It'll be easier to think about how this fits in after the current refactoring work is complete, so I'm going to mark this as "thinking" for the moment and we can return to it once that work has settled.

Thanks again!

[sysadmiral]

Not chasing or anything but just wanted to update to say I was a little hasty and naive in my understanding of the plugin cache work and wrongly thought that the locally downloaded versions are then used directly from the cache whenever terraform runs. I now realise that's not the case and that terraform checks the cache and copies that version to it's ${PWD} and then uses that else goes off to fetch a remote version.

I still think a local module cache could be useful like the provider cache.

But I guess I now have a slightly larger question of would it be useful to be able to run providers and modules from a central location rather than potentially having multiple copies of modules in different terraform run dirs? (I'm particularly thinking of this from an automated build server perspective)

[4dz]

Agree this would be a useful feature - perhaps use hard links or soft links instead of actually copying the modules around?

[matthieudelaro]

Hi @sysadmiral! Thanks for this suggestion.

There is still lots of work going on with module versioning and the module registry so we'd need to hold on this for the moment to avoid conflicting with that other work, but in the long run some mechanism to allow modules to be installed from a local source (cache, local registry, or something else) does seem reasonable. It'll be easier to think about how this fits in after the current refactoring work is complete, so I'm going to mark this as "thinking" for the moment and we can return to it once that work has settled.

Thanks again!

@apparentlymart , is there any update about whether or not such a feature could be implemented?

I created a script which emulates terraform init to download all modules in a cache folder, as if there were a setting module_cache_dir, similar to plugin_cache_dir. I am open to contributing to Terraform to implement the feature.

[jcogilvie]

Our org is implementing github actions caching, which is feasible with providers but not yet with modules. Github caches are on a per-repository basis, so for a case like ours, all we'd need is the ability to generate modules.json with terraform get without downloading the modules, in the same way that we can generate the provider lock file with terraform providers lock. (The hash of the lock file serves as a cache key.)

I realize this may deserve to be a separate feature request, but I thought it might be worth making a note here.