-
-
Notifications
You must be signed in to change notification settings - Fork 981
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Include sample "best practices Terragrunt configuration" #169
Comments
Hello, I expect that this is probably low priority but I'd appreciate some rough guidance around how to organise / write the terraform.tfvars files, specifically on creating DRY remote state configuration. The current README.md suggests:
I'm finding that this fails to configure the remote state as after the initial I've tried putting the remote state (parent and child) Any advice much appreciated! Thanks Tom |
@tomstockton All the |
I've also been struggling with this. Conceptually I think I grok all the individual examples, but when i try to combine some strategies I keep hitting some issues. Here is the combined structure I believe should be in place:
The
The child
The parent
With this setup, and running a The next challenge I ran into with this was when trying to add some So @josh-padnick said, I' still in that still has to make one more leap step of trying to get the right structure in place that plays nice with all the features I want to use. I'm confident I'll get there, but I just wanted to add some supporting feedback on this. |
Apologies for the confusion everyone. We'll definitely add more examples to the docs and perhaps sample code. For now, the most I have time for is to give one quick example layout that will hopefully help everyone understand what I had in mind. There are, of course, many ways to use Terraform and Terragrunt, but the idea behind remote Terraform configurations was that you define your Terraform code (
This contains 100% normal Terraform code to deploy a variety of "components": a single app, or a single MySQL DB, or a single VPC. The only requirements for this code are that each component:
To actually deploy the code in
The root terragrunt = {
remote_state {
backend = "s3"
config {
bucket = "my-terraform-state"
key = "${path_relative_to_include()}/terraform.tfstate"
region = "us-east-1"
encrypt = true
lock_table = "my-lock-table"
}
}
} Each of the child terragrunt = {
# Include all the settings from the root .tfvars file
include {
path = "${find_in_parent_folders()}"
}
# Apply the code in the frontend-app module
terraform {
source = "git::git@github.com:foo/modules.git//frontend-app?ref=v0.0.3"
}
}
# Fill in the environment-specific variables for frontend-app
instance_count = 5
instance_type = "t2.micro" Hope that helps! Feel free to ask questions. |
@brikis98 No need for Apologies, Its totally understandable that as you reach more usage, new cases will arise needing clarification. Thanks for the explanation above, I hadn't considered putting the s3 config stub into the modules. What you have laid out is helpful, however I've got a few more questions/scenarios which I think are fairly common. I'm not asking this because I haven't address them, but mostly to get the idea out in the open in case it helps drive what the sample layout looks like.
Thanks again for taking the time on this thread, please don't take these comments as complaints, just wanted to drop some of the thoughts I've had while working through getting my stuff organized the last couple of days. |
@dpetzel I'll answer your questions one at a time, as I wait for various builds to run :)
The folder structure in the
In the variable "region" {
description = "The region to deploy to"
} You can then use that throughout your Terraform code, including the provider "aws" {
region = "${var.region}"
}
# ... create resources ...
data "terraform_remote_state" "vpc" {
backend = "s3"
config {
region = "${var.region}"
bucket = "my-bucket"
key = "vpc/terraform.tfstate"
}
} If your state is stored in a different region than the one you're deploying to, then you can have two input variables, one called |
Yes, one root
|
Sorry I don't think I clearly articulate the concern with regards to this one:
Totally understand when reading the remote state that is how it would be done, but what I'm suggesting is that you want to configure your remote state to stay in the same region as the resource your deploying. The example above hard codes |
If everything is in one repo, then as soon as you update a module, that change will affect every single environment the next time you run By using multiple repos, you can create an immutable, versioned "artifact" for every change, and carefully promote that artifact through each of your environments (e.g. qa -> stage -> prod). It's easier to understand exactly what is deployed where and you never accidentally push code to prod before it's ready. You could, of course, use tags to do versioning within a repo, but I found it to be a bit confusing to be relying on old versions of the |
I think the issue here is one of terminology. I'm overloading the term module to mean two things:
|
Ah, gotcha. This is a common question discussed in #132, #147 (comment), and elsewhere: how do you share variables between Terraform and Terragrunt? In your case, you want to be able to specify the region in one place, and for that region to be used in both Terragrunt's remote state configuration and your Terraform code. The short answer is that we don't have a good solution yet. Perhaps #147 (comment) is the way to go. I'm open to ideas. |
I've struggled with this as well. I've been using components myself internally, but even that I don't love. Its almost more of an implementation, but as they say..."Naming things is hard" :) Regarding #147 (which I did bump into myself) Having some more sophisticated "deep merging" would certainly help. I might also be interesting if nested includes were possible, so given our earlier multiple region type setup we might have something like this:
I've also been underwhelmed here, especially in the context of multiple regions, and keeping state regionalized. |
I'm glad we're codifying some of our official recommendations, even if just as a GitHub issue for now. I did want to register one potential exception to the following recommendation:
On my vacation, I was playing around with the aws_dynamodb_table resource for use in a serverless app and found that this Terraform resource has so many inline resources that defining a module doesn't make sense unless hashicorp/terraform#14037 is resolved. One option is to define a Terraform module for each possible permutation of aws_dynamodb_table, like one for an aws_dynamodb_table with no Global Secondary Index, one for two Local Secondary Indexes, etc. But this feels clumsy, so I'd prefer to just avoid using Terragrunt's Comments welcome on this use case. |
Yes, a "module" would be very difficult to use with this resource. However, a "component"--that is, a remote Terraform configuration downloaded by Terragrunt--would work just fine. You don't need to define every permutation of a component; just the very small number (usually 1) of permutations your team actually uses. So I stand by the argument that just about all Terraform code ( |
I'd tend to agree with @brikis98 here. A "module" in this context, isn't something that takes all potential options, but rather an implementation. You know which options on that resource you want to deploy and you'd code this "implementation module" as such. Then it would have a few variables that will vary in between your environments IE you may want different read/write capacity between dev and prod but you probably wouldn't have an index block in one environment not another. However for a different project/app you might not have a certain index block, in which case that is a second "implementation module", again exposing only the things you'd vary between environments. The key concept (which took me a bit to a handle on) is that your not building generic multi-use type of modules, but rather modules that are implementing a very specific configuration, unique to your scenario. |
Increasingly, I'm seeing this is as an essential concept. Thanks for sharing your perspective! I took @brikis98's advice and it was indeed quite easy to work with things. |
I just caught up on this thread and wanted to throw in my 2 cents. I would also love to have some example code to get started with Terragrunt. I ran into some of the issues linked in this thread and having example code would have made it much easier to wrap my head around some of the Terragrunt features. I'll be out of town this weekend, but I plan to start on a PR next week that contains tons of examples. I will rely heavily on this thread and also the examples in the README. I will try to have example code for every single Terragrunt keyword/command in the README. |
@conorgil That would be wonderful. Thank you! |
I have a question about remote state. After setting up my simple environment and reading around about setting up a vpc per env I came across this article which suggests maintaining a separate state per env. I noticed that per the docs and reaffirmed above (if using a "live" repo with dev, staging, prod, and a separate modules repo with the .tf files ) that we have a single tfstate for all the envs. If we want seprate buckets and a dynamo lock db per env would I just add a terraform.tfvars not at the root of the live folder but add one in each environment folder? |
First, I think you'll find that a single S3 Bucket for all Terraform state in a given AWS account works just fine. We've used that pattern for multiple clients, and since Terragrunt automatically fills in the paths to each module's remote state, it works great. Is there a reason this approach doesn't work as well for you as the "multiple tfvars files" approach? If you still want to customize your remote state per environment, then yes, you can create a "root" One word of caution, there is only one level of "include" that happens when you use the following in your
The |
Just incase anyone hits the same snag: Terraform was prompting me for my bucket name, key, and region when using:
along with a prefilled terraform.tfvars file that included my terragrunt remote config. The culprit was: https://www.terraform.io/docs/commands/init.html#backend-true This value needed to be set to false to properly initialize using the backend config found in my tfvars file. |
@brikis98 @josh-padnick With the patterns described above, where does the "provider" section go? Ordinarily, I'd have a stanza like this in my
It feels like that would need to be present in each of the I do have a |
Yup, that's correct. If you want to avoid repeating the settings, expose them as variables that you provide from some |
@josh-padnick @brikis98 @dpetzel @tomstockton @woods @dfcarpenter I opened #234 for discussion on a WIP example that I put together to highlight some of the basics for a Terragrunt "best practices" type of thing. I'd appreciate any and all comments on it to see if I am on the right track to creating something useful for the community. |
OK, I just put together a couple repos we can use for large, somewhat real-world examples: https://github.com/gruntwork-io/terragrunt-infrastructure-modules-example Hopefully, this provides some useful guidance for how to use Terragrunt. Feedback and PRs are very welcome! |
Hi @brikis98 , I tried your sample code, but it seems that it will fail to retrieve region.hcl, env.hcl from the root terragrunt.hcl. Or did I missed something? Sample error messages: Thanks |
Even I find myself glazing over a little by the bottom of the README because there are a lot of features here. Also, after reading everything, the Terragrunt user still has to make one more leap on their own: What is the ideal, complete setup with Terragrunt that combines all these use cases?
For that reason, it'd be nice if we had a
_sample
folder with maybelive
andmodules
subfolders which mimicked a real setup and showed exactly what the rootterraform.tfvars
file and each "child"terraform.tfvars
file looks like.Or alternatively, just include those two
terraform.tfvars
files (root and child) as examples.The text was updated successfully, but these errors were encountered: