Fastly Plugin

[apparentlymart]

It's very likely that I'll soon need (and thus write) a Fastly plugin. Fastly is a CDN service.

I'm opening this issue to collect the results of my initial research and to describe an interesting design challenge a Fastly plugin represents, in case others have feedback on the tradeoffs I'm thinking about.

"Service" resource

The main interesting resource type in Fastly is the "service", which is similar in principle to an AWS CloudFront "Web Distribution". However, Fastly implements its own concept of configuration versions for services, which requires an API workflow that isn't just simple CRUD.

Rather than modelling the the version concept explicitly in Terraform, I'm planning to hide it away behind the Update action, with every update always creating a new version and immediately setting it live. This makes the Terraform-based workflow quite different than the usual web UI workflow, but I feel that the versioning construct in the web UI is not really necessary when you're presumably putting the Terraform config under conventional version control, and you have Terraform's own plan/apply cycle to help you understand what is being changed.

A further implication of treating versioning this way is that it's necessary to model the service and all of its descendant objects as a single resource. If they were all their own resources then each Terraform apply would generate potentially dozens of intermediate versions.

This leaves the service resource with quite a large schema:

resource "fastly_service" "foo" {
    // This is actually the only argument that belongs to the
    // service itself. The rest belong to an implicitly-created
    // version object.
    name = "example"

    // The rest of this stuff is added to an implicitly-created version object,
    // with a new one created behind the scenes each time we update the resource.

    default_host = "example.com"
    default_ttl = 600

    domain {
        hostname = "example.com"
        description = "Main Site"
    }
    domain {
        hostname = "www.example.com"
        description = "www alias"
    }

    backend {
        name = "origin"
        address = "203.0.113.83"
        port = 80
        connect_timeout_ms = 10000
        max_conn = 200
        error_threshold_ms= 0
        first_byte_timeout_ms = 300000
        between_bytes_timeout_ms = 100000
        auto_load_balance = true
        weight = 100

        // In the Fastly model conditions are a separate object
        // but it's simpler for Terraform to just model them
        // inline, since re-using conditions is largely a matter
        // of Fastly web UI convenience and doesn't really help
        // so much when using automation.
        condition = "client.ip = \"127.0.0.1\""

        use_ssl {
            check_cert = true
            hostname = "origin.example.com"
            cert_hostname = "origin.example.com"
            sni_hostname = "origin.example.com"
            min_tls_version = ""
            max_tls_version = ""
            ciphers = 'DEFAULT'
        }
    }

    cache_settings {
        name = "static-asset-ttl"
        action = "cache"
        ttl = 100000
        stale_ttl = 100000000
        condition = "req.url ~ \"^/static\""
    }

    director {
        name = "main-director"
        quorum = 75
        type = "random"
        retries = 5
        backends = ["origin"]
    }

    header {
        name = "cutesy-recruiting-header"
        type = "response"
        action = "set"
        ignore_if_set = false
        name = "http.X-Hiring"
        expression = "\"We're Hiring! See http://example.com/jobs\""
        condition = ""

        // "priority" is not settable explicitly because we instead
        // imply it by the order of the header blocks, just setting
        // it to increasing integers to preserve the given ordering.
    }

    healthcheck {
        name = "main"
        method = "GET"
        path = "/.heartbeat"
        http_version = "1.1"
        timeout_ms = 5000
        check_interval_ms = 60000
        expected_response_status = 200
        window = 2
        threshold = 1
        initial = 1
    }

    request_settings {
        name = "default"
        force_miss = false
        force_ssl = false
        action = "lookup"
        bypass_busy_wait = false
        max_stale_age = 60
        hash_keys = ["req.url", "req.http.host", "req.http.Fastly-SSL"]
        x_forwarded_for = "append"
        timer = false
        add_geo_headers = false
        default_host = "example.com"
        condition = ""
    }

    response_object {
        name = "easter-egg"
        status_code = 200
        status_message = "OK"
        content = "<p>Hello world!</p>"
        content_type = "text/html"
        initial_condition = ""
        post_cache_condition = ""
    }

    amazon_s3_logging {
        name = "access-log"
        aws_access_key_id = "..."
        aws_secret_access_key = "..."
        bucket_name = "example-logs"
        s3_domain = "s3.amazonaws.com"
        log_format = "%h %l %u %t %r %>s"
        gzip_level = 0
        path = ""
        period = 3600
        redundancy = "standard"
        condition = ""
        timestamp_format = "%Y-%m-%dT%H:%M:%S.000"
    }

    // Can only have one custom_vcl (the "main" VCL)
    custom_vcl {
        name = "main"
        content = "${file("vcl/main.vcl")}"
    }

    // Can have many of these, to be included in the main one as a library
    custom_vcl_library {
        name = "another"
        content = "${file("vcl/another.vcl")}"
    }

    wordpress_plugin {
        name = "blog"
        path = "/blog"
    }
}

Other resources

The only other resource that seems interesting to model in Terraform is User, allowing the use of Terraform to manage the set of users who have access to an account.

Purge Provisioner?

While not strictly related to a Fastly provider (unless something comes out of the idea in #2756), it could be useful to expose cache purging as a provisioner, e.g. so that Terraform can help ensure that a new app version is available shortly after it's been deployed.

[AlexanderEkdahl]

Concerning the purge part.

How about a resource for a Fastly object similar to aws_s3_bucket_object where the necessary purge calls are made when the content of said object have changed? Even if the file data is not uploaded through terraform explicit dependencies could be used to signal new versions.

[willejs]

👍 this would be epic

[dwradcliffe]

Would really love to see this happen!

Would the sub-resources, such as domain allow counts? So that we could import a list of domains from a variable?

[apparentlymart]

Not supporting counts is one down-side of representing the entire config as one resource, so you're right that this design would make it impossible to dynamically produce sets of child objects (like domains) based on variables.

I think the only way around this would be to model versions explicitly, which is another alternative design I considered. In that case you'd end up with a structure like this:

resource "fastly_service" "example" {
    name = "example"
}

// This weird resource exists only to collect together the versioned resources.
// This resource has a computed attribute called "version" that updates on
// refresh so that it always matches the next unused version number,
// causing all of the downstream resources to update on every apply.
resource "fastly_config_version" "example" {
    service_id = "${fastly_service.example.id}"
}

resource "fastly_domain" "example" {
    service_id = "${fastly_service.example.id}"
    // Since this references the version, it will always show a diff because
    // there is always a new version to create.
    version_id = "${fastly_config_version.example.id}"

    hostname = "example.com"
    description = "Main Site"
}

// This resource finally "sets live" the version we created earlier.
// At that point further updates are impossible and so a new version
// number must be assigned on the next refresh, which will cause
// this whole process to happen again.
resource "fastly_service_version_attachment" "example" {
    service_id = "${fastly.service.example.id}"
    version_id = "${fastly_config_version.example.id}"
}

The huge downside of this design is that it can never converge: by splitting into distinct resources, the fastly_config_version resource can't easily "see" the rest of the config to decide whether a new version is required, and so it must assume a new version is always required. I don't think this is acceptable, so I went with the "everything in one resource" approach to achieve a more usual Terraform workflow.

The alternative alternative I considered was to clone the config and edit it for each individual resource update. In this case, the version concept would not be modeled in the terraform config, and instead each Terraform apply would potentially increment the version number several times, with one config for each individual resource that changed. This would mean that partial configs would temporarily "go live" as the diff is applied, and it would necessarily make big updates slower due to the need to serialize all of the updates into a separate "clone config, change settings, set new config live" cycle.

With all of that said, I'd love to hear alternative design suggestions that make a better compromise between these concerns.

[desmondmorris]

+1

[sethvargo]

Hi there,

I'm going to chime in a bit here 😄. I wrote the Fastly golang API client (https://github.com/sethvargo/go-fastly), so I'm pretty familiar with Fastly's API process.

Speaking totally outside of Terraform, the Fastly Flow ™️ is as follows:

1. Clone the current configuration version of the service
2. Make changes against that configuration version
3. Validate the configuration version
4. Activate the configuration version

I believe Terraform should transparently mask 1 and 2 for the user. Terraform should know if there are any changes to the resources from the last known state. If there exists any changes, clone a new version and update all resources on that version. Then validate and activate that version.

I'm also not opposed to the "all-in-one" option originally proposed, but it seems to go against Terraform's design for other resources.

[dwradcliffe]

I would love that, if it's possible. Can Terraform group resources together and do another action when the group is fully applied?

[apparentlymart]

@sethvargo I took another go at trying to make this work, by working through the steps you listed. (I'm not sure step 1 is always to clone the current configuration, but keeping that simplification for the sake of Terraform seems reasonable...)

I think the main problem with mapping this to Terraform is that in Terraform every action must belong to either a resource or a provisioner, and each resource and provisioner has access only to its own state/diff. Thus it's not clear which resource(s)/provisioner(s) should own each step in a world where each Fastly concept maps to its own resource:

1. Clone the current version: Does the fastly_service resource do this? If so, how does it know it needs to create a new version when it can't "see" the diffs from the other resources? (This is what motivated my proposed design above: nesting everything allows it to see the diffs from all the resources) Alternatively, does each resource handle this itself? If so, how do we then ensure that each resource is working on the same new version of the config?
2. Make changes to the new version: Assuming the rest of this is solved, this is the easy part: each element resource handles its own changes.
3. Validate the configuration version: This is really just a pre-flight check for step 4, so should be done by whatever is about to do step 4.
4. Activate the configuration version: Whatever graph node handles this must have a dependency on both the creation of the new version and all of the changes across potentially many resources made in step 2. AFAIK there's no way to create implied dependencies within a provider, so such dependencies would need to be made explicit in the configuration file somehow.

So with all of that said, I'd love to support separate Terraform resources for each Fastly object (in particular, I have a use-case for piecemeal adding of Domains which the above proposed design wouldn't support), but I don't see any way to model the Fastly API resources separately while preserving the correct order of operations and maintaining a reasonable Terraform user experience.

---

Since I originally wrote this up I was working on a provider for Grafana in #3480 where there's a similar sort of issue where the whole dashboard deploys as a single unit but you'd like to build it from multiple separate resources. (No versioning in this case, but ends up causing similar design problems nonetheless.)

In that case I found a different unusual solution by exploiting the fact that Grafana has its own first-class dashboard configuration format, and so I made the grafana_dashboard resource just take a string containing that configuration format, but then separately provided a number of "logical" resources that can construct that JSON within Terraform, so the user can more easily create a dashboard in multiple parts using Terraform data.

An advantage of this model is that it inverts the dependency tree: the grafana_dashboard resource will depend on the dashboard config and panel configs, rather than the other way around. This achieves a similar workflow to my original proposal -- all the "real work" happens in a single resource -- but allows the configuration to still be built in parts.

So here's how that might look in the Fastly case:

resource "fastly_service" "main" {
    config = "${fastly_service_config.main.serialized_config}"
}

resource "fastly_service_config" "main" {
    name = "example"

    default_host = "example.com"
    default_ttl = 600

    domain_configs = [
        "${fastly_domain_config.apex.serialized_config}",
        "${fastly_domain_config.www.serialized_config}",
    ]

    // etc, etc
}

resource "fastly_domain_config" "apex" {
    hostname = "example.com"
    description = "Main Site"
}

resource "fastly_domain_config" "www" {
    hostname = "www.example.com"
    description = "www Alias"
}

In Grafana's case, this weird design can be further justified by using the same serialization format you get when you export a Dashboard from Grafana's UI, thus giving users a way to mix and match UI-authored configs with Terraform-generated configs. The issue with applying such a model to Fastly is that they don't have any sort of documented serialization of an entire service configuration, so any format used in those serialized_config attributes would be arbitrary and specific to Terraform.

So this gets us a bit closer, but doesn't quite feel right yet.

[dwradcliffe]

If #2275 was implemented it might help.

[dansteen]

+1 for this

Thanks!

[catsby]

Hey friends –

I recently merged #5814 which implements @apparentlymart 's original idea for a Fastly provider. Big thanks to him for the initial thought work 😄

Let me know if you have any other questions!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.