Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource/cloudflare_ruleset: improve dashboard collisions #1393

Merged
merged 1 commit into from
Jan 17, 2022

Conversation

jacobbednarz
Copy link
Member

One of the earliest niggles with customers coming from the dashboard to
Terraform was the collision caused by a Ruleset phase being created in
the UI but the Terraform provider also needing to create the same
phase. This was problematic for a few reasons:

  • The first was that when you deleted Ruleset rules in the UI, it didn't
    remove the phase. This was intentional but hidden behind the UI and
    only accessible via the API.
  • Secondly, when customers wanted to use Terraform, there was an
    assumption they would be starting from scratch and many were not.
  • Finally, in the event of a collision, we didn't know which Ruleset the
    customer wanted to use so we error'd out with a link to manually
    resolve which isn't a great solution but made the issue more
    prominent.

I had a chance to rethink through this issue and managed to find a way
that we improve all three points above and remove majority of the pain
points. With the proposed changes, the only time a customer needs to
manually resolve the Ruleset rules is if there are existing rules in the
UI which requires them to be deleted or migrated.

Achieving this requires the assumption that if the Ruleset has no rules,
it is ok to modify. Unfortunately, it's not that simple in practice. If
the phase already exists, we cannot just update it as the name
attribute is not writable after creation. Along with this, the ref and
version values will be automatically incremented causing a permadiff
in Terraform as the customer hasn't actually modified these values but
the backing service (and API) has. To work around this, if we find a
phase with no rules, we recreate it with the provided values which is
essentially the same the same thing as the "happy path" for a new
Terraform resource would be anyway.

@jacobbednarz jacobbednarz force-pushed the smoother-existing-rulesets-handling branch 2 times, most recently from 449fe21 to 3d1900b Compare January 17, 2022 01:13
@jacobbednarz
Copy link
Member Author

acceptance tests are still passing

TF_ACC=1 go test $(go list ./...) -v -run "^TestAccCloudflareRuleset_" -count 1 -parallel 1 -timeout 120m -parallel 1
?   	github.com/cloudflare/terraform-provider-cloudflare	[no test files]
=== RUN   TestAccCloudflareRuleset_WAFBasic
=== PAUSE TestAccCloudflareRuleset_WAFBasic
=== RUN   TestAccCloudflareRuleset_WAFManagedRuleset
=== PAUSE TestAccCloudflareRuleset_WAFManagedRuleset
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetOWASP
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetOWASP
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetOWASPBlockXSSWithAnomalyOver60
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetOWASPBlockXSSWithAnomalyOver60
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetOWASPOnlyPL1
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetOWASPOnlyPL1
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetDeployMultiple
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetDeployMultiple
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetDeployMultipleWithSkip
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetDeployMultipleWithSkip
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetDeployMultipleWithTopSkipAndLastSkip
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetDeployMultipleWithTopSkipAndLastSkip
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetWithCategoryBasedOverrides
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetWithCategoryBasedOverrides
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetWithIDBasedOverrides
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetWithIDBasedOverrides
=== RUN   TestAccCloudflareRuleset_MagicTransitUpdateWithHigherPriority
    provider_test.go:160: Skipping acceptance test as 0da42c8d2132a9ddaf714f9e7c920711 is not configured for Magic Transit
--- SKIP: TestAccCloudflareRuleset_MagicTransitUpdateWithHigherPriority (0.00s)
=== RUN   TestAccCloudflareRuleset_WAFManagedRulesetWithPayloadLogging
=== PAUSE TestAccCloudflareRuleset_WAFManagedRulesetWithPayloadLogging
=== RUN   TestAccCloudflareRuleset_RateLimit
=== PAUSE TestAccCloudflareRuleset_RateLimit
=== RUN   TestAccCloudflareRuleset_TransformationRuleURIPath
=== PAUSE TestAccCloudflareRuleset_TransformationRuleURIPath
=== RUN   TestAccCloudflareRuleset_TransformationRuleURIQuery
=== PAUSE TestAccCloudflareRuleset_TransformationRuleURIQuery
=== RUN   TestAccCloudflareRuleset_TransformHTTPResponseHeaders
=== PAUSE TestAccCloudflareRuleset_TransformHTTPResponseHeaders
=== RUN   TestAccCloudflareRuleset_TransformationRuleURIPathAndQueryCombination
=== PAUSE TestAccCloudflareRuleset_TransformationRuleURIPathAndQueryCombination
=== RUN   TestAccCloudflareRuleset_TransformationRuleRequestHeaders
=== PAUSE TestAccCloudflareRuleset_TransformationRuleRequestHeaders
=== RUN   TestAccCloudflareRuleset_TransformationRuleResponseHeaders
=== PAUSE TestAccCloudflareRuleset_TransformationRuleResponseHeaders
=== RUN   TestAccCloudflareRuleset_ActionParametersMultipleSkips
=== PAUSE TestAccCloudflareRuleset_ActionParametersMultipleSkips
=== RUN   TestAccCloudflareRuleset_ActionParametersOverridesAction
=== PAUSE TestAccCloudflareRuleset_ActionParametersOverridesAction
=== RUN   TestAccCloudflareRuleset_ActionParametersHTTPDDoSOverride
=== PAUSE TestAccCloudflareRuleset_ActionParametersHTTPDDoSOverride
=== RUN   TestAccCloudflareRuleset_AccountLevelCustomWAFRule
=== PAUSE TestAccCloudflareRuleset_AccountLevelCustomWAFRule
=== RUN   TestAccCloudflareRuleset_ExposedCredentialCheck
=== PAUSE TestAccCloudflareRuleset_ExposedCredentialCheck
=== RUN   TestAccCloudflareRuleset_ConditionallySetActionParameterVersion
=== PAUSE TestAccCloudflareRuleset_ConditionallySetActionParameterVersion
=== CONT  TestAccCloudflareRuleset_WAFBasic
--- PASS: TestAccCloudflareRuleset_WAFBasic (11.06s)
=== CONT  TestAccCloudflareRuleset_TransformationRuleURIPath
--- PASS: TestAccCloudflareRuleset_TransformationRuleURIPath (11.20s)
=== CONT  TestAccCloudflareRuleset_ConditionallySetActionParameterVersion
--- PASS: TestAccCloudflareRuleset_ConditionallySetActionParameterVersion (20.06s)
=== CONT  TestAccCloudflareRuleset_ExposedCredentialCheck
--- PASS: TestAccCloudflareRuleset_ExposedCredentialCheck (10.13s)
=== CONT  TestAccCloudflareRuleset_AccountLevelCustomWAFRule
--- PASS: TestAccCloudflareRuleset_AccountLevelCustomWAFRule (12.19s)
=== CONT  TestAccCloudflareRuleset_ActionParametersHTTPDDoSOverride
--- PASS: TestAccCloudflareRuleset_ActionParametersHTTPDDoSOverride (11.69s)
=== CONT  TestAccCloudflareRuleset_ActionParametersOverridesAction
--- PASS: TestAccCloudflareRuleset_ActionParametersOverridesAction (16.67s)
=== CONT  TestAccCloudflareRuleset_ActionParametersMultipleSkips
--- PASS: TestAccCloudflareRuleset_ActionParametersMultipleSkips (11.88s)
=== CONT  TestAccCloudflareRuleset_TransformationRuleResponseHeaders
--- PASS: TestAccCloudflareRuleset_TransformationRuleResponseHeaders (10.93s)
=== CONT  TestAccCloudflareRuleset_TransformationRuleRequestHeaders
--- PASS: TestAccCloudflareRuleset_TransformationRuleRequestHeaders (11.21s)
=== CONT  TestAccCloudflareRuleset_TransformationRuleURIPathAndQueryCombination
--- PASS: TestAccCloudflareRuleset_TransformationRuleURIPathAndQueryCombination (11.43s)
=== CONT  TestAccCloudflareRuleset_TransformHTTPResponseHeaders
--- PASS: TestAccCloudflareRuleset_TransformHTTPResponseHeaders (11.08s)
=== CONT  TestAccCloudflareRuleset_TransformationRuleURIQuery
--- PASS: TestAccCloudflareRuleset_TransformationRuleURIQuery (11.37s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetDeployMultipleWithSkip
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetDeployMultipleWithSkip (11.36s)
=== CONT  TestAccCloudflareRuleset_RateLimit
--- PASS: TestAccCloudflareRuleset_RateLimit (11.55s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetWithPayloadLogging
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetWithPayloadLogging (10.70s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetWithIDBasedOverrides
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetWithIDBasedOverrides (11.81s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetWithCategoryBasedOverrides
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetWithCategoryBasedOverrides (11.32s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetDeployMultipleWithTopSkipAndLastSkip
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetDeployMultipleWithTopSkipAndLastSkip (11.16s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetOWASPBlockXSSWithAnomalyOver60
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetOWASPBlockXSSWithAnomalyOver60 (10.75s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetDeployMultiple
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetDeployMultiple (11.26s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetOWASPOnlyPL1
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetOWASPOnlyPL1 (11.86s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRulesetOWASP
--- PASS: TestAccCloudflareRuleset_WAFManagedRulesetOWASP (11.09s)
=== CONT  TestAccCloudflareRuleset_WAFManagedRuleset
--- PASS: TestAccCloudflareRuleset_WAFManagedRuleset (12.41s)
PASS
ok  	github.com/cloudflare/terraform-provider-cloudflare/cloudflare	286.687s
?   	github.com/cloudflare/terraform-provider-cloudflare/tools/cmd/changelog-check	[no test files]
?   	github.com/cloudflare/terraform-provider-cloudflare/version	[no test files]

@jacobbednarz
Copy link
Member Author

cc @zakcutner @uhthomas if you're interested

}

rules, err := buildRulesetRulesFromResource(d)
if err != nil {
return errors.Wrap(err, fmt.Sprintf("error building ruleset from resource"))
return errors.Errorf("error building ruleset rules from resource: %s", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fmt.Errorf and %w?

Same comment applies for other modifications in the file.

https://go.dev/blog/go1.13-errors

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally took this approach and I ended up with a circular %w directive can only be used in Errorf style error despite using Errorf! I’ll have another dig into this tomorrow after ☕ and see if I can make sense of it. Otherwise, we’ll roll with this as it works.

One of the earliest niggles with customers coming from the dashboard to
Terraform was the collision caused by a Ruleset phase being created in
the UI but the Terraform provider also needing to create the same
phase. This was problematic for a few reasons:

- The first was that when you deleted Ruleset rules in the UI, it didn't
  remove the phase. This was intentional but hidden behind the UI and
  only accessible via the API.
- Secondly, when customers wanted to use Terraform, there was an
  assumption they would be starting from scratch and many were not.
- Finally, in the event of a collision, we didn't know which Ruleset the
  customer wanted to use so we error'd out with a link to manually
  resolve which isn't a great solution but made the issue more
  prominent.

I had a chance to rethink through this issue and managed to find a way
that we improve all three points above and remove majority of the pain
points. With the proposed changes, the only time a customer needs to
manually resolve the Ruleset rules is if there are existing rules in the
UI which requires them to be deleted or migrated.

Achieving this requires the assumption that if the Ruleset has no rules,
it is ok to modify. Unfortunately, it's not that simple in practice. If
the phase already exists, we cannot just update it as the `name`
attribute is not writable after creation. Along with this, the `ref` and
`version` values will be automatically incremented causing a permadiff
in Terraform as the customer hasn't actually modified these values but
the backing service (and API) has. To work around this, if we find a
phase with no rules, we recreate it with the provided values which is
essentially the same the same thing as the "happy path" for a new
Terraform resource would be anyway.
@jacobbednarz jacobbednarz force-pushed the smoother-existing-rulesets-handling branch from 3d1900b to 7dc1827 Compare January 17, 2022 22:09
@jacobbednarz jacobbednarz merged commit 2123832 into master Jan 17, 2022
@jacobbednarz jacobbednarz deleted the smoother-existing-rulesets-handling branch January 17, 2022 23:28
github-actions bot pushed a commit that referenced this pull request Jan 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants