Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): clear a WAF list when it cannot be deleted #908

Merged
merged 1 commit into from
Aug 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ A feature-rich and robust Cloudflare DDNS updater with a small footprint. The pr

- 😶‍🌫️ You can toggle [Cloudflare proxying](https://developers.cloudflare.com/dns/manage-dns-records/reference/proxied-dns-records/) for each domain.
- 📝 You can set [DNS record comments](https://developers.cloudflare.com/dns/manage-dns-records/reference/record-attributes/) (and probably record tags soon).
- 📜 The updater can maintain [custom lists](https://developers.cloudflare.com/waf/tools/lists/custom-lists/) of detected IP addresses. These lists can then be referenced in [Web Application Firewall (WAF)](https://developers.cloudflare.com/waf/) rules.
- 📜 The updater can maintain [lists](https://developers.cloudflare.com/waf/tools/lists/custom-lists/) of detected IP addresses. These lists can then be referenced in various Cloudflare products that use [Cloudflare’s Rules language](https://developers.cloudflare.com/ruleset-engine/), such as [Web Application Firewall (WAF)](https://developers.cloudflare.com/waf/) and [Cloudflare Rules](https://developers.cloudflare.com/rules/).

### 👁️ Integration with Notification Services

Expand Down Expand Up @@ -260,11 +260,11 @@ _(Click to expand the following items.)_

> You need to specify at least one thing in `DOMAINS`, `IP4_DOMAINS`, `IP6_DOMAINS`, or 🧪 `WAF_LISTS` for the updater to update.

| Name | Meaning |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DOMAINS` | Comma-separated fully qualified domain names or wildcard domain names that the updater should manage for both `A` and `AAAA` records. Listing a domain in `DOMAINS` is equivalent to listing the same domain in both `IP4_DOMAINS` and `IP6_DOMAINS`. |
| `IP4_DOMAINS` | Comma-separated fully qualified domain names or wildcard domain names that the updater should manage for `A` records |
| `IP6_DOMAINS` | Comma-separated fully qualified domain names or wildcard domain names that the updater should manage for `AAAA` records |
| Name | Meaning |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DOMAINS` | Comma-separated fully qualified domain names or wildcard domain names that the updater should manage for both `A` and `AAAA` records. Listing a domain in `DOMAINS` is equivalent to listing the same domain in both `IP4_DOMAINS` and `IP6_DOMAINS`. |
| `IP4_DOMAINS` | Comma-separated fully qualified domain names or wildcard domain names that the updater should manage for `A` records |
| `IP6_DOMAINS` | Comma-separated fully qualified domain names or wildcard domain names that the updater should manage for `AAAA` records |
| 🧪 `WAF_LISTS` | 🧪 Comma-separated references of [WAF lists](https://developers.cloudflare.com/waf/tools/lists/custom-lists/) the updater should manage. A list reference is written in the format `account-id/list-name` where `account-id` is your account ID and `list-name` is the list name; it should look like `0123456789abcdef0123456789abcdef/mylist`. If the referenced WAF list does not exist, the updater will try to create it. 💡 See [how to find your account ID](https://developers.cloudflare.com/fundamentals/setup/find-account-and-zone-ids/). |

> 🧪 The feature to manipulate WAF lists is experimental (introduced in 1.14.0) and is subject to changes. In particular, the updater currently deletes unmanaged IPs from WAF lists (e.g., deleting IPv6 addresses if you disable IPv6), but another reasonable implementation is to leave them alone. Please [open a GitHub issue](https://github.com/favonia/cloudflare-ddns/issues/new) to provide feedback. Thanks!
Expand Down Expand Up @@ -315,7 +315,7 @@ _(Click to expand the following items.)_
| Name | Meaning | Default Value |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- |
| `CACHE_EXPIRATION` | The expiration of cached Cloudflare API responses. It can be any positive time duration accepted by [time.ParseDuration](https://golang.org/pkg/time/#ParseDuration), such as `1h` or `10m`. | `6h0m0s` (6 hours) |
| `DELETE_ON_STOP` | Whether managed DNS records and WAF lists should be deleted on exit. It can be any boolean value accepted by [strconv.ParseBool](https://pkg.go.dev/strconv#ParseBool), such as `true`, `false`, `0` or `1`. 🐞🧪 **KNOWN ISSUE: if a WAF list is used in a rule, you cannot delete it.** In future versions, the updater may attempt to empty a list when it fails to delete it, but this has not been implemented yet. | `false` |
| `DELETE_ON_STOP` | Whether managed DNS records and WAF lists should be deleted on exit. It can be any boolean value accepted by [strconv.ParseBool](https://pkg.go.dev/strconv#ParseBool), such as `true`, `false`, `0` or `1`. If a WAF list is used in a rule expression, the updater cannot delete it and will attempt to clear it instead. | `false` |
| `TZ` | The timezone used for logging messages and parsing `UPDATE_CRON`. It can be any timezone accepted by [time.LoadLocation](https://pkg.go.dev/time#LoadLocation), including any IANA Time Zone. 🤖 The pre-built Docker images come with the embedded timezone database via the [time/tzdata](https://pkg.go.dev/time/tzdata) package. | `UTC` |
| `UPDATE_CRON` | The schedule to re-check IP addresses and update DNS records and WAF lists (if needed). The format is [any cron expression accepted by the `cron` library](https://pkg.go.dev/github.com/robfig/cron/v3#hdr-CRON_Expression_Format) or the special value `@once`. The special value `@once` means the updater will terminate immediately after updating the DNS records or WAF lists, effectively disabling the scheduling feature. 🤖 The update schedule _does not_ take the time to update records into consideration. For example, if the schedule is `@every 5m`, and if the updating itself takes 2 minutes, then the actual interval between adjacent updates is 3 minutes, not 5 minutes. | `@every 5m` (every 5 minutes) |
| `UPDATE_ON_START` | Whether to check IP addresses (and possibly update DNS records and WAF lists) _immediately_ on start, regardless of the update schedule specified by `UPDATE_CRON`. It can be any boolean value accepted by [strconv.ParseBool](https://pkg.go.dev/strconv#ParseBool), such as `true`, `false`, `0` or `1`. | `true` |
Expand Down
8 changes: 4 additions & 4 deletions cmd/ddns/ddns.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@
func stopUpdating(ctx context.Context, ppfmt pp.PP, c *config.Config, s setter.Setter) {
if c.DeleteOnStop {
msg := updater.DeleteIPs(ctx, ppfmt, c, s)
monitor.LogMessageAll(ctx, ppfmt, c.Monitors, msg)
notifier.SendMessageAll(ctx, ppfmt, c.Notifiers, msg)
monitor.LogMessageAll(ctx, ppfmt, c.Monitors, msg.MonitorMessage)
notifier.SendMessageAll(ctx, ppfmt, c.Notifiers, msg.NotifierMessage)

Check warning on line 61 in cmd/ddns/ddns.go

View check run for this annotation

Codecov / codecov/patch

cmd/ddns/ddns.go#L60-L61

Added lines #L60 - L61 were not covered by tests
}
}

Expand Down Expand Up @@ -125,8 +125,8 @@
monitor.SuccessAll(ctx, ppfmt, c.Monitors, "Started (no action)")
} else {
msg := updater.UpdateIPs(ctxWithSignals, ppfmt, c, s)
monitor.PingMessageAll(ctx, ppfmt, c.Monitors, msg)
notifier.SendMessageAll(ctx, ppfmt, c.Notifiers, msg)
monitor.PingMessageAll(ctx, ppfmt, c.Monitors, msg.MonitorMessage)
notifier.SendMessageAll(ctx, ppfmt, c.Notifiers, msg.NotifierMessage)

Check warning on line 129 in cmd/ddns/ddns.go

View check run for this annotation

Codecov / codecov/patch

cmd/ddns/ddns.go#L128-L129

Added lines #L128 - L129 were not covered by tests
}

if ctxWithSignals.Err() != nil {
Expand Down
15 changes: 12 additions & 3 deletions internal/api/base.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,11 @@ type Handle interface {
ListRecords(ctx context.Context, ppfmt pp.PP, ipNet ipnet.Type, domain domain.Domain) ([]Record, bool, bool)

// DeleteRecord deletes one DNS record.
DeleteRecord(ctx context.Context, ppfmt pp.PP, ipNet ipnet.Type, domain domain.Domain, id ID) bool
//
// Note: the keepCacheWhenFails is to optimize the deletion when exiting the program. The cache
// from list names to list IDs should not be cleared if we are only deleting things.
DeleteRecord(ctx context.Context, ppfmt pp.PP, ipNet ipnet.Type, domain domain.Domain, id ID,
keepCacheWhenFails bool) bool

// UpdateRecord updates one DNS record.
UpdateRecord(ctx context.Context, ppfmt pp.PP, ipNet ipnet.Type, domain domain.Domain, id ID, ip netip.Addr) bool
Expand All @@ -64,8 +68,13 @@ type Handle interface {
// The second return value indicates whether the list already exists.
EnsureWAFList(ctx context.Context, ppfmt pp.PP, list WAFList, description string) (ID, bool, bool)

// DeleteWAFList deletes a WAF list with IP ranges.
DeleteWAFList(ctx context.Context, ppfmt pp.PP, list WAFList) bool
// ClearWAFListAsync deletes or clears a WAF list with IP ranges.
// The first return value indicates whether the list was deleted: If it's true, then it's deleted.
// If it's false, then it's being cleared asynchronously instead of being deleted.
//
// Note: the keepCacheWhenFails is to optimize the deletion when exiting the program. The cache
// from list names to list IDs should not be cleared if we are only deleting things.
ClearWAFListAsync(ctx context.Context, ppfmt pp.PP, list WAFList, keepCacheWhenFails bool) (bool, bool)

// ListWAFListItems retrieves a WAF list with IP rages.
//
Expand Down
6 changes: 4 additions & 2 deletions internal/api/cloudflare_records.go
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ func (h CloudflareHandle) ListRecords(ctx context.Context, ppfmt pp.PP,

// DeleteRecord calls cloudflare.DeleteDNSRecord.
func (h CloudflareHandle) DeleteRecord(ctx context.Context, ppfmt pp.PP,
ipNet ipnet.Type, domain domain.Domain, id ID,
ipNet ipnet.Type, domain domain.Domain, id ID, keepCacheWhenFails bool,
) bool {
zone, ok := h.ZoneOfDomain(ctx, ppfmt, domain)
if !ok {
Expand All @@ -155,7 +155,9 @@ func (h CloudflareHandle) DeleteRecord(ctx context.Context, ppfmt pp.PP,
ppfmt.Noticef(pp.EmojiError, "Failed to delete a stale %s record of %q (ID: %s): %v",
ipNet.RecordType(), domain.Describe(), id, err)

h.cache.listRecords[ipNet].Delete(domain.DNSNameASCII())
if !keepCacheWhenFails {
h.cache.listRecords[ipNet].Delete(domain.DNSNameASCII())
}

return false
}
Expand Down
8 changes: 4 additions & 4 deletions internal/api/cloudflare_records_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -593,14 +593,14 @@ func TestDeleteRecordValid(t *testing.T) {
drh := newDeleteRecordHandler(t, mux, "record1", ipnet.IP6, "sub.test.org", "::1")
drh.setRequestLimit(1)

ok = h.DeleteRecord(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"), "record1")
ok = h.DeleteRecord(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"), "record1", false)
require.True(t, ok)
require.True(t, drh.isExhausted())

drh.setRequestLimit(1)
mockPP = mocks.NewMockPP(mockCtrl)
h.ListRecords(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"))
_ = h.DeleteRecord(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"), "record1")
_ = h.DeleteRecord(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"), "record1", false)
rs, cached, ok := h.ListRecords(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"))
require.True(t, ok)
require.True(t, cached)
Expand All @@ -625,7 +625,7 @@ func TestDeleteRecordInvalid(t *testing.T) {
api.ID("record1"),
gomock.Any(),
)
ok = h.DeleteRecord(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"), "record1")
ok = h.DeleteRecord(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"), "record1", false)
require.False(t, ok)
}

Expand All @@ -641,7 +641,7 @@ func TestDeleteRecordZoneInvalid(t *testing.T) {
"sub.test.org",
gomock.Any(),
)
ok = h.DeleteRecord(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"), "record1")
ok = h.DeleteRecord(context.Background(), mockPP, ipnet.IP6, domain.FQDN("sub.test.org"), "record1", false)
require.False(t, ok)
}

Expand Down
Loading
Loading