Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(ovh): add cache based on DNS zone SOA value #3026

Merged
merged 1 commit into from
Aug 12, 2023
Merged

feat(ovh): add cache based on DNS zone SOA value #3026

merged 1 commit into from
Aug 12, 2023

Conversation

rbeuque74
Copy link
Contributor

Hello external-dns maintainers, OVHcloud API team here 👋

Description
This pull-request patches the 'ovh' provider to add a small local cache of the fetched DNS records.

Context: we detected that numerous of our customers use your external-dns software to manage their DNS records, sending waves of API queries to our APIs. As these records are retrieved one by one, it generates a high amount of requests on our backend.

As we would like to keep allowing our customers using external-dns with the best experience, we leveraged the usage of the DNS zone SOA Serial to detect if the zone records might have changed since the last time we generated the cache.
This way, we spare all the calls to retrieve the records that have been kept in cache, and serve them directly, since the SOA Serial haven't changed, guaranteeing that the DNS zone hasn't changed on our side.

This will also add a speed-up once the cache is warm, as long as the zone doesn't change.

This should also prevent users of external-dns from being targeted by potential rate-limiting in case of high activity on our backend.

Thanks in advance for your review.
Romain

Checklist

  • Unit tests updated
  • End user documentation updated

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Sep 15, 2022 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: rbeuque74 / name: Romain Beuque (1a48cb8)

@k8s-ci-robot k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Sep 15, 2022
@k8s-ci-robot
Copy link
Contributor

Welcome @rbeuque74!

It looks like this is your first PR to kubernetes-sigs/external-dns 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/external-dns has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Sep 15, 2022
@rbeuque74
Copy link
Contributor Author

Hello,
Kind reminder about this pull-request.
Feel free to tell me if you want me to do some modification !

Best,
Romain

@rbeuque74
Copy link
Contributor Author

/auto-cc /cc @szuecs

@scraly
Copy link

scraly commented Oct 17, 2022

Hi, any news aboutt his PR? :)
Thanks

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 11, 2022
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 28, 2022
@rbeuque74
Copy link
Contributor Author

/assign @njuettner
/assign @szuecs

@rbeuque74
Copy link
Contributor Author

/assign @seanmalloy

@szuecs
Copy link
Contributor

szuecs commented Dec 19, 2022

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rbeuque74, szuecs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 19, 2022
@szuecs
Copy link
Contributor

szuecs commented Dec 19, 2022

/assign @njuettner

@szuecs
Copy link
Contributor

szuecs commented Dec 19, 2022

I don't have the button to run the tests, so assigned @njuettner

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 8, 2023
@Raffo
Copy link
Contributor

Raffo commented Jan 8, 2023

@rbeuque74 can you rebase so that I can let the test run and review the PR? Thanks!

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 9, 2023
@rbeuque74 rbeuque74 requested review from Raffo and removed request for seanmalloy January 12, 2023 09:13
@rbeuque74
Copy link
Contributor Author

Hello,
Any news regarding this PR ?

Thanks,
Romain

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 11, 2023
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 9, 2023
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 12, 2023
@rbeuque74
Copy link
Contributor Author

Hello,

PR is rebased.
Thanks,
Romain

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 12, 2023
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 24, 2023
@szuecs
Copy link
Contributor

szuecs commented Jun 30, 2023

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Jun 30, 2023
@szuecs
Copy link
Contributor

szuecs commented Jun 30, 2023

Please rebase

@szuecs szuecs assigned rbeuque74 and unassigned seanmalloy, njuettner and szuecs Jun 30, 2023
@rbeuque74
feat(ovh): add cache based on DNS zone SOA value
1a48cb8 
OVHcloud customers using 'ovh' provider for large DNS zone will spend a
lot of time waiting for all records to be retrieved from the API. As
records can't change without a modification of the SOA Serial of the DNS
zone, this commit adds a client-side cache that stores the value of the
DNS records for each zone, based of the current SOA Serial value of the
zone. In case the SOA Serial of the zone is identical to the one in
cache, we can serve all the records directly from cache. Otherwise,
fallback on refreshing the cache from the API.

Signed-off-by: Romain Beuque <556072+rbeuque74@users.noreply.github.com>
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 17, 2023
@rbeuque74
Copy link
Contributor Author

Hello,
pull-request is now rebased, thanks for approving the test run :)

/assign @szuecs

Romain

@johngmyers
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 12, 2023
@johngmyers
Copy link
Contributor

/test pull-external-dns-lint
/test pull-external-dns-unit-test

@johngmyers
Copy link
Contributor

/retest

@k8s-ci-robot k8s-ci-robot merged commit 13d09b2 into kubernetes-sigs:master Aug 12, 2023
1 check passed
@rbeuque74 rbeuque74 deleted the ovhcloud-cache branch October 12, 2023 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Raffo Raffo Awaiting requested review from Raffo

Assignees

@szuecs szuecs

@rbeuque74 rbeuque74

@johngmyers johngmyers

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@rbeuque74 @k8s-ci-robot @scraly @szuecs @Raffo @k8s-triage-robot @johngmyers @seanmalloy @njuettner