Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[management] Skip account peers update if no changes affect peers #2310

Merged

Conversation

bcmmbaga
Copy link
Contributor

@bcmmbaga bcmmbaga commented Jul 23, 2024

Describe your changes

This PR optimizes account peer updates to trigger only when necessary, reducing unnecessary network map processing and updates. These changes improve efficiency by avoiding redundant updates while ensuring all necessary updates are still performed.

Note: Network map updates are sent only if the new network map differs from the last one sent, except for TURN credential refresh updates. This occurs when the network map has been updated and has a higher serial number than the previous one.

Changes:

  1. Posture Checks:

    • Creating new posture checks no longer triggers account peer updates or sends network map updates.
    • Updating posture checks only triggers account peer updates when they are attached to a policy.
    • Delete posture check is allowed only if it is not linked to a policy.
  2. Policies:

    • Saving a policy with empty rules (no source or destination groups) does not trigger account peer updates.
    • Updating a policy triggers account peer updates and sends network maps to peers, but only if the new network map differs from the last sent one.
    • Deleting a policy always triggers peer updates and sends the updated network map to peers. (not changed)
  3. Setup Keys:

    • Saving a setup key does not trigger account peer updates.
  4. Groups:

    • Saving a group not used in nameserver groups, policies, or routes does not trigger account peer updates.
    • Adding or removing a peer to a group triggers account peer updates if the group is in use with DNS, policies, or routes.
    • Deleting a group does not trigger account peer updates. This is true because a group can only be deleted when it is not linked to any resource.
  5. DNS Settings:

    • Saving DNS settings triggers account peer updates when before/after groups has peers. Updates will be checked against the sent map and skipped if they are the same.
  6. NameServer Groups:

    • Creating a new nameserver group triggers account peer updates when distribution groups have peers.
    • Updating a nameserver group triggers account peer updates when the before or after state of distribution groups have peers.
    • Deleting a nameserver group triggers account peer updates when distribution groups have peers.
  7. Routes:

    • Creating a route triggers account peer updates when distribution groups or routing groups are not empty or there is a routing peer.
    • Saving (updating) a route triggers account peer updates when before and after distribution groups or routing groups are not empty or there is a routing peer.
    • Deleting a route triggers account peer updates when distribution groups or routing groups are not empty or there is a routing peer.
  8. Users:

    • Creating new users no longer triggers account peer updates.
    • Updating existing users triggers account peer updates if the user has a linked peer and groups propagation is enabled.
    • Deleting a regular user triggers account peer updates if there is a peer linked to the user
  9. Peers:

    • Adding a peer triggers account peer updates if the peer is part of a group used in active DNS, routes, or ACL.
    • Updating a peer does not trigger account peer updates if the peer is not expired and peer login expiration is not enabled.
    • Deleting a peer triggers account peer updates if the peer was part of a group used in active DNS, routes, or policies.
  10. Removed unused UpdatePeerSSHKey method.

Issue ticket number and link

Checklist

  • Is it a bug fix
  • Is a typo/documentation fix
  • Is a feature enhancement
  • It is a refactor
  • Created tests that fail without the change (if possible)
  • Extended the README / documentation, if necessary

@bcmmbaga bcmmbaga changed the title Skip account peer update if no changes affect peers Skip account peers update if no changes affect peers Jul 23, 2024
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
@bcmmbaga bcmmbaga force-pushed the feature/validate-group-association branch from a9ac02a to ea6d037 Compare October 21, 2024 14:26
bcmmbaga and others added 12 commits October 21, 2024 17:30
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
@bcmmbaga bcmmbaga force-pushed the feature/validate-group-association branch from 85ffbd1 to d8e4ea4 Compare October 22, 2024 15:20
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
@bcmmbaga bcmmbaga force-pushed the feature/validate-group-association branch from d8e4ea4 to 74e2986 Compare October 22, 2024 15:21
Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
@bcmmbaga bcmmbaga force-pushed the feature/validate-group-association branch from de86e3c to 27afc61 Compare October 22, 2024 15:45
Copy link

@bcmmbaga bcmmbaga merged commit 06082b0 into feature/optimize-network-map-updates Oct 23, 2024
21 checks passed
@bcmmbaga bcmmbaga deleted the feature/validate-group-association branch October 23, 2024 08:56
bcmmbaga added a commit that referenced this pull request Oct 23, 2024
* Skip peer update on unchanged network map (#2236)

* Enhance network updates by skipping unchanged messages

Optimizes the network update process
by skipping updates where no changes in the peer update message received.

* Add unit tests

* add locks

* Improve concurrency and update peer message handling

* Refactor account manager network update tests

* fix test

* Fix inverted network map update condition

* Add default group and policy to test data

* Run peer updates in a separate goroutine

* Refactor

* Refactor lock

* Fix peers update by including NetworkMap and posture Checks

* go mod tidy

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* [management] Skip account peers update if no changes affect peers (#2310)

* Remove incrementing network serial and updating peers after group deletion

* Update account peer if posture check is linked to policy

* Remove account peers update on saving setup key

* Refactor group link checking into re-usable functions

* Add HasPeers function to group

* Refactor group management

* Optimize group change effects on account peers

* Update account peers if ns group has peers

* Refactor group changes

* Optimize account peers update in DNS settings

* Optimize update of account peers on jwt groups sync

* Refactor peer account updates for efficiency

* Optimize peer update on user deletion and changes

* Remove condition check for network serial update

* Optimize account peers updates on route changes

* Remove UpdatePeerSSHKey method

* Remove unused isPolicyRuleGroupsEmpty

* Add tests for peer update behavior on posture check changes

* Add tests for peer update behavior on policy changes

* Add tests for peer update behavior on group changes

* Add tests for peer update behavior on dns settings changes

* Refactor

* Add tests for peer update behavior on name server changes

* Add tests for peer update behavior on user changes

* Add tests for peer update behavior on route changes

* fix tests

* Add tests for peer update behavior on setup key changes

* Add tests for peer update behavior on peers changes

* fix merge

* Fix tests

* go mod tidy

* Add NameServer and Route comparators

* Update network map diff logic with custom comparators

* Add tests

* Refactor duplicate diff handling logic

* fix linter

* fix tests

* Refactor policy group handling and update logic.

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update route check by checking if group has peers

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor posture check policy linking logic

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Simplify peer update condition in DNS management

Refactor the condition for updating account peers to remove redundant checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add posture checks tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix user and setup key tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix account and route tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix typo

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix nameserver tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix routes tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix group tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* upgrade diff package

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix nameserver tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* use generic differ for netip.Addr and netip.Prefix

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* go mod tidy

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add peer tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix merge

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix management suite tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* fix postgres tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* enable diff nil structs comparison

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip the update only last sent the serial is larger

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* refactor peer and user

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip spell check for groupD

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Refactor group, ns group, policy and posture checks

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* skip spell check for GroupD

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* update account policy check before verifying policy status

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* Update management/server/route_test.go

Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

* add tests missing tests for dns setting groups

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add tests for posture checks changes

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add ns group and policy tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add route and group tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* increase Linux test timeout to 10 minutes

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Run diff for client posture checks only

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* add panic recovery and detailed logging in peer update comparison

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

* Fix tests

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>

---------

Signed-off-by: bcmmbaga <bethuelmbaga12@gmail.com>
Co-authored-by: Maycon Santos <mlsmaycon@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants