Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a limit to the number of journal entries requestable in an NRTM response #742

Closed
mxsasha opened this issue Feb 6, 2023 · 0 comments · Fixed by #745
Closed

Add a limit to the number of journal entries requestable in an NRTM response #742

mxsasha opened this issue Feb 6, 2023 · 0 comments · Fixed by #745
Assignees
@mxsasha
Milestone
Release 4.3

Comments

@mxsasha
Copy link
Collaborator

mxsasha commented Feb 6, 2023 •
edited
Loading

When mirror clients lag behind very far (e.g. months), they will make large NRTM requests. The server will take some time to reply, and consume quite some resources, and in the mean time the client may time out, only to make the same large query on the next attempt. This can cause load issues for the mirror server. The best option is for the client to reload their full data, but there is no way to signal that in NRTMv3. Therefore, the server should be allowed to reject the request, which will protect the server and probably lead to manual intervention from the mirror client operator.

This will be a new optional setting per source, with a maximum serial range per query. Probably I can include #634 with this as well, and use the same setting.

Note that #691 also "fixes" this problem, as serials can not be requested when they have been deleted, but operators should not need to delete things from their local journal to prevent excessive queries.
@mxsasha mxsasha added this to the Release 4.3 milestone Feb 6, 2023
@mxsasha mxsasha self-assigned this Feb 6, 2023
mxsasha added a commit that referenced this issue Feb 10, 2023
@mxsasha
Fix #742 - Add setting to limit NRTM response size
b175c01
@mxsasha mxsasha linked a pull request Feb 10, 2023 that will close this issue
Fix #742 - Add setting to limit NRTM response size #745
Merged
mxsasha added a commit that referenced this issue Feb 10, 2023
@mxsasha
Fix #742 - Add setting to limit NRTM response size
217f400
mxsasha added a commit that referenced this issue Feb 10, 2023
@mxsasha
Fix #742 - Add setting to limit NRTM response size (#745)
0e33e13
mergify bot pushed a commit that referenced this issue Feb 15, 2023
@mxsasha @mergify
Fix #742 - Add setting to limit NRTM response size (#745)
37836bb 
(cherry picked from commit 0e33e13)

# Conflicts:
#	docs/releases/4.3.0.rst
#	irrd/conf/known_keys.py
#	irrd/conf/test_conf.py
#	irrd/mirroring/nrtm_generator.py
mxsasha added a commit that referenced this issue Feb 15, 2023
@mxsasha
Fix #742 - Add setting to limit NRTM response size (#745)
d324d05 
(cherry picked from commit 0e33e13)

# Conflicts:
#	docs/releases/4.3.0.rst
#	irrd/conf/known_keys.py
#	irrd/conf/test_conf.py
#	irrd/mirroring/nrtm_generator.py
mxsasha added a commit that referenced this issue Feb 15, 2023
@mergify @mxsasha
[4.2.x] Fix #742 - Add setting to limit NRTM response size (backport #…
6093f6d 
…745) (#751)

(cherry picked from commit 0e33e13)

Co-authored-by: Sasha Romijn <github@mxsasha.eu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mxsasha mxsasha

Labels
None yet
Projects
None yet
Milestone
Release 4.3
Development

Successfully merging a pull request may close this issue.

Fix #742 - Add setting to limit NRTM response size irrdnet/irrd
1 participant
@mxsasha