Assign unknown client IPs a random geo ip in the contiguous US & cache the result #1608
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This assigns a random lat/long bounded within the contiguous US whenever the Director is unable to determine where the client is coming from. We're moving to this because the next step is to cache the assignment so that clients who fall into this trap can improve follow-up reads through Pelican caches, and because most of our client base falls within that box (and don't want to be redirected to caches in the UK, Russia, or Singapore).
This caches the randomly-assigned lat/long for a given unresolvable client IP, ensuring that repeated requests from the same client are delivered a mostly-consistent ordering of caches.
turetske
requested changes
Oct 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aims to improve the Director's cache sorting algorithm, specifically in the case of clients whose GeoIP resolution can't be determined.
With these changes, client IPs that don't have an entry in MaxMind entry (or that can't be parsed as an IP in the first case) are assigned a random location in the contiguous United States. Doing this makes the Director behave as though it knows the client's true location by delivering caches sorted based on the new location. This differs from the previous algorithm, where those IP addresses resulted in completely random cache sorting.
When a client IP is assigned a random location, that location is cached for 20 minutes, unless refreshed by further client interactions. This is done so that repetitive client requests get a consistent cache ordering (barring any other load information or the sort algorithm's stochastic jiggle), which should further improve cache hits.
Closes #1587