Use fresh top500 list from moz.com #10
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Refresh top500 list every time sites.py is imported. Keep cached list in top500.json and use it as a backup.
oh2fih
force-pushed
the
dynamic-top500-sites
branch
from
0a69e3f to
4679524
Compare
oh2fih
force-pushed
the
dynamic-top500-sites
branch
from
4679524 to
ed25a4f
Compare
Since the GitHub workflow updates the site daily at the same time, the CACHE_MAX_AGE has to be lower than 24 hours.
|
@OllieJC It is good to see findsecuritycontacts.com is updating daily again! Could you check this PR out to keep the domain list up-to-date, too? I just checked (with |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current top500 site list is rather old (from Feb 8, 2022). This PR converts the static list into a dynamic one.
sites.pyas a class that handles cache of top500 sites intop500.json.