Blocklist for newly created scam, phishing, and malware domains automatically retrieved daily using Google Search API, automated detection, and public databases.
This blocklist aims to detect new malicious domains within a short period of their registration date. Since the project began, the blocklist has expanded to include not only scam websites but also malware domains.
For extended protection, use xRuffKez's NRD Lists to block all newly registered domains (NRDs), and Hagezi's Threat Intelligence Feed (full version) which includes this blocklist.
Sources include:
- Public databases
- Google Search indexing to find common scam site templates
- Detection of common cybersquatting techniques like typosquatting, doppelganger domains, and IDN homograph attacks using dnstwist and URLCrazy
- Domain generation algorithm (DGA) domain detection using DGA Detector
- Regex expression matching for phishing NRDs. See the list of expressions here
A list of all sources can be found in SOURCES.md.
The automated retrieval is done daily at 16:00 UTC.
| Format | Syntax |
|---|---|
| Adblock Plus | ||scam.com^ |
| Wildcard Domains | scam.com |
Total domains: 217124
Light version: 20372
New domains after filtering:
Today | Monthly | %Monthly | %Filtered | Source
11 | 11 | 0 % | 37 % | 165 Anti-fraud
80 | 80 | 3 % | 13 % | Artists Against 419
0 | 0 | 0 % | 18 % | Česká Obchodní Inspekce
49 | 49 | 1 % | 1 % | Cybersquatting
873 | 873 | 35 % | 0 % | DGA Detector
35 | 35 | 1 % | 21 % | Emerging Threats
0 | 0 | 0 % | 20 % | FakeWebshopListHUN
30 | 30 | 1 % | 3 % | Google Search
146 | 146 | 5 % | 7 % | Gridinsoft
375 | 375 | 15 % | 10 % | Jeroengui
25 | 25 | 1 % | 0 % | Jeroengui (NRDs)
355 | 355 | 14 % | 25 % | PhishStats
30 | 30 | 1 % | 0 % | PhishStats (NRDs)
7 | 7 | 0 % | 10 % | PuppyScams.org
461 | 461 | 18 % | 1 % | Regex Matching
39 | 39 | 1 % | 7 % | Scam Directory
0 | 0 | 0 % | 32 % | ScamAdviser
0 | 0 | 0 % | 5 % | StopGunScams.com
0 | 0 | 0 % | 3 % | Verbraucherzentrale Hamburg
0 | 0 | 0 % | 31 % | ViriBack C2 Tracker
2461 | 2461 | 100 % | 22 % | All sources
- %Monthly: percentage out of total domains from all sources.
- %Filtered: percentage of dead, whitelisted, and parked domains.
Dead domains removed today: 2005
Resurrected domains added today: 1347
Parked domains removed this month: 0
Unparked domains added today: 98
Domains over time (days)
Courtesy of iam-py-test/blocklist_stats.
- Domains are filtered against an actively maintained whitelist
- Domains are checked against the Tranco Top Sites Ranking for potential false positives which are then vetted manually
- Common subdomains like 'www' are stripped
- Non-domain entries are removed
- Redundant rules are removed via wildcard matching. For example, 'abc.example.com' is a wildcard match of 'example.com' and, therefore, is redundant and removed. Wildcards are occasionally added to the blocklist manually to further optimize the number of entries
Entries that require manual verification/intervention are notified to the maintainer for fast remediations.
The full filtering process can be viewed in the repository's code.
Dead domains are removed daily using AdGuard's Dead Domains Linter.
Dead domains that are resolving again are included back into the blocklist.
Parked domains are removed weekly while unparked domains are added back daily. A list of common parked domain messages is used to automatically detect parked domains. This list can be viewed here: parked_terms.txt.
Parked sites no longer containing any of the parked messages are assumed to be unparked.
For collated blocklists cautious about size, a light version of the blocklist is available in the lists directory. Sources excluded from the light version are marked in SOURCES.md.
Note that dead and parked domains that become alive/unparked are not added back into the light version due to limitations in how these domains are recorded.
A blocklist for NSFW domains is available in Adblock Plus format here: nsfw.txt.
Details
- Domains are automatically retrieved from the Tranco Top Sites Ranking daily
- Dead domains are removed daily
- Note that resurrected domains are not added back
- Note that parked domains are not checked for
This blocklist does not just include adult videos, but also NSFW content of the artistic variety (rule34, illustrations, etc).
For list maintainers interested in using the parked domains as a source, the list of parked domains can be found here: parked_domains.txt. This list is capped at 50,000 domains.
- AdGuard's Hostlist Compiler: simple tool that compiles hosts blocklists and removes redundant rules
- Elliotwutingfeng's repositories: various original blocklists
- Google's Shell Style Guide: Shell script style guide
- Grammarly: spelling and grammar checker
- Hagezi's DNS Blocklists: various curated blocklists including threat intelligence feeds
- Jarelllama's Blocklist Checker: generate a simple static report for blocklists or see previous reports of requested blocklists
- ShellCheck: static analysis tool for Shell scripts
- VirusTotal: analyze suspicious files, domains, IPs, and URLs to detect malware (also includes WHOIS lookup)
- iam-py-test/blocklist_stats: statistics on various blocklists