Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ClaudeBot to bad_bots + reg.ru to spam_ref #202

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

ShaiMagal
Copy link

@ShaiMagal ShaiMagal commented May 13, 2024

Match "ClaudeBot" user-agent. Very annoying flood/crawl bot.

cat apache-access.log | grep ClaudeBot | wc -l
185447570

Match "reg.ru" referer. Spamming every day.

cat apache-access.log | grep reg.ru | wc -l
10779030

@ShaiMagal
Copy link
Author

ShaiMagal commented May 14, 2024

@mitchellkrogza Can we approve it please? :) This bota are crazy, still spamming :/

@ShaiMagal ShaiMagal changed the title Add ClaudeBot to bad_bots Add ClaudeBot to bad_bots + reg.ru to spam_ref May 14, 2024
@ShaiMagal
Copy link
Author

ShaiMagal commented May 23, 2024

UPDATE:

still very hard spamming:

domain.tld 111.222.333.444 - - [23/May/2024:19:00:12 +0200] "GET /XXXy?p=110&q=Farba-Biela-Ru%C5%BEov%C3%A1%2FV%C3%BDkon-35+W+3+2242424+asdasd HTTP/2.0" 200 41382 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"

And a lot of random strings...

@mitchellkrogza

UPDATE2:
more people have same problem (Google is full of this stories)
https://www.phpbb.com/community/viewtopic.php?t=2652265
https://www.reddit.com/r/singularity/comments/1cdm97j/anthropics_claudebot_is_aggressively_scraping_the/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant