Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Amazon bot match to cover Amazonbot user agent #156

Merged
merged 2 commits into from
Sep 19, 2024
Merged

Update Amazon bot match to cover Amazonbot user agent #156

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
81b9bfc
cover Amazonbot in Amazon bot match
gabrieljablonski Sep 19, 2024
0660712
fix domain, add dns mock and update tests
gabrieljablonski Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ end
## Supported

- [Ahrefs](https://ahrefs.com/robot)
- [Amazonbot](https://developer.amazon.com/amazonbot)
- [Amazon AdBot](https://adbot.amazon.com/index.html)
- [Applebot](https://support.apple.com/en-us/119829)
- [Baidu spider](http://help.baidu.com/question?prod_en=master&class=498&id=1000973)
Expand Down
5 changes: 3 additions & 2 deletions lib/legitbot/amazon.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@

module Legitbot # :nodoc:
# https://adbot.amazon.com/index.html
# https://developer.amazon.com/amazonbot
class Amazon < BotMatch
domains 'amazonadbot.com.'
domains 'amazon.', 'amazonadbot.com.'
end

rule Legitbot::Amazon, %w[AmazonAdBot]
rule Legitbot::Amazon, %w[Amazonbot AmazonAdBot]
end
27 changes: 25 additions & 2 deletions test/amazon_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ def test_malicious_ua
refute_predicate bot, :valid?
end

def test_valid_ua
def test_user_agent1
bot = Legitbot.bot(
'Mozilla/5.0 (compatible; AmazonAdBot/1.0; +https://adbot.amazon.com)',
'54.166.7.90'
Expand All @@ -40,7 +40,19 @@ def test_valid_ua
assert_predicate bot, :valid?
end

def test_valid_name
# rubocop:disable Layout/LineLength
def test_user_agent2
bot = Legitbot.bot(
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML\, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)',
'52.70.240.171'
)

assert bot
assert_predicate bot, :valid?
end
# rubocop:enable Layout/LineLength

def test_valid_name1
bot = Legitbot.bot(
'Mozilla/5.0 (compatible; AmazonAdBot/1.0; +https://adbot.amazon.com)',
'54.166.7.90'
Expand All @@ -49,6 +61,17 @@ def test_valid_name
assert_equal :amazon, bot.detected_as
end

# rubocop:disable Layout/LineLength
def test_valid_name2
bot = Legitbot.bot(
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML\, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)',
'52.70.240.171'
)

assert_equal :amazon, bot.detected_as
end
# rubocop:enable Layout/LineLength

def test_fake_name
bot = Legitbot.bot(
'Mozilla/5.0 (compatible; AmazonAdBot/1.0; +https://adbot.amazon.com)',
Expand Down
6 changes: 6 additions & 0 deletions test/lib/dns_server_mock.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@
'54.166.7.90' => {
ptr: %w[crawler-54-166-7-90.amazonadbot.com]
},
'52-70-240-171.crawl.amazonbot.amazon' => {
a: %w[52.70.240.171]
},
'52.70.240.171' => {
ptr: %w[52-70-240-171.crawl.amazonbot.amazon]
},

# Apple
'17-58-98-60.applebot.apple.com' => {
Expand Down
Loading