Support for Twitterbot

v0.4.1

Bump to v0.4.1

"Declarative"

This is severe refactoring of Legitbot code to make it much easier to define new bots in a Ruby idiomatic declarative way. The public interface was still left intact to avoid breaking existing library consumers.

Fix DuckDuckGo user agent

v0.3.2

Bump to v0.3.2

Update the list of DuckDuckGo IPs

v0.3.1

Bump to v0.3.1

New bot added

• AhrefsBot

Switch to "interval_tree"

Legitbot needs to quickly determine if an IP belongs to a set of ranges, because that's how Facebook returns its valid IPs. Issue #10 uncovered a bug in "segment_tree" library Legitbot was using to make this lookup. The most practical way of resolving the issue turned out to switch to another library "interval_tree".

Since the public API of Legitbot did not change, only the minor version number was incremented.

Minor changes

• Update the list of IPs for DuckDuckBot

Minor changes

• Update the list of Facebook user agents.

Minor fixes

• It turns out that Yandex's list of bot user agents changes over time. So, the list in "legitbot" is updated.
• "legitbot" used to select the first matched rule in case of several matches. This behavior will be confusing in case of custom rules. So, now "legitbot" chooses the last match.

Bug fixes

• Some Android browsers can have Pinterest extension or plugin. Legitbot was wrongly detecting such requests as "Pinterest" bot.
• Some IP addresses do not have reverse records. Fixed.