A rust ported robotstxt

[Folyd]

I have ported a Rust version of this library recently. The Rust version keeps the same behavior of the original library, provides a consistent API and passed 100% C++ testing cases via Rust FFI.

Also, Rust version licensed with Apache 2.0 too, and I have preserved existing copyright notices and licensing within the appropriate source files.

Here is my repository: https://github.com/Folyd/robotstxt