dyer is designed for reliable, flexible and fast Request-Response based service, including data processing, web-crawling and so on, providing some friendly, interoperable, comprehensive features without compromising speed.
- asynchronous, concurrent streaming and I/O, make the best of thread pool, network, and system resource.
- Event-driven, once you set the initials and recursive generator,
dyerwill handle the rest of it interoperably. - User-friendly and flexible,
dyeroffers high-level, flexible, easy to use wrappers and APIs what does a lot for you.
To reduce code redundancy and speed up compilation, dyer use feature flag to mark the necessary modules/functions, Currently here are some supported Features:
xpath-stable: stably parse the response html with third-party C librarylibxml2xpath-alpha: rust-native parse the html response, NOTE that it is experimental and unstale, remains to be improvedcompression: Enable HTTP Compression:br,deflate,gzipproxy: Enable use proxiesfull: Enable all features
Get started by installing dyer-cli and looking over the examples and quick start.
Crates: Link 🔗
Documentation: Link 🔗
The Cookbook gives a detailed view of dyer.