First git clone https://github.com/spider-rs/spider.git and cd spider.
Simple concurrent crawl Simple.
cargo run --example example
Subscribe to realtime changes Subscribe.
cargo run --example subscribe
Live handle index mutation example Callback.
cargo run --example callback
Enable log output Debug.
cargo run --example debug
Scrape the webpage with and gather html Scrape.
cargo run --example scrape
Scrape and download the html file to fs Download HTML. *Note: Enable feature flag [full_resources] to gather all files like css, jss, and etc.
cargo run --example download
Scrape and download html to react components and store to fs Download to React Component.
cargo run --example download_to_react
Crawl the page and output the links via Serde.
cargo run --example serde --features serde
Crawl links with a budget of amount of pages allowed Budget.
cargo run --example budget