Skip to content

Latest commit

 

History

History
17 lines (11 loc) · 445 Bytes

README.md

File metadata and controls

17 lines (11 loc) · 445 Bytes

Parallel scraping

See my blog post for a tutorial!

Setup

Be sure you have PhantomJS > 1.9.2 installed globally, and GNU Parallel installed.

Run npm install && npm update.

To run the scraper:

chmod u+x scrape.sh # you only need to do this once
./scrape.sh