Skip to content

g-liu/parallel-scraping

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Parallel scraping

See my blog post for a tutorial!

Setup

Be sure you have PhantomJS > 1.9.2 installed globally, and GNU Parallel installed.

Run npm install && npm update.

To run the scraper:

chmod u+x scrape.sh # you only need to do this once
./scrape.sh

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published