-
Notifications
You must be signed in to change notification settings - Fork 18
CLI Support
Sometimes we need to start our scraping work automatically or periodically.
You can setup a bat
or sh
file containing the following command and setup a task scheduler or even start your scraping work programmatically.
chrome "chrome-extension://[ExtensionID]/devtools/devtools_scraper_panel.html?run=[sitemapid]&[anti=true|false]&[distinct=true|false]"
chrome.exe "chrome-extension://fedhdngpddfjodmnfopokanlgnapnmeb/devtools/devtools_scraper_panel.html?run=test3&anti=false"
This is the ID of your Web Scraper Plus. Right click the icon and select option, then you can find it in your address bar.
The address will be like chrome-extension://fedhdngpddfjodmnfopokanlgnapnmeb/options_page/options.html
and the ExtensionID
is fedhdngpddfjodmnfopokanlgnapnmeb
This is the sitemap id of which the task you want to start.
Enable the Anti Lazy-Loading or not. Default value is true
.
Enable the Distinct or not. Default value is true
.
This feature is very useful if you want to controll the scraping by program.
For example, you can create some sitemaps by program and store it into MySQL just like you store anything else.
Indicate the parameters and construct a URL.
Call the chrome
application and open the URL you just created.