Releases: yujiosaka/headless-chrome-crawler
Releases · yujiosaka/headless-chrome-crawler
1.2.5
Added
- Support
obeyRobotsTxt
for crawler.queue()'s options. - Support
persist
for RedisCache's constructing options.
changed
- Make
cache
to be required for HCCrawler.connect() and HCCrawler.launch()'s options. - Provide
skipDuplicates
to remember and skip duplicate URLs, instead of passingnull
tocache
option. - Modify
BaseCache
interface.
1.2.4
Added
- Support CSV and JSON Lines formats for exporting results
- Emit
requeststarted
,requestskipped
,requestfinished
,requestfailed
,maxdepthreached
,maxrequestreached
anddisconnected
events. - Improve debug logs by tracing public APIs and events.
Changed
- Allow
onSuccess
andevaluatePage
options asnull
. - Change
crawler.isPaused
,crawler.queueSize
,crawler.pendingQueueSize
andcrawler.requestedCount
from read-only properties to methods.
Fixed
- Fix a bug of ignoring maxDepth option.
1.2.3
1.2.2
Added
- Automatically collect and follow links found in the requested page.
- Support
maxDepth
for crawler.queue()'s options.
1.2.1
Added
- Support
screenshot
for crawler.queue()'s options.
1.2.0
Changed
- Rename
ensureCacheClear
topersistCache
for HCCrawler.connect([options]) and HCCrawler.launch([options])'s options
1.1.2
Added
- Support
maxRequest
,allowedDomains
anduserAgent
option for crawler.queue([options]) - Support pluggable cache
- Add crawler.setMaxRequest(maxRequest), crawler.pause() and crawler.resume() methods
- Add crawler.pendingQueueSize and crawler.requestedCount read-only properties
1.1.1
1.1.0
- Refactor by separating HCCrawler and Crawler classes
- Public API to launch a browser has changed. Now you can launch browser by
HCCrawler.launch()
- Rename
shouldRequest
topreRequest
- Modify README according to new public API
- Modify examples according to new public API
- Support
extraHeaders
option - Refactor handlers for options
- Add comment in JSDoc style
1.0.0
- Add test with mocha and power-assert
- Add coverage with istanbul
- Add setting for CircleCI
- Add .editorconfig
- Migrate from NPM to Yarn
- Refactor helper to class static method style
- Add debug log