Crawl sync api fix

This release brings fixing the thread handling of async task with the client that is established on the main thread.
Crawl speed is improved drastically due to the incorrect handling between the client pool releasing between threads.

What's Changed

• chore(log): add log util by @j-mendez in #31
• feat(regex): add optional regex black listing by @j-mendez in #36
• perf(crawl): improve crawl link exclusion by @j-mendez in #37
• perf(parsing): add async parallel page handling by @j-mendez in #38
• perf(client): fix blocking and async mixture by @j-mendez in #39

Full Changelog: v1.5.1...v1.6.0

v1.5.1

What's Changed

• chore(ua): add cargo env ua defaults by @j-mendez in #27
• chore(links): fix parsing valid website pages by @j-mendez in #30

Full Changelog: v1.5.0...v1.5.1

v1.4.0

• feat(cli): add cli ability (#16 thanks to @j-mendez)
• feat(concurrency): dynamic concurrent cpu defaults (#15 thanks to @j-mendez)
• docs: add a changelog

v1.3.1

• fix(crawl): fix field type (#14 thanks to @j-mendez)

v1.3.0

• feat(crawl): callback to run when link is found (#13 thanks to @j-mendez)

v1.2.0

• Add User Agent configuration (#5 thanks to @Dragnucs)
• Add polite delay (#6 thanks to @Dragnucs )

v1.1.3

• Handle page get errors (#4 thanks to @Dragnucs)
• Fix link resolution (#3 thanks to @Dragnucs)

v1.1.2

Update dependencies

v1.1.1

Correct tag

v1.0.3

Merge branch 'master' of https://github.com/madeindjs/spider