Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmark with other scraper/crawling tools #1

Open
entrptaher opened this issue Jan 1, 2021 · 1 comment
Open

Benchmark with other scraper/crawling tools #1

entrptaher opened this issue Jan 1, 2021 · 1 comment

Comments

@entrptaher
Copy link

How would this perform in a benchmark? Any benchmark in the roadmap?
@mattsse
Copy link
Owner

mattsse commented Jan 17, 2021

Hi,
Currently there no plans for a benchmark but I'm interested what other frameworks you would like to voyager being benchmarked against? I know there is gocolly/colly, anything else?
Also I'm not sure how to implement this in a reproducible manner. Most likely this would require setting up a local http server that serves some html pages which the crawler benchmark would use as target?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@entrptaher @mattsse