-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added Search feature to Youtube Scraper #194
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you very much for your contribution! Really looking forward to this feature.
One little thing - current structure prefers common fetchPosts method with bunch of extensions for provider-specific scrapers, which incapsulates path and query params building. From my perspective it is quite relative case.
Regarding issue with Int restrictions - I haven't met yet cases when size exceeds integer boundaries, but if it is - it'a not a big deal to increase it up to long boundaries, sure. It possibly would break compatibility, but anyway this tool definitely will never have major first version due to specifics of scraping and changing document structure of data providers without any notice.
Love to contribute in such a good library and clean code! |
Yes, I got your point. fun YoutubeSkraper.getSearchPosts(keyword: String): Flow<Post> {
return getPosts(path = "/results?search_query=${keyword.replace(" ","+")}")
} And your function logic should be inside |
Thank you for elaboration, It's much cleaner now. |
Thank you, that's exactly what I meant! |
Added Search feature with keyword with a corresponding test using
assertPosts
.Also fixed an issue where video's views would be in this format
150,100,100
format (with commas).One problem we need to address that the
views
property inPostStatistics
isInt?
where there are videos that exceeds that limit, didn't want to change it as it's tightly coupled with the rest of the scrapers.