Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How I can pass a HTML string to LinkThumbnailer #138

Open
vedmant opened this issue Jun 28, 2019 · 8 comments
Open

How I can pass a HTML string to LinkThumbnailer #138

vedmant opened this issue Jun 28, 2019 · 8 comments
Labels
enhancement question

Comments

@vedmant
Copy link

vedmant commented Jun 28, 2019

How I can pass a HTML string to LinkThumbnailer to make it parse it instead of opening URL?
Couldn't find anything about it in the documentation or issues.
@gottfrois
Copy link
Owner

There is no public method available for this use case. However, you can try calling the following directly code directly:

https://github.com/gottfrois/link_thumbnailer/blob/master/lib/link_thumbnailer/scraper.rb#L25

source = "<html>...</html>"
scraper = LinkThumbnailer::Scraper.new(source, "http://fake.come")
scraper.call

Note that you will need to pass a URL anyway because LinkThumbnailer expects one, but it's only to prefill the URL attribute of the response object.

@vedmant
Copy link
Author

vedmant commented Jul 1, 2019 •
edited
Loading

@gottfrois Thanks, I'll try this. Would be nice to have public method for this, in my case I need to scrap html for other data also and currently I didn't find other option but to make two requests one from LinkThumbnailer another to get html content for my needs.

@vedmant
Copy link
Author

vedmant commented Jul 2, 2019 •
edited
Loading

@gottfrois I tried your suggestion but have an error: undefined method 'host' for #String:0x00007f9fcd043e80 on line scraper.call

@gottfrois
Copy link
Owner

Ah, the URL should be a URI object i think, not a string https://docs.ruby-lang.org/en/2.1.0/URI.html

@vedmant
Copy link
Author

vedmant commented Jul 22, 2019 •
edited
Loading

@gottfrois Thanks, that worked. However I have another issue now, I moved the code into a service class, and it stopped working, I have following error undefined method 'config' for nil:NilClass from /usr/local/bundle/gems/link_thumbnailer-3.3.2/lib/link_thumbnailer/scraper.rb:28:in 'initialize'
There is a line in gem: @config = ::LinkThumbnailer.page.config and looks like LinkThumbnailer.page is nil.

@gottfrois
Copy link
Owner

Yeah this code is not the best... :( https://github.com/gottfrois/link_thumbnailer/blob/master/lib/link_thumbnailer/scraper.rb#L28

Don't remember why i proxied config through the Page class but it should use LinkThumbnailer.config directly IMO.

I don't see a quick fix for this other than monkey patching or changing the code of the gem and make it easier to scrap html directly.

@vedmant
Copy link
Author

vedmant commented Mar 18, 2021

Are there plans for new release with direct html parsing support? I'd use that.

@gottfrois
Copy link
Owner

No plans really but I'd gladly accept a PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gottfrois @vedmant