Wants to extract the html of the whole document but kept getting no node found

[jslearn81]

Hi there,

Thanks for creating the package. I think it is super awesome and will be useful for webscrapping.
I kept getting this error:

'Error in onRejected(reason) : code: -32000
message: No node found for given backend id'

My sample code is:

`
library(chromote)

b <- ChromoteSession$new()

b$view()

b$Page$navigate("https://sc.macromicro.me/charts/449/us-cboe-options-put-call-ratio",wait_ = TRUE) #as they block CORS

Sys.sleep(3)

b$Page$navigate("https://sc.macromicro.me/charts/data/449",wait_ = TRUE)

data <- b$DOM$getDocument()

data_content <- b$DOM$getOuterHTML(data$root$nodeId)

json_data <- stringr::str_extract_all(data_content,'\{(.*|\n)?\}')[[1]]

json_data <- fromJSON(json_data)

b$close()
`

Any tips to get this working ? Or is there a better command to extract the content of the page ?

Regards
J

[jslearn81]

Hi did i get anyone's attention so far :( ?

[RLesur]

Hi @quantumfusionlearn ,

You can try something like that:

json_data <- jsonlite::fromJSON(xml2::xml_text(xml2::read_html(data_content$outerHTML)))

[hadley]

I'm about to add support live scraping via chromote to rvest: tidyverse/rvest#362. That will be more user facing (rather than developer) and I'll be actively looking for scraping challenges.