-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Combine error from vec_c()
requiring columns to be same type
#337
Comments
It makes sense that this returns an error (type stability is a core principle of {vctrs}) and to me this is more a feature than a bug. Otherwise you won't know whether So, this should clearly be handled by the function you use to parse the response. Out of curiosity: do you have an example where this happened? |
Unfortunately I can't make a reprex. The first page happened to have only scalars under column X. Another page happened to have said column containing scalars and vectors. The same thing happened for another column Y when I was pulling different dates (hence different pages). In our case we don't have a data dictionary outlining types. However, I'm not convinced that types (i.e. having a complete data dictionary of endpoint columns) should be imposed before pulling from an API. Our API is in development such that columns are added depending on users' needs. To account for that in R functions would add unnecessary complexity. I recall @hadley suggested a paginated perform could return a list so that it is on the user on how to handle it. Indeed that's the route I took by using |
You don't need to use ``` r
devtools::load_all("~/GitHub/httr2/")
#> ℹ Loading httr2
page_size <- 40
parse_flower_resp <- function(resp) {
parsed <- resp_body_json(resp)
results <- parsed$data
data <- data.frame(
Sepal.Length = sapply(results, `[[`, "Sepal.Length"),
Sepal.Width = sapply(results, `[[`, "Sepal.Width"),
Petal.Length = sapply(results, `[[`, "Petal.Length"),
Petal.Width = sapply(results, `[[`, "Petal.Width"),
Species = sapply(results, `[[`, "Species")
)
# list(data = data, count = parsed$count)
# wrap data in a list
list(data = list(data), count = parsed$count)
}
req_flowers <- request(example_url()) %>%
req_url_path("/iris") %>%
req_url_query(limit = page_size) %>%
req_paginate_page_index(
page_index = function(req, page) {
req %>% req_url_query(page_index = page)
},
parse_resp = parse_flower_resp,
n_pages = function(parsed) {
total <- parsed$count
ceiling(total / page_size)
}
)
paginate_req_perform(req_flowers)
#> [[1]]
#> Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1 5.1 3.5 1.4 0.2 setosa
#> 2 4.9 3.0 1.4 0.2 setosa
#> 3 4.7 3.2 1.3 0.2 setosa
#> 4 4.6 3.1 1.5 0.2 setosa
#> 5 5.0 3.6 1.4 0.2 setosa
#> 6 5.4 3.9 1.7 0.4 setosa
#> 7 4.6 3.4 1.4 0.3 setosa
#> 8 5.0 3.4 1.5 0.2 setosa
#> 9 4.4 2.9 1.4 0.2 setosa
#> 10 4.9 3.1 1.5 0.1 setosa
#> 11 5.4 3.7 1.5 0.2 setosa
#> 12 4.8 3.4 1.6 0.2 setosa
#> 13 4.8 3.0 1.4 0.1 setosa
#> 14 4.3 3.0 1.1 0.1 setosa
#> 15 5.8 4.0 1.2 0.2 setosa
#> 16 5.7 4.4 1.5 0.4 setosa
#> 17 5.4 3.9 1.3 0.4 setosa
#> 18 5.1 3.5 1.4 0.3 setosa
#> 19 5.7 3.8 1.7 0.3 setosa
#> 20 5.1 3.8 1.5 0.3 setosa
#> 21 5.4 3.4 1.7 0.2 setosa
#> 22 5.1 3.7 1.5 0.4 setosa
#> 23 4.6 3.6 1.0 0.2 setosa
#> 24 5.1 3.3 1.7 0.5 setosa
#> 25 4.8 3.4 1.9 0.2 setosa
#> 26 5.0 3.0 1.6 0.2 setosa
#> 27 5.0 3.4 1.6 0.4 setosa
#> 28 5.2 3.5 1.5 0.2 setosa
#> 29 5.2 3.4 1.4 0.2 setosa
#> 30 4.7 3.2 1.6 0.2 setosa
#> 31 4.8 3.1 1.6 0.2 setosa
#> 32 5.4 3.4 1.5 0.4 setosa
#> 33 5.2 4.1 1.5 0.1 setosa
#> 34 5.5 4.2 1.4 0.2 setosa
#> 35 4.9 3.1 1.5 0.2 setosa
#> 36 5.0 3.2 1.2 0.2 setosa
#> 37 5.5 3.5 1.3 0.2 setosa
#> 38 4.9 3.6 1.4 0.1 setosa
#> 39 4.4 3.0 1.3 0.2 setosa
#> 40 5.1 3.4 1.5 0.2 setosa
#>
#> [[2]]
#> Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1 5.0 3.5 1.3 0.3 setosa
#> 2 4.5 2.3 1.3 0.3 setosa
#> 3 4.4 3.2 1.3 0.2 setosa
#> 4 5.0 3.5 1.6 0.6 setosa
#> 5 5.1 3.8 1.9 0.4 setosa
#> 6 4.8 3.0 1.4 0.3 setosa
#> 7 5.1 3.8 1.6 0.2 setosa
#> 8 4.6 3.2 1.4 0.2 setosa
#> 9 5.3 3.7 1.5 0.2 setosa
#> 10 5.0 3.3 1.4 0.2 setosa
#> 11 7.0 3.2 4.7 1.4 versicolor
#> 12 6.4 3.2 4.5 1.5 versicolor
#> 13 6.9 3.1 4.9 1.5 versicolor
#> 14 5.5 2.3 4.0 1.3 versicolor
#> 15 6.5 2.8 4.6 1.5 versicolor
#> 16 5.7 2.8 4.5 1.3 versicolor
#> 17 6.3 3.3 4.7 1.6 versicolor
#> 18 4.9 2.4 3.3 1.0 versicolor
#> 19 6.6 2.9 4.6 1.3 versicolor
#> 20 5.2 2.7 3.9 1.4 versicolor
#> 21 5.0 2.0 3.5 1.0 versicolor
#> 22 5.9 3.0 4.2 1.5 versicolor
#> 23 6.0 2.2 4.0 1.0 versicolor
#> 24 6.1 2.9 4.7 1.4 versicolor
#> 25 5.6 2.9 3.6 1.3 versicolor
#> 26 6.7 3.1 4.4 1.4 versicolor
#> 27 5.6 3.0 4.5 1.5 versicolor
#> 28 5.8 2.7 4.1 1.0 versicolor
#> 29 6.2 2.2 4.5 1.5 versicolor
#> 30 5.6 2.5 3.9 1.1 versicolor
#> 31 5.9 3.2 4.8 1.8 versicolor
#> 32 6.1 2.8 4.0 1.3 versicolor
#> 33 6.3 2.5 4.9 1.5 versicolor
#> 34 6.1 2.8 4.7 1.2 versicolor
#> 35 6.4 2.9 4.3 1.3 versicolor
#> 36 6.6 3.0 4.4 1.4 versicolor
#> 37 6.8 2.8 4.8 1.4 versicolor
#> 38 6.7 3.0 5.0 1.7 versicolor
#> 39 6.0 2.9 4.5 1.5 versicolor
#> 40 5.7 2.6 3.5 1.0 versicolor
#>
#> [[3]]
#> Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1 5.5 2.4 3.8 1.1 versicolor
#> 2 5.5 2.4 3.7 1.0 versicolor
#> 3 5.8 2.7 3.9 1.2 versicolor
#> 4 6.0 2.7 5.1 1.6 versicolor
#> 5 5.4 3.0 4.5 1.5 versicolor
#> 6 6.0 3.4 4.5 1.6 versicolor
#> 7 6.7 3.1 4.7 1.5 versicolor
#> 8 6.3 2.3 4.4 1.3 versicolor
#> 9 5.6 3.0 4.1 1.3 versicolor
#> 10 5.5 2.5 4.0 1.3 versicolor
#> 11 5.5 2.6 4.4 1.2 versicolor
#> 12 6.1 3.0 4.6 1.4 versicolor
#> 13 5.8 2.6 4.0 1.2 versicolor
#> 14 5.0 2.3 3.3 1.0 versicolor
#> 15 5.6 2.7 4.2 1.3 versicolor
#> 16 5.7 3.0 4.2 1.2 versicolor
#> 17 5.7 2.9 4.2 1.3 versicolor
#> 18 6.2 2.9 4.3 1.3 versicolor
#> 19 5.1 2.5 3.0 1.1 versicolor
#> 20 5.7 2.8 4.1 1.3 versicolor
#> 21 6.3 3.3 6.0 2.5 virginica
#> 22 5.8 2.7 5.1 1.9 virginica
#> 23 7.1 3.0 5.9 2.1 virginica
#> 24 6.3 2.9 5.6 1.8 virginica
#> 25 6.5 3.0 5.8 2.2 virginica
#> 26 7.6 3.0 6.6 2.1 virginica
#> 27 4.9 2.5 4.5 1.7 virginica
#> 28 7.3 2.9 6.3 1.8 virginica
#> 29 6.7 2.5 5.8 1.8 virginica
#> 30 7.2 3.6 6.1 2.5 virginica
#> 31 6.5 3.2 5.1 2.0 virginica
#> 32 6.4 2.7 5.3 1.9 virginica
#> 33 6.8 3.0 5.5 2.1 virginica
#> 34 5.7 2.5 5.0 2.0 virginica
#> 35 5.8 2.8 5.1 2.4 virginica
#> 36 6.4 3.2 5.3 2.3 virginica
#> 37 6.5 3.0 5.5 1.8 virginica
#> 38 7.7 3.8 6.7 2.2 virginica
#> 39 7.7 2.6 6.9 2.3 virginica
#> 40 6.0 2.2 5.0 1.5 virginica
#>
#> [[4]]
#> Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1 6.9 3.2 5.7 2.3 virginica
#> 2 5.6 2.8 4.9 2.0 virginica
#> 3 7.7 2.8 6.7 2.0 virginica
#> 4 6.3 2.7 4.9 1.8 virginica
#> 5 6.7 3.3 5.7 2.1 virginica
#> 6 7.2 3.2 6.0 1.8 virginica
#> 7 6.2 2.8 4.8 1.8 virginica
#> 8 6.1 3.0 4.9 1.8 virginica
#> 9 6.4 2.8 5.6 2.1 virginica
#> 10 7.2 3.0 5.8 1.6 virginica
#> 11 7.4 2.8 6.1 1.9 virginica
#> 12 7.9 3.8 6.4 2.0 virginica
#> 13 6.4 2.8 5.6 2.2 virginica
#> 14 6.3 2.8 5.1 1.5 virginica
#> 15 6.1 2.6 5.6 1.4 virginica
#> 16 7.7 3.0 6.1 2.3 virginica
#> 17 6.3 3.4 5.6 2.4 virginica
#> 18 6.4 3.1 5.5 1.8 virginica
#> 19 6.0 3.0 4.8 1.8 virginica
#> 20 6.9 3.1 5.4 2.1 virginica
#> 21 6.7 3.1 5.6 2.4 virginica
#> 22 6.9 3.1 5.1 2.3 virginica
#> 23 5.8 2.7 5.1 1.9 virginica
#> 24 6.8 3.2 5.9 2.3 virginica
#> 25 6.7 3.3 5.7 2.5 virginica
#> 26 6.7 3.0 5.2 2.3 virginica
#> 27 6.3 2.5 5.0 1.9 virginica
#> 28 6.5 3.0 5.2 2.0 virginica
#> 29 6.2 3.4 5.4 2.3 virginica
#> 30 5.9 3.0 5.1 1.8 virginica Created on 2023-10-11 with reprex v2.0.2
and parse it in the end. Does that help you? |
Thank you! Wrapping data in a list is brilliant. Being unfamiliar with |
When using pagination, an error is returned from
paginate_req_perform()
if columns of the same name have different types across pages.The text was updated successfully, but these errors were encountered: