Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 2.0: chunk_size parameter broken in read_delim_chunked function #1248

Closed
timothy-barry opened this issue Jul 27, 2021 · 2 comments
Closed

Comments

@timothy-barry
Copy link
Contributor

timothy-barry commented Jul 27, 2021

Hi,

I just updated to version 2.0. The chunk_size parameter of the read_delim_chunked function does not seem to work. Regardless of the argument that I pass, the chunk size remains fixed at 10000.

Thanks.

EDIT: Here is a reproducible example. Chunk size argument does not work when reading mtcars.csv:

library(readr)
f <- function(x, pos) print(nrow(x))
read_delim_chunked(file = readr_example("mtcars.csv"),
                   delim = ",",
                   callback = SideEffectChunkCallback$new(f),
                   chunk_size = 5)

The output is "32," indicating that all 32 rows were read at once.

@svraka
Copy link

svraka commented Jul 27, 2021

Ran into the same problem, it looks like chunk_size is not passed to the read_delimited_chunked() call in read_delim_chunked().

read_delimited_chunked(file,
callback = callback, tokenizer = tokenizer, col_names = col_names, col_types = col_types,
locale = locale, skip = skip, skip_empty_rows = skip_empty_rows,
comment = comment, guess_max = guess_max, progress = progress
)

@jimhester
Copy link
Collaborator

Thanks for opening the issue and providing a reproducible example.
This was a regression in readr 2.0.0, it should be fixed now in the current master branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants