ARROW-14653: [R] head() hangs on CSV datasets > 600MB

r/NEWS.md

@@ -26,6 +26,7 @@
 * Added `decimal128()` (identical to `decimal()`) as the name is more explicit and updated docs to encourage its use. 
 * Source builds now by default use `pkg-config` to search for system dependencies (such as `libz`) and link to them 
 if present. To retain the previous behaviour of downloading and building all dependencies, set `ARROW_DEPENDENCY_SOURCE=BUNDLED`. 
+* Opening datasets now use async scanner by default which resolves a deadlock issues related to reading in large multi-CSV datasets
 
 # arrow 6.0.1
 

r/R/dataset-scan.R

@@ -34,7 +34,7 @@
 #'    to keep all rows.
 #' * `use_threads`: logical: should scanning use multithreading? Default `TRUE`
 #' * `use_async`: logical: should the async scanner (performs better on
-#'    high-latency/highly parallel filesystems like S3) be used? Default `FALSE`
+#'    high-latency/highly parallel filesystems like S3) be used? Default `TRUE`
 #' * `...`: Additional arguments, currently ignored
 #' @section Methods:
 #' `ScannerBuilder` has the following methods:
@@ -73,7 +73,7 @@ Scanner$create <- function(dataset,
                            projection = NULL,
                            filter = TRUE,
                            use_threads = option_use_threads(),
-                           use_async = getOption("arrow.use_async", FALSE),
+                           use_async = getOption("arrow.use_async", TRUE),
                            batch_size = NULL,
                            fragment_scan_options = NULL,
                            ...) {