Skip to content

Commit

Permalink
Add Guess Type Option for Bulk Query
Browse files Browse the repository at this point in the history
Add the option to turn off the guessing of types in the bulk query

Closes #12
  • Loading branch information
StevenMMortimer committed Dec 17, 2018
1 parent bb6384d commit 7bd75f4
Show file tree
Hide file tree
Showing 50 changed files with 224 additions and 180 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ Suggests:
here,
RForcecom
VignetteBuilder: knitr
RoxygenNote: 6.0.1
RoxygenNote: 6.1.0
2 changes: 2 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,8 @@ importFrom(methods,as)
importFrom(purrr,map)
importFrom(purrr,map_df)
importFrom(purrr,map_dfc)
importFrom(readr,col_character)
importFrom(readr,col_guess)
importFrom(readr,cols)
importFrom(readr,read_csv)
importFrom(readr,type_convert)
Expand Down
1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

* Add **RForcecom** backward compatibile version of `rforcecom.getObjectDescription()`
* Add `sf_describe_object_fields()` which is a tidyier version of `rforcecom.getObjectDescription()`
* Allow users to control whether bulk query results are kept as all character or the types are guessed (#12)

### Bug Fixes

Expand Down
11 changes: 9 additions & 2 deletions R/bulk-query.R
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ sf_submit_query_bulk <- function(job_id, soql,
#' This function returns the row-level recordset of a bulk query
#' which has already been submitted to Bulk API Job and has Completed state
#'
#' @importFrom readr read_csv
#' @importFrom readr read_csv col_guess col_character
#' @importFrom httr content
#' @importFrom XML xmlToList
#' @importFrom dplyr as_tibble
Expand All @@ -85,6 +85,7 @@ sf_submit_query_bulk <- function(job_id, soql,
#' }
#' @export
sf_query_result_bulk <- function(job_id, batch_id, result_id,
guess_types = TRUE,
api_type = c("Bulk 1.0"),
verbose = FALSE){

Expand All @@ -101,7 +102,8 @@ sf_query_result_bulk <- function(job_id, batch_id, result_id,
if (grepl('xml', content_type)) {
res <- as_tibble(xmlToList(response_text))
} else if(grepl('text/csv', content_type)) {
res <- read_csv(response_text)
cols_default <- if(guess_types) col_guess() else col_character()
res <- read_csv(response_text, col_types = cols(.default=cols_default))
} else {
message(sprintf("Unhandled content-type: %s", content_type))
res <- content(httr_response, as="parsed", encoding="UTF-8")
Expand All @@ -116,6 +118,9 @@ sf_query_result_bulk <- function(job_id, batch_id, result_id,
#'
#' @template soql
#' @template object_name
#' @param guess_types logical; indicating whether or not to use \code{col_guess()}
#' to try and cast the data returned in the query recordset. TRUE uses \code{col_guess()}
#' and FALSE returns all values as character strings.
#' @template api_type
#' @param interval_seconds integer; defines the seconds between attempts to check
#' for job completion
Expand All @@ -132,6 +137,7 @@ sf_query_result_bulk <- function(job_id, batch_id, result_id,
#' @export
sf_query_bulk <- function(soql,
object_name,
guess_types=TRUE,
api_type = c("Bulk 1.0"),
interval_seconds=5,
max_attempts=100,
Expand Down Expand Up @@ -182,6 +188,7 @@ sf_query_bulk <- function(soql,
res <- sf_query_result_bulk(job_id = batch_query_info$jobId,
batch_id = batch_query_info$id,
result_id = batch_query_details$result,
guess_types = guess_types,
api_type = "Bulk 1.0",
verbose = verbose)
close_job_info <- sf_close_job_bulk(job_info$id, api_type = api_type,
Expand Down
26 changes: 13 additions & 13 deletions docs/articles/getting-started.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 21 additions & 21 deletions docs/articles/transitioning-from-RForcecom.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

26 changes: 13 additions & 13 deletions docs/articles/working-with-bulk-api.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 7bd75f4

Please sign in to comment.