Skip to content

Commit

Permalink
Update embedded curl
Browse files Browse the repository at this point in the history
  • Loading branch information
gaborcsardi committed Nov 15, 2024
1 parent 397a525 commit 1518384
Show file tree
Hide file tree
Showing 46 changed files with 1,090 additions and 464 deletions.
39 changes: 17 additions & 22 deletions src/library/curl/DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,39 +1,34 @@
Package: curl
Type: Package
Title: A Modern and Flexible Web Client for R
Version: 5.2.1
Version: 6.0.1
Authors@R: c(
person("Jeroen", "Ooms", role = c("aut", "cre"), email = "jeroen@berkeley.edu",
person("Jeroen", "Ooms", role = c("aut", "cre"), email = "jeroenooms@gmail.com",
comment = c(ORCID = "0000-0002-4035-0289")),
person("Hadley", "Wickham", , "hadley@rstudio.com", role = "ctb"),
person("RStudio", role = "cph")
)
Description: The curl() and curl_download() functions provide highly
configurable drop-in replacements for base url() and download.file() with
better performance, support for encryption (https, ftps), gzip compression,
authentication, and other 'libcurl' goodies. The core of the package implements a
framework for performing fully customized requests where data can be processed
either in memory, on disk, or streaming via the callback or connection
interfaces. Some knowledge of 'libcurl' is recommended; for a more-user-friendly
web client see the 'httr' package which builds on this package with http
specific tools and logic.
person("Hadley", "Wickham", role = "ctb"),
person("Posit Software, PBC", role = "cph"))
Description: Bindings to 'libcurl' <https://curl.se/libcurl/> for performing fully
configurable HTTP/FTP requests where responses can be processed in memory, on
disk, or streaming via the callback or connection interfaces. Some knowledge
of 'libcurl' is recommended; for a more-user-friendly web client see the
'httr2' package which builds on this package with http specific tools and logic.
License: MIT + file LICENSE
SystemRequirements: libcurl: libcurl-devel (rpm) or
libcurl4-openssl-dev (deb).
URL: https://jeroen.r-universe.dev/curl https://curl.se/libcurl/
SystemRequirements: libcurl (>= 7.62): libcurl-devel (rpm) or
libcurl4-openssl-dev (deb)
URL: https://jeroen.r-universe.dev/curl
BugReports: https://github.com/jeroen/curl/issues
Suggests: spelling, testthat (>= 1.0.0), knitr, jsonlite, later,
rmarkdown, httpuv (>= 1.4.4), webutils
VignetteBuilder: knitr
Depends: R (>= 3.0.0)
RoxygenNote: 7.3.0
RoxygenNote: 7.3.2.9000
Encoding: UTF-8
Language: en-US
NeedsCompilation: yes
Packaged: 2024-02-26 21:12:31 UTC; jeroen
Packaged: 2024-11-13 13:14:47 UTC; jeroen
Author: Jeroen Ooms [aut, cre] (<https://orcid.org/0000-0002-4035-0289>),
Hadley Wickham [ctb],
RStudio [cph]
Maintainer: Jeroen Ooms <jeroen@berkeley.edu>
Posit Software, PBC [cph]
Maintainer: Jeroen Ooms <jeroenooms@gmail.com>
Repository: CRAN
Date/Publication: 2024-03-01 23:22:46 UTC
Date/Publication: 2024-11-14 09:20:02 UTC
4 changes: 2 additions & 2 deletions src/library/curl/LICENSE
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
YEAR: 2022
COPYRIGHT HOLDER: Jeroen Ooms; RStudio
YEAR: 2024
COPYRIGHT HOLDER: Jeroen Ooms; Posit Software, PBC
2 changes: 2 additions & 0 deletions src/library/curl/NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export(curl_fetch_memory)
export(curl_fetch_multi)
export(curl_fetch_stream)
export(curl_options)
export(curl_parse_url)
export(curl_symbols)
export(curl_unescape)
export(curl_upload)
Expand Down Expand Up @@ -76,6 +77,7 @@ useDynLib(curl,R_new_file_writer)
useDynLib(curl,R_new_handle)
useDynLib(curl,R_nslookup)
useDynLib(curl,R_option_types)
useDynLib(curl,R_parse_url)
useDynLib(curl,R_proxy_info)
useDynLib(curl,R_split_string)
useDynLib(curl,R_total_handles)
Expand Down
25 changes: 25 additions & 0 deletions src/library/curl/NEWS
Original file line number Diff line number Diff line change
@@ -1,3 +1,28 @@
6.0.1
- Fix a build issue with libcurl 8.11.0
- Support multi_fdset() for connection objects (#355)

6.0.0
- New curl_parse_url() function to expose curl URL parsing interface.
- Errors now have classes and more detailed messages
- Do not use shared connection pool for FTP requests (works around bug in
libcurl < 8.8.0)
- Properly clean handle when open() fails for a curl() connection
- Parameter 'timeout' renamed to 'multi-timeout' in multi_download()
- Default behavior is now to error if a download is stalled for 600 seconds
- The MacOS binary packages now require at least libcurl 7.80 which is included
with all current MacOS versions (13+) as well as recent patch releases for
legacy MacOS versions (11.7 and 12.7).
- Windows: update to libcurl 8.10.1

5.2.3
- Remove some CMD check verbosity per new CRAN rules
- New maintainer email address

5.2.2
- Disable a compression test for libcurl 8.7.1 which has a bug for deflate
- Change a unit test to work around firewall problems on CRANs WinBuilder

5.2.1
- In handle_setheader(), setting a header to a non-empty whitespace string will
now send a blank header. Using an empty string will still remove the header
Expand Down
12 changes: 6 additions & 6 deletions src/library/curl/R/curl.R
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#' Curl connection interface
#'
#' Drop-in replacement for base \code{\link{url}} that supports https, ftps,
#' gzip, deflate, etc. Default behavior is identical to \code{\link{url}}, but
#' request can be fully configured by passing a custom \code{\link{handle}}.
#' Drop-in replacement for base [url()] that supports https, ftps,
#' gzip, deflate, etc. Default behavior is identical to [url()], but
#' request can be fully configured by passing a custom [handle()].
#'
#' As of version 2.3 curl connections support \code{open(con, blocking = FALSE)}.
#' In this case \code{readBin} and \code{readLines} will return immediately with data
#' As of version 2.3 curl connections support `open(con, blocking = FALSE)`.
#' In this case `readBin` and `readLines` will return immediately with data
#' that is available without waiting. For such non-blocking connections the caller
#' needs to call \code{\link{isIncomplete}} to check if the download has completed
#' needs to call [isIncomplete()] to check if the download has completed
#' yet.
#'
#' @useDynLib curl R_curl_connection
Expand Down
20 changes: 11 additions & 9 deletions src/library/curl/R/download.R
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
#' Download file to disk
#'
#' Libcurl implementation of \code{C_download} (the "internal" download method)
#' Libcurl implementation of `C_download` (the "internal" download method)
#' with added support for https, ftps, gzip, etc. Default behavior is identical
#' to \code{\link{download.file}}, but request can be fully configured by passing
#' a custom \code{\link{handle}}.
#' to [download.file()], but request can be fully configured by passing
#' a custom [handle()].
#'
#' The main difference between \code{curl_download} and \code{curl_fetch_disk}
#' is that \code{curl_download} checks the http status code before starting the
#' The main difference between `curl_download` and `curl_fetch_disk`
#' is that `curl_download` checks the http status code before starting the
#' download, and raises an error when status is non-successful. The behavior of
#' \code{curl_fetch_disk} on the other hand is to proceed as normal and write
#' `curl_fetch_disk` on the other hand is to proceed as normal and write
#' the error page to disk in case of a non success response.
#'
#' The `curl_download` function does support resuming and removes the temporary
#' file if the download did not complete successfully.
#' For a more advanced download interface which supports concurrent requests and
#' resuming large files, have a look at the [multi_download] function.
#'
Expand All @@ -19,11 +21,11 @@
#' @param url A character string naming the URL of a resource to be downloaded.
#' @param destfile A character string with the name where the downloaded file
#' is saved. Tilde-expansion is performed.
#' @param quiet If \code{TRUE}, suppress status messages (if any), and the
#' @param quiet If `TRUE`, suppress status messages (if any), and the
#' progress bar.
#' @param mode A character string specifying the mode with which to write the file.
#' Useful values are \code{"w"}, \code{"wb"} (binary), \code{"a"} (append)
#' and \code{"ab"}.
#' Useful values are `"w"`, `"wb"` (binary), `"a"` (append)
#' and `"ab"`.
#' @param handle a curl handle object
#' @return Path of downloaded file (invisibly).
#' @export
Expand Down
48 changes: 24 additions & 24 deletions src/library/curl/R/email.R
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
#' Send email
#'
#' Use the curl SMTP client to send an email. The \code{message} argument must be
#' properly formatted \href{https://www.rfc-editor.org/rfc/rfc2822}{RFC2822} email message with From/To/Subject headers and CRLF
#' Use the curl SMTP client to send an email. The `message` argument must be
#' properly formatted [RFC2822](https://www.rfc-editor.org/rfc/rfc2822) email message with From/To/Subject headers and CRLF
#' line breaks.
#'
#' @section Specifying the server, port, and protocol:
#'
#' The \code{smtp_server} argument takes a hostname, or an SMTP URL:
#' The `smtp_server` argument takes a hostname, or an SMTP URL:
#'
#' \itemize{
#' \item \code{mail.example.com} - hostname only
#' \item \code{mail.example.com:587} - hostname and port
#' \item \code{smtp://mail.example.com} - protocol and hostname
#' \item \code{smtp://mail.example.com:587} - full SMTP URL
#' \item \code{smtps://mail.example.com:465} - full SMTPS URL
#' \item `mail.example.com` - hostname only
#' \item `mail.example.com:587` - hostname and port
#' \item `smtp://mail.example.com` - protocol and hostname
#' \item `smtp://mail.example.com:587` - full SMTP URL
#' \item `smtps://mail.example.com:465` - full SMTPS URL
#' }
#'
#' By default, the port will be 25, unless \code{smtps://} is specified--then
#' By default, the port will be 25, unless `smtps://` is specified--then
#' the default will be 465 instead.
#'
#' For internet SMTP servers you probably need to pass a
#' \href{https://curl.se/libcurl/c/CURLOPT_USERNAME.html}{username} and
#' \href{https://curl.se/libcurl/c/CURLOPT_PASSWORD.html}{passwords} option.
#' [username](https://curl.se/libcurl/c/CURLOPT_USERNAME.html) and
#' [passwords](https://curl.se/libcurl/c/CURLOPT_PASSWORD.html) option.
#' For some servers you also need to pass a string with
#' \href{https://curl.se/libcurl/c/CURLOPT_LOGIN_OPTIONS.html}{login_options}
#' for example \code{login_options="AUTH=NTLM"}.
#' [login_options](https://curl.se/libcurl/c/CURLOPT_LOGIN_OPTIONS.html)
#' for example `login_options="AUTH=NTLM"`.
#'
#' @section Encrypting connections via SMTPS or STARTTLS:
#'
Expand All @@ -34,31 +34,31 @@
#' secure TLS connection using the STARTTLS command. It is important to know
#' which method your server expects.
#'
#' If your smtp server listens on port 465, then use a \code{smtps://hostname:465}
#' URL. The SMTPS protocol \emph{guarantees} that TLS will be used to protect
#' If your smtp server listens on port 465, then use a `smtps://hostname:465`
#' URL. The SMTPS protocol *guarantees* that TLS will be used to protect
#' all communications from the start.
#'
#' If your email server listens on port 25 or 587, use an \code{smtp://} URL in
#' combination with the \code{use_ssl} parameter to control if the connection
#' should be upgraded with STARTTLS. The default value \code{"try"} will
#' \emph{opportunistically} try to upgrade to a secure connection if the server
#' If your email server listens on port 25 or 587, use an `smtp://` URL in
#' combination with the `use_ssl` parameter to control if the connection
#' should be upgraded with STARTTLS. The default value `"try"` will
#' *opportunistically* try to upgrade to a secure connection if the server
#' supports it, and proceed as normal otherwise.
#'
#' @export
#' @param mail_rcpt one or more recipient email addresses. Do not include names,
#' these go into the \code{message} headers.
#' these go into the `message` headers.
#' @param mail_from email address of the sender.
#' @param message either a string or connection with (properly formatted) email
#' message, including sender/recipient/subject headers. See example.
#' @param smtp_server hostname or address of the SMTP server, or, an
#' \code{smtp://} or \code{smtps://} URL. See "Specifying the server, port,
#' `smtp://` or `smtps://` URL. See "Specifying the server, port,
#' and protocol" below.
#' @param use_ssl Request to upgrade the connection to SSL using the STARTTLS command,
#' see \href{https://curl.se/libcurl/c/CURLOPT_USE_SSL.html}{CURLOPT_USE_SSL}
#' see [CURLOPT_USE_SSL](https://curl.se/libcurl/c/CURLOPT_USE_SSL.html)
#' for details. Default will try to SSL, proceed as normal otherwise.
#' @param verbose print output
#' @param ... other options passed to \code{\link{handle_setopt}}. In most cases
#' you will need to set a \code{username} and \code{password} or \code{login_options}
#' @param ... other options passed to [handle_setopt()]. In most cases
#' you will need to set a `username` and `password` or `login_options`
#' to authenticate with the SMTP server, see details.
#' @examples \dontrun{# Set sender and recipients (email addresses only)
#' recipients <- readline("Enter your email address to receive test: ")
Expand Down
42 changes: 42 additions & 0 deletions src/library/curl/R/errcodes.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# This file is autogenerated using make-errorcodes.R
libcurl_error_codes <-
c("curl_error_unsupported_protocol", "curl_error_failed_init",
"curl_error_url_malformat", "curl_error_not_built_in", "curl_error_couldnt_resolve_proxy",
"curl_error_couldnt_resolve_host", "curl_error_couldnt_connect",
"curl_error_weird_server_reply", "curl_error_remote_access_denied",
"curl_error_ftp_accept_failed", "curl_error_ftp_weird_pass_reply",
"curl_error_ftp_accept_timeout", "curl_error_ftp_weird_pasv_reply",
"curl_error_ftp_weird_227_format", "curl_error_ftp_cant_get_host",
"curl_error_http2", "curl_error_ftp_couldnt_set_type", "curl_error_partial_file",
"curl_error_ftp_couldnt_retr_file", "curl_error_obsolete20",
"curl_error_quote_error", "curl_error_http_returned_error", "curl_error_write_error",
"curl_error_obsolete24", "curl_error_upload_failed", "curl_error_read_error",
"curl_error_out_of_memory", "curl_error_operation_timedout",
"curl_error_obsolete29", "curl_error_ftp_port_failed", "curl_error_ftp_couldnt_use_rest",
"curl_error_obsolete32", "curl_error_range_error", "curl_error_http_post_error",
"curl_error_ssl_connect_error", "curl_error_bad_download_resume",
"curl_error_file_couldnt_read_file", "curl_error_ldap_cannot_bind",
"curl_error_ldap_search_failed", "curl_error_obsolete40", "curl_error_function_not_found",
"curl_error_aborted_by_callback", "curl_error_bad_function_argument",
"curl_error_obsolete44", "curl_error_interface_failed", "curl_error_obsolete46",
"curl_error_too_many_redirects", "curl_error_unknown_option",
"curl_error_setopt_option_syntax", "curl_error_obsolete50", "curl_error_peer_failed_verification",
"curl_error_got_nothing", "curl_error_ssl_engine_notfound", "curl_error_ssl_engine_setfailed",
"curl_error_send_error", "curl_error_recv_error", "curl_error_obsolete57",
"curl_error_ssl_certproblem", "curl_error_ssl_cipher", "curl_error_peer_failed_verification",
"curl_error_bad_content_encoding", "curl_error_ldap_invalid_url",
"curl_error_filesize_exceeded", "curl_error_use_ssl_failed",
"curl_error_send_fail_rewind", "curl_error_ssl_engine_initfailed",
"curl_error_login_denied", "curl_error_tftp_notfound", "curl_error_tftp_perm",
"curl_error_remote_disk_full", "curl_error_tftp_illegal", "curl_error_tftp_unknownid",
"curl_error_remote_file_exists", "curl_error_tftp_nosuchuser",
"curl_error_conv_failed", "curl_error_conv_reqd", "curl_error_ssl_cacert_badfile",
"curl_error_remote_file_not_found", "curl_error_ssh", "curl_error_ssl_shutdown_failed",
"curl_error_again", "curl_error_ssl_crl_badfile", "curl_error_ssl_issuer_error",
"curl_error_ftp_pret_failed", "curl_error_rtsp_cseq_error", "curl_error_rtsp_session_error",
"curl_error_ftp_bad_file_list", "curl_error_chunk_failed", "curl_error_no_connection_available",
"curl_error_ssl_pinnedpubkeynotmatch", "curl_error_ssl_invalidcertstatus",
"curl_error_http2_stream", "curl_error_recursive_api_call", "curl_error_auth_error",
"curl_error_http3", "curl_error_quic_connect_error", "curl_error_proxy",
"curl_error_ssl_clientcert", "curl_error_unrecoverable_poll",
"curl_error_too_large", "curl_error_ech_required")
29 changes: 14 additions & 15 deletions src/library/curl/R/fetch.R
Original file line number Diff line number Diff line change
@@ -1,30 +1,29 @@
#' Fetch the contents of a URL
#'
#' Low-level bindings to write data from a URL into memory, disk or a callback
#' function. These are mainly intended for \code{httr}, most users will be better
#' off using the \code{\link{curl}} or \code{\link{curl_download}} function, or the
#' http specific wrappers in the \code{httr} package.
#' function.
#'
#' The curl_fetch functions automatically raise an error upon protocol problems
#' (network, disk, ssl) but do not implement application logic. For example for
#' you need to check the status code of http requests yourself in the response,
#' The `curl_fetch_*()` functions automatically raise an error upon protocol problems
#' (network, disk, TLS, etc.) but do not implement application logic. For example,
#' you need to check the status code of HTTP requests in the response by yourself,
#' and deal with it accordingly.
#'
#' Both \code{curl_fetch_memory} and \code{curl_fetch_disk} have a blocking and
#' Both `curl_fetch_memory()` and `curl_fetch_disk` have a blocking and a
#' non-blocking C implementation. The latter is slightly slower but allows for
#' interrupting the download prematurely (using e.g. CTRL+C or ESC). Interrupting
#' is enabled when R runs in interactive mode or when
#' \code{getOption("curl_interrupt") == TRUE}.
#' `getOption("curl_interrupt") == TRUE`.
#'
#' The \code{curl_fetch_multi} function is the asynchronous equivalent of
#' \code{curl_fetch_memory}. It wraps \code{multi_add} to schedule requests which
#' are executed concurrently when calling \code{multi_run}. For each successful
#' request the \code{done} callback is triggered with response data. For failed
#' requests (when \code{curl_fetch_memory} would raise an error), the \code{fail}
#' function is triggered with the error message.
#' The `curl_fetch_multi()` function is the asynchronous equivalent of
#' `curl_fetch_memory()`. It wraps [`multi_add()`][multi_add] to
#' schedule requests which are executed concurrently when calling
#' [`multi_run()`][multi_run]\code{}. For each successful request, the
#' `done` callback is triggered with response data. For failed requests
#' (when `curl_fetch_memory()` would raise an error), the `fail` function
#' is triggered with the error message.
#'
#' @param url A character string naming the URL of a resource to be downloaded.
#' @param handle a curl handle object
#' @param handle A curl handle object.
#' @export
#' @rdname curl_fetch
#' @useDynLib curl R_curl_fetch_memory
Expand Down
4 changes: 2 additions & 2 deletions src/library/curl/R/form.R
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#' POST files or data
#'
#' Build multipart form data elements. The \code{form_file} function uploads a
#' file. The \code{form_data} function allows for posting a string or raw vector
#' Build multipart form data elements. The `form_file` function uploads a
#' file. The `form_data` function allows for posting a string or raw vector
#' with a custom content-type.
#'
#' @param path a string with a path to an existing file on disk
Expand Down
Loading

0 comments on commit 1518384

Please sign in to comment.