Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Http Headers #52

Closed
2moe opened this issue Jan 14, 2023 · 1 comment · Fixed by #53
Closed

Custom Http Headers #52

2moe opened this issue Jan 14, 2023 · 1 comment · Fixed by #53
Labels
feedback/question Further information is requested

Comments

@2moe
Copy link
Contributor

2moe commented Jan 14, 2023

Problem

Some websites restrict the UA, so if it is an invalid UA (null UA or IE), then it will deny service.
So I need the ua header.

When I used trauma::downloader::DownloaderBuilder, I found that I could not add headers directly.


const FIREFOX_UA: &str =
    "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0";
let ua = http::header::HeaderValue::from_str(FIREFOX_UA)?;

trauma/src/downloader.rs

Lines 156 to 161 in a5f5bb7

// Request the file.
debug!("Fetching {}", &download.url);
let mut req = client.get(download.url.clone());
if self.resumable && can_resume {
req = req.header(RANGE, format!("bytes={}-", size_on_disk));
}

It is not difficult to add the header.

req = req.headers(HeaderMap::from_iter([(http::header::USER_AGENT, ua)]));

But I had to modify the trauma lib, instead of using DownloaderBuilder::new().headers(...).

Incidentally, I think it should also support custom filename, instead of relying entirely on form_urlencoded to automatically parse

trauma/src/downloader.rs

Lines 27 to 38 in a5f5bb7

#[derive(Debug, Clone)]
pub struct Downloader {
/// Directory where to store the downloaded files.
directory: PathBuf,
/// Number of retries per downloaded file.
retries: u32,
/// Number of maximum concurrent downloads.
concurrent_downloads: usize,
/// Downloader style options.
style_options: StyleOptions,
/// Resume the download if necessary and possible.
resumable: bool,

filename: Option<OsString>,
headermap: Option<HeaderMap>,
@2moe 2moe added the feedback/question Further information is requested label Jan 14, 2023
@2moe 2moe changed the title Custom Http Header Custom Http Headers Jan 14, 2023
@2moe 2moe changed the title Custom Http Headers Custom Http Headers & filename Jan 14, 2023
@rgreinho
Copy link
Owner

Oh I see! That's a good idea for a feature to add. Do you happen to have a PR I could review by any chance?

Regarding the custom filename, this feature already exists. You can look at the Download::new() method. It allows you to pass a custom name to the download.

@2moe 2moe changed the title Custom Http Headers & filename Custom Http Headers Jan 15, 2023
@kodiakhq kodiakhq bot closed this as completed in #53 Jan 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feedback/question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants