Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new flag: Enable renaming during file upload if duplicate exists #1453

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

Atreyagaurav
Copy link
Contributor

Fixes #1452

I think this will work as a flag to rename files. It won't work when the file contains multiple extension like .tar.gz which is hard to do, as we could make it work with that, but any file with multiple . in the text for no meaning will mess it up. But I can make that change if you think that's acceptable.

Also, it will break compatibility, but we could combine the overwrite and rename flags into a single flag like --duplicate-files that takes enum (rename/overwrite).

@Atreyagaurav
Copy link
Contributor Author

I also thought about just renaming when overwrite is false instead of error. Which is probably a reasonable thing to do as well. Let me know what you think.

@svenstaro
Copy link
Owner

svenstaro commented Sep 21, 2024

Thanks for this! I'd really like to see some tests here, though. It just occurred to me that we have no tests for conflicting files without any flags set nor did we have one for --overwrite-files. Would be up for making a bunch of tests showing

  1. Overwriting files is not possible without extra flags
  2. --overwrite-files works as expected
  3. --rename-duplicate works as expected
    ?

I also thought about just renaming when overwrite is false instead of error. Which is probably a reasonable thing to do as well. Let me know what you think.

I think we want an error here by default. It's safer.

Also, it will break compatibility, but we could combine the overwrite and rename flags into a single flag like --duplicate-files that takes enum (rename/overwrite).

I do like the enum idea. I'm fine breaking compatibility here. We have to make sure it's well pointed out in the CHANGELOG so people can quickly spot it.

@Atreyagaurav
Copy link
Contributor Author

Thanks for the review, I'll push with the tests once I have a bit more free time.

@Atreyagaurav
Copy link
Contributor Author

I have added the tests for the three conditions mentioned previously.

I copied the tests from the upload_file test, and modified them. The test are for failing to upload a duplicate file, checking overwrite with new contents, and checking rename flag that creates an additional file.

@svenstaro
Copy link
Owner

@Atreyagaurav Looking great! What do you think about the enum thing that we discussed above?

@Atreyagaurav
Copy link
Contributor Author

Atreyagaurav commented Sep 23, 2024

I added it as enum, and made the default be error. So only breaking change is
-o needs a value as it is no longer bool. and --overwrite-files is not available. I named the new flag on-duplicate-files

I have the tests updated as well. And I also manually tested.

@svenstaro
Copy link
Owner

Hey, I'd like to get this into the next release if possible. Could you rebase this and then it should be good to merge?

@Atreyagaurav
Copy link
Contributor Author

Hi, it looks like a lot of things were changed while doing the multipart file upload. Would have been nice to merge it before that. I'll have to look into the conflicts a little more deeply and understand the new process as well, do you think you can wait like a week for it?

Comment on lines +205 to +211
/// What to do if existing files with same name is present during file upload
///
/// If you enable renaming files, the renaming will occur by
/// adding numerical suffix to the filename before the final
/// extension. For example file.txt will be uploaded as
/// file-1.txt, the number will be increased untill a available
/// slot is found.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// What to do if existing files with same name is present during file upload
///
/// If you enable renaming files, the renaming will occur by
/// adding numerical suffix to the filename before the final
/// extension. For example file.txt will be uploaded as
/// file-1.txt, the number will be increased untill a available
/// slot is found.
/// What to do if existing files with same name is present during file upload
///
/// If you enable renaming files, the renaming will occur by
/// adding numerical suffix to the filename before the final
/// extension. For example file.txt will be uploaded as
/// file-1.txt, the number will be increased until an available
/// filename is found.

Comment on lines +292 to +298
-R, --rename-duplicate
Enable renaming files during file upload if duplicate exists

The renaming will occur by adding numerical suffix to the filename before the final extension. For example file.txt will be uploaded as file-1.txt, the number will
be increased untill a available slot is found.

[env: RENAME_DUPLICATE_FILES=]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be updated as it's not synced with the text in the --help output.

Comment on lines +443 to +449
fn write_file_contents(path: PathBuf, filename: &str, contents: &str) -> PathBuf {
let file_path = path.join(filename);
let mut file = File::create(&file_path).unwrap();
file.write_all(contents.as_bytes())
.expect("Couldn't write file");
file_path
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this function and can't we use std::fs::write() instead?

Comment on lines +287 to +300
let test_file_name = "duplicate test file.txt";
let test_file_contents = "Test File Contents";

// create the file
let test_file_path = write_file_contents(
server.path().to_path_buf(),
test_file_name,
test_file_contents,
);

// Before uploading, make sure the file is there.
let body = reqwest::blocking::get(server.url())?.error_for_status()?;
let parsed = Document::from_read(body)?;
assert!(parsed.find(Text).any(|x| x.text() == test_file_name));
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can get rid of all of this in every test because you can use fixtures::FILES and try to overwrite them. They already exist and this should allow you to cut the tests by a lot. We use fixtures::FILES in many places in the test suite.

@svenstaro
Copy link
Owner

Hi, it looks like a lot of things were changed while doing the multipart file upload. Would have been nice to merge it before that. I'll have to look into the conflicts a little more deeply and understand the new process as well, do you think you can wait like a week for it?

Yeah, sorry about that. Life happened. Anyway, I can certainly wait a week. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

No rename option, only overwrite
2 participants