Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subprocess.run -> run_subprocess #352

Merged
merged 5 commits into from
Apr 26, 2022
Merged

subprocess.run -> run_subprocess #352

merged 5 commits into from
Apr 26, 2022

Conversation

LysandreJik
Copy link
Member

Implement a run_subprocess method to remove most of the verbosity introduced by subprocess.run

@@ -86,6 +86,17 @@ def __repr__(self):
return f"[{self.title} command, status code: {status}, {'in progress.' if not self.is_done else 'finished.'} PID: {self._process.pid}]"


def run_subprocess(command, folder, check=True) -> subprocess.CompletedProcess:
return subprocess.run(
command.split(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong I think, you really want to pass list of arguments and not split arbitrarily.

For instance on filename containing spaces it would break:

filename = "my filename.txt, command is now: `["xxx", "my", "filename.txt"], instead of expected ["xxx", "my filename.txt"].

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review, will adapt the draft

@LysandreJik LysandreJik marked this pull request as ready for review October 28, 2021 20:57
@LysandreJik LysandreJik requested review from sgugger and Narsil October 28, 2021 20:57
Copy link
Contributor

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so much nicer, thanks a lot for refactoring those!

src/huggingface_hub/repository.py Outdated Show resolved Hide resolved

def run_subprocess(command, folder, check=True) -> subprocess.CompletedProcess:
if isinstance(command, str):
logger.error("`run_subprocess` should be called with a list of strings.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not a hard error?

Copy link
Contributor

@adrinjalali adrinjalali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks quite good. Pinging @LysandreJik as a reminder.

logger = get_logger(__name__)


def run_subprocess(command, folder, check=True) -> subprocess.CompletedProcess:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docstring documentation here would be nice.

@LysandreJik
Copy link
Member Author

Thanks for the reminder! Would indeed like to update this and wrap this up.

@LysandreJik
Copy link
Member Author

Should be good to go!

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Apr 18, 2022

The documentation is not available anymore as the PR was closed or merged.

logger = get_logger(__name__)


def run_subprocess(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had bad experiences in including functions in __init__ files since soon it'll cause some circular import issues. Could we move this to a utils/_subprocess.py file (I don't really feel strongly about the file name)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sounds good. Will move it around!

@LysandreJik
Copy link
Member Author

@merveenoyan are you aware of why this test might fail?

______ HubKerasFunctionalTest.test_save_pretrained_task_name_deprecation _______

self = <tests.test_keras_integration.HubKerasFunctionalTest testMethod=test_save_pretrained_task_name_deprecation>

    def test_save_pretrained_task_name_deprecation(self):
        REPO_NAME = repo_name("save")
        model = self.model_init()
        model.build((None, 2))
    
        with pytest.warns(
            FutureWarning,
            match="`task_name` input argument is removed. Pass `tags` instead.",
        ):
            save_pretrained_keras(
                model,
                f"{WORKING_REPO_DIR}/{REPO_NAME}",
                tags=["test"],
                task_name="test",
>               save_traces=True,
            )
E           Failed: DID NOT WARN. No warnings of type (<class 'FutureWarning'>,) matching ('`task_name` input argument is removed. Pass `tags` instead.') were emitted. The list of emitted warnings is: [FutureWarning('`task_name` input argument is deprecated. Pass `tags` instead.')].

Failed before and after rebasing on current main.

@adrinjalali adrinjalali merged commit e61f39b into main Apr 26, 2022
@adrinjalali adrinjalali deleted the qol-improvements branch April 26, 2022 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants