Hub test: avoid repo creation deadlocks #32151
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| @@ -435,7 +435,8 @@ def test_push_to_hub_dynamic_processor(self): | |||
| random_repo_id = f"{USER}/test-dynamic-processor-{uuid4()}" | |||
| try: | |||
| with tempfile.TemporaryDirectory() as tmp_dir: | |||
| create_repo(random_repo_id, token=self._token) | |||
| # exist_ok=True to avoid unlikely deadlocks | |||
| create_repo(random_repo_id, token=self._token, exist_ok=True) | |||
There was a problem hiding this comment.
This is odd - given the name coming being generated with uuid, I'm surprised we're ever hitting a clash. Could you share a link to a ci run where this happens?
There was a problem hiding this comment.
@amyeroberts more than one case actually:
- https://app.circleci.com/pipelines/github/huggingface/transformers/98387/workflows/d2fbd8e3-d3ef-4a37-bdeb-0c8eee38aee3/jobs/1305499 (from RoPE: model-agnostic RoPE refactor #31999 )
- https://app.circleci.com/pipelines/github/huggingface/transformers/98491/workflows/f5c9191a-7f40-4306-acfe-a65f7663a175/jobs/1307216 (from Llama: RoPE refactor #32135 )
These two PRs are built on top of the same PR (#30910) 🤔
Important note: on these PRs, the failure is present at every commit!
There was a problem hiding this comment.
given the name coming being generated with uuid, I'm surprised we're ever hitting a clash
agreed 😅 I didn't dive to find the root cause (perhaps something seed related?), since this fix seems to solve the case and should be harmless 👼
There was a problem hiding this comment.
@amyeroberts
There was a problem hiding this comment.
since this fix seems to solve the case and should be harmless 👼
Unfortunately not completely harmless! We want to make sure we're only creating and deleting the same repo in the test.
|
@amyeroberts Rebasing fixed the persistent error 🤔 Could be related to some older bug, the root PR was a few weeks old. Closing the PR |
|
It's due to this commit being included after rebase: #32140 |
|
@amyeroberts makes complete sense! Thank you :) |
What does this PR do?
see title :)
tests/models/auto/test_processor_auto.py::ProcessorPushToHubTester::test_push_to_hub_dynamic_processor