FIX ?? send ref in LFS endpoint

[Wauplin]

cc @coyotte508 @Pierrci @severo @SBrandeis

Related to https://github.com/huggingface/moon-landing/pull/8103 and slack thread (internal links).

TL;DR: this fix will allow the parquet converter to push to refs/convert/parquet even on repos that have their discussion tab disabled.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[severo]

Tried on OpenOrca and still got:

{
    "error": "Commit 0/1 could not be created on the Hub (after 6 attempts).",
    "cause_exception": "HfHubHTTPError",
    "cause_message": "403 Client Error: Forbidden for url: https://huggingface.co/datasets/Open-Orca/OpenOrca.git/info/lfs/objects/batch (Request ID: Root=1-655b5da1-38fde8e96340f8505ab01ade;230a4803-659c-4cf4-9346-d12dce0de7a8)\n\nAuthorization error.",
    "cause_traceback": [
        "Traceback (most recent call last):\n",
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py", line 270, in hf_raise_for_status\n response.raise_for_status()\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status\n raise HTTPError(http_error_msg, response=self)\n',
        "requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/datasets/Open-Orca/OpenOrca.git/info/lfs/objects/batch\n",
        "\nThe above exception was the direct cause of the following exception:\n\n",
        "Traceback (most recent call last):\n",
        ' File "/src/services/worker/src/worker/utils.py", line 205, in decorator\n return func(*args, **kwargs)\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn\n return fn(*args, **kwargs)\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 1150, in _inner\n return fn(self, *args, **kwargs)\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 3491, in create_commit\n self.preupload_lfs_files(\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/hf_api.py", line 3961, in preupload_lfs_files\n _upload_lfs_files(\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn\n return fn(*args, **kwargs)\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/_commit_api.py", line 353, in _upload_lfs_files\n batch_actions_chunk, batch_errors_chunk = post_lfs_batch_info(\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn\n return fn(*args, **kwargs)\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/lfs.py", line 160, in post_lfs_batch_info\n hf_raise_for_status(resp)\n',
        ' File "/src/services/worker/.venv/lib/python3.9/site-packages/huggingface_hub/utils/_errors.py", line 330, in hf_raise_for_status\n raise HfHubHTTPError(str(e), response=response) from e\n',
        "huggingface_hub.utils._errors.HfHubHTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/datasets/Open-Orca/OpenOrca.git/info/lfs/objects/batch (Request ID: Root=1-655b5da1-38fde8e96340f8505ab01ade;230a4803-659c-4cf4-9346-d12dce0de7a8)\n\nAuthorization error.\n",
        "\nThe above exception was the direct cause of the following exception:\n\n",
        "Traceback (most recent call last):\n",
        ' File "/src/services/worker/src/worker/job_runners/config/parquet_and_info.py", line 913, in create_commits\n commit_info = retry_create_commit(\n',
        ' File "/src/services/worker/src/worker/utils.py", line 210, in decorator\n raise RuntimeError(f"Give up after {attempt} attempts. The last one raised {type(last_err)}") from last_err\n',
        "RuntimeError: Give up after 6 attempts. The last one raised <class 'huggingface_hub.utils._errors.HfHubHTTPError'>\n",
    ],
}

[Wauplin]

After some local debugging together with @severo , we found out that the revision we were passing to the Hub was url-encoded which is not expected by the server. I pushed a fix in 5136b6d and this time it should be finally good!

[Pierrci]

Thanks @Wauplin, LGTM!