Git LFS GET request malformed #21525
Comments
|
Looks like the LFS server implementation hasn't followed |
|
From the git history it looks like serve direct was added in #16731 and that is Gitea 1.16. So it's a bit surprising it worked for you in 1.15.8. |
|
Could you get the generated s3 link and try to download it manually? |
Indeed. We suspect it worked for us in 1.15.8 because serve direct didn't apply to LFS at the time. The update to 1.17.3 effectively enabled serve direct for LFS and revealed the bug. We have temporarily disabled serve direct to work around this issue and everything looks okay so far. We would like to re-enable serve direct at some point, though.
We tried the generated signed URLs and they are valid. We were able to download the files manually. However, when used through git-lfs, git-lfs sends this We used curl to try the signed URLs with and without the |
|
Ok, then the fix should be something like this: @@ -438,13 +438,14 @@ func buildObjectResponse(rc *requestContext, pointer lfs_module.Pointer, downloa
}
if download {
- rep.Actions["download"] = &lfs_module.Link{Href: rc.DownloadLink(pointer), Header: header}
if setting.LFS.ServeDirect {
// If we have a signed url (S3, object storage), redirect to this directly.
u, err := storage.LFS.URL(pointer.RelativePath(), pointer.Oid)
if u != nil && err == nil {
- rep.Actions["download"] = &lfs_module.Link{Href: u.String(), Header: header}
+ rep.Actions["download"] = &lfs_module.Link{Href: u.String()}
}
+ } else {
+ rep.Actions["download"] = &lfs_module.Link{Href: rc.DownloadLink(pointer), Header: header}
}
}
if upload {(removed the "headers to send" field) |
|
But when get LFS url failed, we need fallback |
|
Fix in #21531 |
Fixes #21525 Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Fixes go-gitea#21525 Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Description
Hello,
We have updated our gitea to version 1.17.3. We were coming from version 1.15.8.
We then had an issue where we were not able to fetch LFS files anymore.
On our test server, we have been able to reproduce the issue with gitea v1.16.0. After a downgrade to v1.15.8, we think that it was working fine.
Important note: Git LFS clients are directly communicate with our s3. (means gitea is configured with SERVE_DIRECT: true)
After investigation, we saw that the git lfs client was receiving from gitea invalid credentials for the LFS requests.
The LFS client was receiving a correct presigned url to download the file but it was doing the query to this url with an HTTP "Authorization" header which contains a bearer token.
Please note that this token is exactly the same as the token used to authenticate to gitea for LFS POST requests.
With manual tests, we were able to download the file but as soon as we were adding this famous HTTP header, the request was failing with HTTP 400 with the following response from our S3:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidArgument</Code><RequestId>tx000000000000000000792-006351619a-1101ec-default</RequestId><HostId>1101ec-default-default</HostId></Error>%To sumup:
The presigned URL received by gitea is ok and is usable but then the git lfs client is adding this HTTP header. The addition of this HTTP header in the request is making it failing.
We are wondering why the token to authenticate to gitea is used in the request to s3.
It may come from this part of the code:
gitea/cmd/serv.go
Lines 279 to 283 in 6a03309
As a workaround, we have switched the "SERVE_DIRECT" to "false" but we would like to come back to true because it means that it is our gitea machine which is getting the whole "LFS load" instead of our s3.
The git trace of what we have seen is here:
Gitea Version
1.17.3
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
2.28.0
Operating System
Centos 7.9.2009
How are you running Gitea?
We are running the binary downloaded from the gitea website through a systemd service.
Database
MySQL
The text was updated successfully, but these errors were encountered: