Merging pull requests on repository with LFS not working #4463

severgun · 2018-07-18T07:01:08Z

Gitea version 1.4.3/1.5.0-rc installed from binary
Git version: 2.17.1
Operating system: Ubuntu Server 18.04 LTS
Database (use [x]):
- PostgreSQL
- MySQL
- MSSQL
- SQLite
Can you reproduce the bug at https://try.gitea.io:
- Yes (provide example URL)
- No
- Not relevant
Log gist:

Description

Merging pull requests from web interface not working on repository with LFS.
I can commit, merge and push branches on local machine with git CLI or Github for Windows, but not in gitea web.

Web Interface just fall to error 500 page after pressing Merge button.

Gitea.log:

2018/07/17 23:07:35 [...routers/repo/pull.go:533 MergePullRequest()] [E] Merge: git clone: Cloning into '/var/lib/gitea/data/tmp/local-repo/merge-192609140.git'...
done.
Downloading File.bin (11 KB)
Error downloading object: File.bin (08afa7b): Smudge error: Error downloading File.bin (08afa7bbdbb8224c12d26dad8bb3641a8c0eacd032c47bba0f91275e0d892fef): batch request: missing protocol: "/home/git/gitea-repositories/liquid/test.git/info/lfs"

Errors logged to /var/lib/gitea/data/tmp/local-repo/merge-192609140.git/.git/lfs/objects/logs/20180717T230735.214492563.log
Use git lfs logs last to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: File.bin: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

Seems like something related to lfs.url on server side, but what and how to fix?

The text was updated successfully, but these errors were encountered:

lunny · 2018-07-20T02:29:41Z

Have you installed git-lfs on your sever?

severgun · 2018-07-20T15:57:10Z

Yes.
Also, broken once LFS pull request broke all webui merges for that repo.

severgun · 2018-07-21T08:40:08Z

After I add lfs.url = http://94.000.000.000:3000/Liquid/Test.git/info/lfs in .lfsconfig and push it to master I got new error.
From batch request: missing protocol: to Merge: git push: Invalid remote name "/home/git/gitea-repositories/liquid/Test.git"

gerroon · 2018-08-07T02:38:18Z

@severgun

Did you solve this issue? I am facing it as well.

severgun · 2018-08-10T23:48:39Z

Nope. Tagged as a bug. Waiting for a fix.
Seems like gitea now in pre-alpha unstable test because there is dozens of new issues per day.

gerroon · 2018-08-10T23:52:15Z

@severgun thanks

chowey · 2018-09-12T18:30:51Z

I'm getting the following error log when merging a pull request with LFS:

2018/09/12 11:24:52 [...routers/repo/pull.go:533 MergePullRequest()] [E] Merge: git push: Invalid remote name "C:\gitea\data\repos\my\repo.git"
error: failed to push some refs to 'C:\gitea\data\repos\my\repo.git'

I'm using Gitea on Windows.

Haixing-Hu · 2018-11-22T13:22:36Z

same problem. any solution?

ghost · 2018-11-27T02:43:10Z

same problem.
OS:centos7

$ git lfs env
git-lfs/2.6.0 (GitHub; linux amd64; go 1.11.1; git 93702d75)
git version 2.16.5

LocalWorkingDir=
LocalGitDir=
LocalGitStorageDir=
LocalMediaDir=lfs/objects
LocalReferenceDirs=
TempDir=lfs/tmp
ConcurrentTransfers=3
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneRemoteName=origin
LfsStorageDir=lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic
UploadTransfers=basic
git config filter.lfs.process = "git-lfs filter-process"
git config filter.lfs.smudge = "git-lfs smudge -- %f"
git config filter.lfs.clean = "git-lfs clean -- %f"

gitea.log

2018/11/27 11:19:39 [I] LFS server enabled
2018/11/27 11:20:22 [...routers/repo/pull.go:579 MergePullRequest()] [E] Merge: git merge --no-ff --no-commit [/var/apps/gitea/data/tmp/local-repo/merge-249727395.git]: exec(11:PullRequest.Merge (git merge --no-ff --no-commit): /var/apps/gitea/data/tmp/local-repo/merge-249727395.git) failed: exit status 128(<nil>) stdout:  stderr:
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent.  See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
Downloading a.zip (34 B)

hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent.  See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
Error downloading object: a.zip (0a12b35): Smudge error: Error downloading a.zip (0a12b35d75289e158ab0d06fe388303c7275c458d51261dc44f747be6dfedc23): batch request: missing protocol: "file:///var/apps/gitea/repositories/developer/lfs-test.git/info/lfs"

Errors logged to /var/apps/gitea/data/tmp/local-repo/merge-249727395.git/.git/lfs/logs/20181127T112022.279642685.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: a.zip: smudge filter lfs failed
 -
hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent.  See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
Downloading a.zip (34 B)

hint: The remote resolves to a file:// URL, which can only work with a
hint: standalone transfer agent.  See section "Using a Custom Transfer Type
hint: without the API server" in custom-transfers.md for details.
Error downloading object: a.zip (0a12b35): Smudge error: Error downloading a.zip (0a12b35d75289e158ab0d06fe388303c7275c458d51261dc44f747be6dfedc23): batch request: missing protocol: "file:///var/apps/gitea/repositories/developer/lfs-test.git/info/lfs"

Errors logged to /var/apps/gitea/data/tmp/local-repo/merge-249727395.git/.git/lfs/logs/20181127T112022.279642685.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: a.zip: smudge filter lfs failed

stale · 2019-01-26T03:23:13Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

linghu86 · 2019-05-13T06:23:46Z

same problem

Begounet · 2019-05-23T13:19:41Z

same problem

zeripath · 2019-05-23T14:23:13Z

Which version of Gitea are you using?

Begounet · 2019-05-23T14:28:54Z

Gitea Version: 1.7.1
Edit: Do you think the issue has been fixed since?
Edit2: We just saw that the Gitea version goes from 1.7.1 to 1.8.1 in 5 months! You are so efficient!
Our admin has been told about it and will update the version tonight. I will be able to tell you about it tomorrow.
Edit3: Got an emergency, we had to delay the update. We will go back to you when it will be done :)

Begounet · 2019-05-24T10:29:23Z

Ok, so we updated to 1.8.1 but the issue is still present.
Edit: the issue occured when I tried a "merge with rebase" but not with a "standard merge". Still investigating on reproducing the issue.
Edit: I made a new test by just adding one single 4x4 image (so it is filtered by Git LFS) and the merge didn't work.

zeripath · 2019-05-25T19:32:14Z

OK so I bet this is same issue as #4760 and #732

zeripath · 2019-05-25T19:33:29Z

Now I am willing to bet that you have git-lfs version greater than 2.3.4 installed on your server and gitea user has a .gitconfig file containing:

[filter "lfs"]
	clean = git-lfs clean -- %f
	smudge = git-lfs smudge -- %f
	process = git-lfs filter-process
	required = true

If you remove that that should act as a workaround.

~~OK so that's not enough. Do your repos have .lfsconfig or is there an additional lfsconfig you're running?~~

~~In particular I'm highly suspicious that the there is lfs.storage set.~~

zeripath · 2019-05-25T19:33:55Z

~~I bet if you remove that - the problem will disappear.~~

zeripath · 2019-05-25T19:35:06Z

~~Which is obviously not a complete solution but if it works then I know that I am at least on the right track.~~

zeripath · 2019-05-25T23:08:23Z

OK, I think I've looked at this a bit more I think this is due to us previously cloning without setting --shared and without adding an alternative object dir reference for the pulled branch. What doesn't make sense though is that @Begounet has reported this on 1.8.1 - which also shouldn't be cloning without shared repository settings...

@Begounet I wonder can you reproduce this on master? What is your version of git lfs? Mine is:

:; git lfs version
git-lfs/2.3.4 (GitHub; linux amd64; go 1.8.3)

ghost · 2019-05-26T00:17:21Z

This issue is not related to the version of git-lfs.
Gitea does not store the LFS objects in repositories/xxx.git/lfs/objects, but Git transfers the files from the local directory instead of Gitea's LFS server.

local-repo/merge-xxxxx.git/.git:

# git lfs env
Endpoint=/path/repositories/test1.git/info/lfs (auth=none)
Endpoint (head_repo)=/path/repositories/test1.git/info/lfs (auth=none)

Also, I don't think it's a good idea to download the LFS file to the local temp folder and merge the PR.

zeripath · 2019-05-26T05:20:09Z

@yasuokav it might actually be related to LFS version. For example prior to 11 months ago git LFS would definitely fail on multiple alternative repositories. I don't get this problem on 2.3.4 and I know git-lfs changed their handling of local clones at some point - likely around 2.6. So if you're experiencing this bug would you please tell me what your server's version of git-lfs is? I suspect it is highly pertinent.

Now, I agree that I'm not certain that grabbing the LFS objects in order to do a merge is a good idea. I will look at stopping this. If you are experiencing this bug there are a couple of things I would like to know:

If you edit the $HOME/.gitconfig for the user running git (or wherever you've changed $HOME to be) and either comment out or remove the LFS filter declarations does that stop this from occurring?
Do any of your affected repositories have .lfsconfig files?

So that's 3 questions but the git-lfs version question is pertinent. So please tell me.

ghost · 2019-05-26T05:39:04Z

@zeripath

git-lfs/2.4.0
It does help
No

zeripath · 2019-05-26T05:58:33Z

Cool. I can check if I can reproduce this with 2.4 - (looking at the release information for 2.4 they rewrote their file API at that point so that's highly suspicious.)

Ok.

In terms of quick fix for this I would guess that the most sensible thing is for Gitea to have its own .gitconfig in which LFS is not installed. Either that or determine a way to commit without running filters.

The correct answer is to change to an index based merging style - then filters etc don't matter as we determine what goes in to the index. You shouldn't be experiencing this bug on repo edit, or upload is because I rewrote these sections to do their work directly with the index.

zeripath · 2019-05-26T05:59:33Z

But this at least finally gives those of you experiencing this bug a workaround until I get a fix in.

zeripath · 2019-05-26T06:06:59Z

Sorry this has taken so long to figure out.

zeripath · 2019-05-27T18:16:10Z

OK, I've finally had a chance to download git-lfs 2.4.0 and git-lfs 2.7.2 and I still can't reproduce this.

If you can work out a minimal testcase that breaks for you I can try to replicate and figure out exactly what's causing the breakage, however what I might do is add a note to the troubleshooting page.

zeripath · 2019-05-27T18:18:12Z

I also think we should mark this as a duplicate of #732

lafriks added the type/bug label Jul 26, 2018

lunny mentioned this issue Aug 22, 2018

500 error creating pull request for LFS repo #4760

Closed

7 tasks

stale bot added the issue/stale label Jan 26, 2019

lunny added issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented and removed issue/stale labels Feb 6, 2019

zeripath added issue/duplicate The issue has already been reported. and removed issue/confirmed Issue has been reviewed and confirmed to be present or accepted to be implemented labels May 27, 2019

zeripath closed this as completed May 27, 2019

kleszcz mentioned this issue Jul 9, 2019

LFS: Error 500 when merging pull request #732

Closed

6 tasks

go-gitea locked and limited conversation to collaborators Nov 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merging pull requests on repository with LFS not working #4463

Merging pull requests on repository with LFS not working #4463

severgun commented Jul 18, 2018

lunny commented Jul 20, 2018

severgun commented Jul 20, 2018 •

edited

Loading

severgun commented Jul 21, 2018 •

edited

Loading

gerroon commented Aug 7, 2018

severgun commented Aug 10, 2018

gerroon commented Aug 10, 2018

chowey commented Sep 12, 2018

Haixing-Hu commented Nov 22, 2018

ghost commented Nov 27, 2018 •

edited by ghost

Loading

stale bot commented Jan 26, 2019

linghu86 commented May 13, 2019

Begounet commented May 23, 2019

zeripath commented May 23, 2019

Begounet commented May 23, 2019 •

edited

Loading

Begounet commented May 24, 2019 •

edited

Loading

zeripath commented May 25, 2019

zeripath commented May 25, 2019 •

edited

Loading

zeripath commented May 25, 2019 •

edited

Loading

zeripath commented May 25, 2019 •

edited

Loading

zeripath commented May 25, 2019

ghost commented May 26, 2019 •

edited by ghost

Loading

zeripath commented May 26, 2019

ghost commented May 26, 2019

zeripath commented May 26, 2019 •

edited

Loading

zeripath commented May 26, 2019

zeripath commented May 26, 2019

zeripath commented May 27, 2019

zeripath commented May 27, 2019

Merging pull requests on repository with LFS not working #4463

Merging pull requests on repository with LFS not working #4463

Comments

severgun commented Jul 18, 2018

Description

lunny commented Jul 20, 2018

severgun commented Jul 20, 2018 • edited Loading

severgun commented Jul 21, 2018 • edited Loading

gerroon commented Aug 7, 2018

severgun commented Aug 10, 2018

gerroon commented Aug 10, 2018

chowey commented Sep 12, 2018

Haixing-Hu commented Nov 22, 2018

ghost commented Nov 27, 2018 • edited by ghost Loading

stale bot commented Jan 26, 2019

linghu86 commented May 13, 2019

Begounet commented May 23, 2019

zeripath commented May 23, 2019

Begounet commented May 23, 2019 • edited Loading

Begounet commented May 24, 2019 • edited Loading

zeripath commented May 25, 2019

zeripath commented May 25, 2019 • edited Loading

zeripath commented May 25, 2019 • edited Loading

zeripath commented May 25, 2019 • edited Loading

zeripath commented May 25, 2019

ghost commented May 26, 2019 • edited by ghost Loading

zeripath commented May 26, 2019

ghost commented May 26, 2019

zeripath commented May 26, 2019 • edited Loading

zeripath commented May 26, 2019

zeripath commented May 26, 2019

zeripath commented May 27, 2019

zeripath commented May 27, 2019

severgun commented Jul 20, 2018 •

edited

Loading

severgun commented Jul 21, 2018 •

edited

Loading

ghost commented Nov 27, 2018 •

edited by ghost

Loading

Begounet commented May 23, 2019 •

edited

Loading

Begounet commented May 24, 2019 •

edited

Loading

zeripath commented May 25, 2019 •

edited

Loading

zeripath commented May 25, 2019 •

edited

Loading

zeripath commented May 25, 2019 •

edited

Loading

ghost commented May 26, 2019 •

edited by ghost

Loading

zeripath commented May 26, 2019 •

edited

Loading