Add basic repository lfs management

[zeripath]

This PR adds some LFS management settings to repositories. It adds an LFS tab to the settings page.

Associated LFS files

Clicking on this tab will list all the LFS files associated with the repository:

• In the top right there is a button that will search the repository for possible LFS pointers showing Pointer Files view below.
• Clicking on the brown OID will allow you to view the contents of the LFS file (see below).
• Clicking on the Find Commits will search the repository for commits with the probable pointer file for this OID.
• Clicking on the trashcan will allow you to disassociate and possibly delete this file from the repository - this may not be reversible.

View LFS File

Clicking on a brown OID will show this page:

Pointer Files

This may take some time to process and cannot be paged.

It searches the whole object database for blobs less than 1kb in size that match an LFS pointer file. This checks whether they exist in the LFS database and are associated with the current repo. There are various features allowing you autoassociate if files are not associated in the repository and you can see them from another repo.

• Similar to the Associated LFS files view clicking on the brown OID button will show the LFS file.
• Clicking on the grey SHA button will show the raw pointer file
• Clicking on the blue Find Commits button will find commits adding the SHA (please note the subtle difference - the previous screen guesses the pointer file SHA.)

Find Commits

This may take sometime in a big repository and cannot be paged.

Find commits searches the commits in the repository for those that add the appropriate pointer file for the provided LFS file.

• Clicking on the file name will view that file in the tree from the commit.
• Clicking on the commit message will show the commit diff that adds the file.
• The branch name is generated in the same way as that on the diff page.
• Clicking on the blue SHAs will show the commit diffs for the parents and commit as appropriate.

[codecov-io]

Codecov Report

Merging #7199 into master will decrease coverage by 0.28%.
The diff coverage is 15.31%.

@@            Coverage Diff             @@
##           master    #7199      +/-   ##
==========================================
- Coverage   41.67%   41.38%   -0.29%     
==========================================
  Files         533      537       +4     
  Lines       68723    69239     +516     
==========================================
+ Hits        28641    28657      +16     
- Misses      36407    36906     +499     
- Partials     3675     3676       +1

Impacted Files	Coverage Δ
modules/git/repo.go	55.94% <0%> (-0.56%)	⬇️
routers/repo/lfs.go	0% <0%> (ø)
modules/lfs/server.go	45.7% <0%> (ø)	⬆️
modules/git/pipeline/namerev.go	0% <0%> (ø)
modules/repofiles/upload.go	0% <0%> (ø)	⬆️
modules/repofiles/update.go	38.75% <0%> (ø)	⬆️
models/repo_list.go	74.27% <100%> (+0.12%)	⬆️
services/pull/lfs.go	71.26% <100%> (+0.09%)	⬆️
routers/routes/routes.go	82.57% <100%> (+0.19%)	⬆️
models/lfs.go	17.27% <5.17%> (-15.46%)	⬇️
... and 10 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af8957b...134a946. Read the comment docs.

[zeripath]

Ok, so things left to do for this.

• it would be nice if there was some way of mapping the git repo blob SHA for the pointer - or most likely pointer back to the tree and filename that has this SHA. There is no easy way to do this although likely replicate the technique used by @filipnavara in repo.Home would be best.
• I don't think there is any sensible way of paging the LFS pointer search - but it might be possible.

[guillep2k]

Perhaps in the future there could be an LFS database table that you could be fed in action.go, so queries can be made on that table instead of needing to scan all the repos. That would allow for pagination.

[lunny]

Please resolve the conflicts

[zeripath]

Conflicts resolved.

[zeripath]

@guillep2k to precache these we'd have to run a filter similar to those suggested above every time there is a commit and cache the results. The worry I have is that these may not be appropriate to be stored in the database directly - these results will simply grow and grow and probably don't need the benefits of a SQL server. I guess we could store the results of this in files inside the git or LFS object db - basically as LFS file OID ->list of Commit SHA, Tree SHA, filepath. (The git object db manages concurrency for us.)

[guillep2k]

Should we support custom renderers? Perhaps as a future PR?

[zeripath]

? is this a new feature that I've missed.

[guillep2k]

I've seen that custom renderers were affected by a regression and then a PR fixed that. May be it doesn't apply here? #4105 and #7869

[zeripath]

Ah we don't know the name so we can't actually attempt to render with markup! I knew there was a reason why I hadn't implemented this.

[6543]

the only question i had is: can I reupload deleted lfs files? but this is Off-Topic

[6543]

@zeripath are you done with this?

[6543]

@zeripath #8719 ... what do you think of adding this to your page too?

[zeripath]

@6543 Yeah you're right that would probably be helpful. Why don't we add that as part of that PR or another? Having checked the markup issue I'm gonna merge this now.