Inherit submodules from template repository content

[sschroe]

This PR addresses issue #10316 and makes submodules available in repositories created from templates.
Submodules are now inherited when creating a new repository from a template with "Git Content" checked,

Unlike the initial suggestion provided by @davidsvantesson in the original issue i decided not to use git submodule status initially because its output can look like this:

 0dff7e5493898954537202c0c975f7b1c55d0845 lib (no%_()delay)1 (heads/master)

Submodule names can contain all sorts of special characters and the (heads/master) part is not always present in the output which makes reliably parsing the name out of this quite a pain if not impossible. I replaced this step with git config -f .gitmodules --list --name-only which will give:

submodule.lib (no%_()delay)1.path
submodule.lib (no%_()delay)1.url

This is far easier to parse and provides the necessary submodule name. git submodule status lib (no%_()delay)1 is then used as second step to get the commit hash which is now easily parsed as well as there is only a single line of output.

These steps happen in the getSubmodules function and are the largest part of the change. This function does not throw any errors but will just skip broken submodules so that a template with a malformed .gitmodules file can still be used.

To re add the submodules i use git update-index --add --cacheinfo 160000 0dff7e5493898954537202c0c975f7b1c55d0845 lib (no%_()delay)1 instead of the regular git submodule add .... Updating the index directly skips cloning the submodule which is far faster and entirely avoids the problem of users adding excessively large submodules to possibly DoS attack the Gitea server. It also avoids the issue where a submodule repository might not be accessible by the Gitea server due to authentication/permissions or other reasons such as absolute local file paths being used.

[6543]

@sschroe If you dont force-push and not swash, it's simpler to review and see made changes ...

... wills will be squash-merged if accepted anyways

[6543]

@sschroe avter #16243 got merged you dont need to rename import :)

and I think it should be more log.Debug sorry for writing log.Info first ...

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (main@1f05417). Click here to learn what that means.
The diff coverage is 70.42%.

@@           Coverage Diff           @@
##             main   #16237   +/-   ##
=======================================
  Coverage        ?   47.42%           
=======================================
  Files           ?      951           
  Lines           ?   132431           
  Branches        ?        0           
=======================================
  Hits            ?    62801           
  Misses          ?    62071           
  Partials        ?     7559           

Impacted Files	Coverage Δ
modules/repository/generate.go	29.85% <40.00%> (ø)
modules/git/submodule.go	74.62% <72.72%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1f05417...a467a2b. Read the comment docs.

[sschroe]

@6543 all changes should be implemented so far. Just for the pipelines i need a little more info, is it enough to change the command to use RunInDirPipeline or is there more to that?

[6543]

https://github.com/go-gitea/gitea/blob/main/modules/git/repo_ref_nogogit.go#L16 is a nice example

[sschroe]

Any other input or changes needed here to proceed?

[wxiaoguang]

Ok will do so (and try to resolve conflichts ... next week :) )

Maybe we could get it in Jan 1 2025