Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support system proxy #15234

Closed
wants to merge 6 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion contrib/pr/checkout.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
"time"

"code.gitea.io/gitea/models"
teagit "code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/markup/external"
"code.gitea.io/gitea/modules/setting"
Expand Down Expand Up @@ -80,7 +81,7 @@ func runPR() {
setting.RunUser = curUser.Username

log.Printf("[PR] Loading fixtures data ...\n")
setting.CheckLFSVersion()
teagit.CheckLFSVersion()
//models.LoadConfigs()
/*
setting.Database.Type = "sqlite3"
Expand Down
8 changes: 8 additions & 0 deletions custom/conf/app.example.ini
Original file line number Diff line number Diff line change
Expand Up @@ -1302,3 +1302,11 @@ STORAGE_TYPE = local
;MINIO_LOCATION = us-east-1
; Minio enabled ssl only available when STORAGE_TYPE is `minio`
;MINIO_USE_SSL = false

[proxy]
; Enable the proxy, all requests to external via HTTP will be affected
PROXY_ENABLED = false
; Proxy server URL, support http://, https//, socks://, blank will follow environment http_proxy/https_proxy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
; Proxy server URL, support http://, https//, socks://, blank will follow environment http_proxy/https_proxy
; Proxy server URL, support http://, https//, socks://, blank will follow environment http_proxy/https_proxy/no_proxy

PROXY_URL =
; Comma separated list of host names requiring proxy. Glob patterns (*) are accepted; use ** to match all hosts.
PROXY_HOSTS =
18 changes: 16 additions & 2 deletions docs/content/doc/advanced/config-cheat-sheet.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -523,8 +523,8 @@ Define allowed algorithms and their minimum key length (use -1 to disable a type
- `DELIVER_TIMEOUT`: **5**: Delivery timeout (sec) for shooting webhooks.
- `SKIP_TLS_VERIFY`: **false**: Allow insecure certification.
- `PAGING_NUM`: **10**: Number of webhook history events that are shown in one page.
- `PROXY_URL`: ****: Proxy server URL, support http://, https//, socks://, blank will follow environment http_proxy/https_proxy
- `PROXY_HOSTS`: ****: Comma separated list of host names requiring proxy. Glob patterns (*) are accepted; use ** to match all hosts.
- `PROXY_URL`: ****: Proxy server URL, support http://, https//, socks://, blank will follow environment http_proxy/https_proxy. If not given, will use global proxy setting.
- `PROXY_HOSTS`: ****: Comma separated list of host names requiring proxy. Glob patterns (*) are accepted; use ** to match all hosts. If not given, will use global proxy setting.

## Mailer (`mailer`)

Expand Down Expand Up @@ -902,6 +902,7 @@ Task queue configuration has been moved to `queue.task`. However, the below conf
- `ALLOWED_DOMAINS`: **\<empty\>**: Domains allowlist for migrating repositories, default is blank. It means everything will be allowed. Multiple domains could be separated by commas.
- `BLOCKED_DOMAINS`: **\<empty\>**: Domains blocklist for migrating repositories, default is blank. Multiple domains could be separated by commas. When `ALLOWED_DOMAINS` is not blank, this option will be ignored.
- `ALLOW_LOCALNETWORKS`: **false**: Allow private addresses defined by RFC 1918, RFC 1122, RFC 4632 and RFC 4291
- `SKIP_TLS_VERIFY`: **false**: Allow skip tls verify

## Mirror (`mirror`)

Expand Down Expand Up @@ -958,6 +959,19 @@ MINIO_USE_SSL = false

And used by `[attachment]`, `[lfs]` and etc. as `STORAGE_TYPE`.

## Proxy (`proxy`)

- `PROXY_ENABLED`: **true**: Enable the proxy if true, all requests to external via HTTP will be affected, if false, no proxy will be used even environment http_proxy/https_proxy
- `PROXY_URL`: ****: Proxy server URL, support http://, https//, socks://, blank will follow environment http_proxy/https_proxy
- `PROXY_HOSTS`: ****: Comma separated list of host names requiring proxy. Glob patterns (*) are accepted; use ** to match all hosts.

i.e.
```ini
PROXY_ENABLED = true
PROXY_URL = socks://127.0.0.1:1080
PROXY_HOSTS = *.github.com
```

## Other (`other`)

- `SHOW_FOOTER_BRANDING`: **false**: Show Gitea branding in the footer.
Expand Down
16 changes: 15 additions & 1 deletion docs/content/doc/advanced/config-cheat-sheet.zh-cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,8 @@ IS_INPUT_FILE = false
- `RETRY_BACKOFF`: **3**: 等待下一次重试的时间,单位秒。
- `ALLOWED_DOMAINS`: **\<empty\>**: 迁移仓库的域名白名单,默认为空,表示允许从任意域名迁移仓库,多个域名用逗号分隔。
- `BLOCKED_DOMAINS`: **\<empty\>**: 迁移仓库的域名黑名单,默认为空,多个域名用逗号分隔。如果 `ALLOWED_DOMAINS` 不为空,此选项将会被忽略。
- `ALLOW_LOCALNETWORKS`: **false**: Allow private addresses defined by RFC 1918
- `ALLOW_LOCALNETWORKS`: **false**: 允许访问私有地址,定义在 RFC 1918
- `SKIP_TLS_VERIFY`: **false**: 允许忽略 TLS 认证

## LFS (`lfs`)

Expand Down Expand Up @@ -377,6 +378,19 @@ MINIO_USE_SSL = false

然后你在 `[attachment]`, `[lfs]` 等中可以把这个名字用作 `STORAGE_TYPE` 的值。

## Proxy (`proxy`)

- `PROXY_ENABLED`: **true**: 是否启用全局代理。如果为否,则不使用代理,环境变量中的代理也不使用
- `PROXY_URL`: ****: 代理服务器地址,支持 http://, https//, socks://,为空则不启用代理而使用环境变量中的 http_proxy/https_proxy
- `PROXY_HOSTS`: ****: 逗号分隔的多个需要代理的网址,支持 * 号匹配符号, ** 表示匹配所有网站

i.e.
```ini
PROXY_ENABLED = true
PROXY_URL = socks://127.0.0.1:1080
PROXY_HOSTS = *.github.com
```

## Other (`other`)

- `SHOW_FOOTER_BRANDING`: 为真则在页面底部显示Gitea的字样。
Expand Down
6 changes: 3 additions & 3 deletions integrations/git_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ func standardCommitAndPushTest(t *testing.T, dstPath string) (little, big string
func lfsCommitAndPushTest(t *testing.T, dstPath string) (littleLFS, bigLFS string) {
t.Run("LFS", func(t *testing.T) {
defer PrintCurrentTest(t)()
setting.CheckLFSVersion()
git.CheckLFSVersion()
if !setting.LFS.StartServer {
t.Skip()
return
Expand Down Expand Up @@ -213,7 +213,7 @@ func rawTest(t *testing.T, ctx *APITestContext, little, big, littleLFS, bigLFS s
resp := session.MakeRequestNilResponseRecorder(t, req, http.StatusOK)
assert.Equal(t, littleSize, resp.Length)

setting.CheckLFSVersion()
git.CheckLFSVersion()
if setting.LFS.StartServer {
req = NewRequest(t, "GET", path.Join("/", username, reponame, "/raw/branch/master/", littleLFS))
resp := session.MakeRequest(t, req, http.StatusOK)
Expand Down Expand Up @@ -255,7 +255,7 @@ func mediaTest(t *testing.T, ctx *APITestContext, little, big, littleLFS, bigLFS
resp := session.MakeRequestNilResponseRecorder(t, req, http.StatusOK)
assert.Equal(t, littleSize, resp.Length)

setting.CheckLFSVersion()
git.CheckLFSVersion()
if setting.LFS.StartServer {
req = NewRequest(t, "GET", path.Join("/", username, reponame, "/media/branch/master/", littleLFS))
resp = session.MakeRequestNilResponseRecorder(t, req, http.StatusOK)
Expand Down
3 changes: 2 additions & 1 deletion integrations/integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (

"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/base"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/graceful"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/queue"
Expand Down Expand Up @@ -163,7 +164,7 @@ func initIntegrationTest() {
setting.SetCustomPathAndConf("", "", "")
setting.NewContext()
util.RemoveAll(models.LocalCopyPath())
setting.CheckLFSVersion()
git.CheckLFSVersion()
setting.InitDBConfig()
if err := storage.Init(); err != nil {
fmt.Printf("Init storage failed: %v", err)
Expand Down
13 changes: 7 additions & 6 deletions integrations/lfs_getobject_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import (
"testing"

"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/lfs"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/routers/routes"
Expand Down Expand Up @@ -90,7 +91,7 @@ func checkResponseTestContentEncoding(t *testing.T, content *[]byte, resp *httpt

func TestGetLFSSmall(t *testing.T) {
defer prepareTestEnv(t)()
setting.CheckLFSVersion()
git.CheckLFSVersion()
if !setting.LFS.StartServer {
t.Skip()
return
Expand All @@ -103,7 +104,7 @@ func TestGetLFSSmall(t *testing.T) {

func TestGetLFSLarge(t *testing.T) {
defer prepareTestEnv(t)()
setting.CheckLFSVersion()
git.CheckLFSVersion()
if !setting.LFS.StartServer {
t.Skip()
return
Expand All @@ -119,7 +120,7 @@ func TestGetLFSLarge(t *testing.T) {

func TestGetLFSGzip(t *testing.T) {
defer prepareTestEnv(t)()
setting.CheckLFSVersion()
git.CheckLFSVersion()
if !setting.LFS.StartServer {
t.Skip()
return
Expand All @@ -140,7 +141,7 @@ func TestGetLFSGzip(t *testing.T) {

func TestGetLFSZip(t *testing.T) {
defer prepareTestEnv(t)()
setting.CheckLFSVersion()
git.CheckLFSVersion()
if !setting.LFS.StartServer {
t.Skip()
return
Expand All @@ -163,7 +164,7 @@ func TestGetLFSZip(t *testing.T) {

func TestGetLFSRangeNo(t *testing.T) {
defer prepareTestEnv(t)()
setting.CheckLFSVersion()
git.CheckLFSVersion()
if !setting.LFS.StartServer {
t.Skip()
return
Expand All @@ -176,7 +177,7 @@ func TestGetLFSRangeNo(t *testing.T) {

func TestGetLFSRange(t *testing.T) {
defer prepareTestEnv(t)()
setting.CheckLFSVersion()
git.CheckLFSVersion()
if !setting.LFS.StartServer {
t.Skip()
return
Expand Down
3 changes: 2 additions & 1 deletion integrations/migration-test/migration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (
"code.gitea.io/gitea/models/migrations"
"code.gitea.io/gitea/modules/base"
"code.gitea.io/gitea/modules/charset"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"

Expand Down Expand Up @@ -61,7 +62,7 @@ func initMigrationTest(t *testing.T) func() {
assert.NoError(t, util.RemoveAll(setting.RepoRootPath))
assert.NoError(t, util.CopyDir(path.Join(filepath.Dir(setting.AppPath), "integrations/gitea-repositories-meta"), setting.RepoRootPath))

setting.CheckLFSVersion()
git.CheckLFSVersion()
setting.InitDBConfig()
setting.NewLogServices(true)
return deferFn
Expand Down
3 changes: 2 additions & 1 deletion models/migrations/migrations_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import (

"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/base"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/timeutil"
"code.gitea.io/gitea/modules/util"
Expand Down Expand Up @@ -55,7 +56,7 @@ func TestMain(m *testing.M) {

setting.SetCustomPathAndConf("", "", "")
setting.NewContext()
setting.CheckLFSVersion()
git.CheckLFSVersion()
setting.InitDBConfig()
setting.NewLogServices(true)

Expand Down
51 changes: 37 additions & 14 deletions modules/git/command.go
Original file line number Diff line number Diff line change
Expand Up @@ -109,48 +109,71 @@ func (c *Command) RunInDirTimeoutEnvFullPipeline(env []string, timeout time.Dura
// RunInDirTimeoutEnvFullPipelineFunc executes the command in given directory with given timeout,
// it pipes stdout and stderr to given io.Writer and passes in an io.Reader as stdin. Between cmd.Start and cmd.Wait the passed in function is run.
func (c *Command) RunInDirTimeoutEnvFullPipelineFunc(env []string, timeout time.Duration, dir string, stdout, stderr io.Writer, stdin io.Reader, fn func(context.Context, context.CancelFunc) error) error {
if timeout == -1 {
timeout = DefaultCommandExecutionTimeout
return c.RunWithContext(&RunContext{
Env: env,
Timeout: timeout,
Dir: dir,
Stdout: stdout,
Stderr: stderr,
Stdin: stdin,
CancelFunc: fn,
})
}

// RunContext represents parameters to run the command
type RunContext struct {
Env []string
Timeout time.Duration
Dir string
Stdout, Stderr io.Writer
Stdin io.Reader
CancelFunc func(context.Context, context.CancelFunc) error
}

// RunWithContext run the command with context
func (c *Command) RunWithContext(rc *RunContext) error {
if rc.Timeout == -1 {
rc.Timeout = DefaultCommandExecutionTimeout
}

if len(dir) == 0 {
if len(rc.Dir) == 0 {
log(c.String())
} else {
log("%s: %v", dir, c)
log("%s: %v", rc.Dir, c)
}

ctx, cancel := context.WithTimeout(c.parentContext, timeout)
ctx, cancel := context.WithTimeout(c.parentContext, rc.Timeout)
defer cancel()

cmd := exec.CommandContext(ctx, c.name, c.args...)
if env == nil {
if rc.Env == nil {
cmd.Env = append(os.Environ(), fmt.Sprintf("LC_ALL=%s", DefaultLocale))
} else {
cmd.Env = env
cmd.Env = rc.Env
cmd.Env = append(cmd.Env, fmt.Sprintf("LC_ALL=%s", DefaultLocale))
}

// TODO: verify if this is still needed in golang 1.15
if goVersionLessThan115 {
cmd.Env = append(cmd.Env, "GODEBUG=asyncpreemptoff=1")
}
cmd.Dir = dir
cmd.Stdout = stdout
cmd.Stderr = stderr
cmd.Stdin = stdin
cmd.Dir = rc.Dir
cmd.Stdout = rc.Stdout
cmd.Stderr = rc.Stderr
cmd.Stdin = rc.Stdin
if err := cmd.Start(); err != nil {
return err
}

desc := c.desc
if desc == "" {
desc = fmt.Sprintf("%s %s %s [repo_path: %s]", GitExecutable, c.name, strings.Join(c.args, " "), dir)
desc = fmt.Sprintf("%s %s %s [repo_path: %s]", GitExecutable, c.name, strings.Join(c.args, " "), rc.Dir)
}
pid := process.GetManager().Add(desc, cancel)
defer process.GetManager().Remove(pid)

if fn != nil {
err := fn(ctx, cancel)
if rc.CancelFunc != nil {
err := rc.CancelFunc(ctx, cancel)
if err != nil {
cancel()
_ = cmd.Wait()
Expand Down
36 changes: 36 additions & 0 deletions modules/git/git.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"time"

"code.gitea.io/gitea/modules/process"
"code.gitea.io/gitea/modules/setting"

"github.com/hashicorp/go-version"
)
Expand Down Expand Up @@ -122,10 +123,45 @@ func SetExecutablePath(path string) error {
return nil
}

// VersionInfo returns git version information
func VersionInfo() string {
var format = "Git Version: %s"
var args = []interface{}{gitVersion.Original()}
// Since git wire protocol has been released from git v2.18
if setting.Git.EnableAutoGitWireProtocol && CheckGitVersionAtLeast("2.18") == nil {
format += ", Wire Protocol %s Enabled"
args = append(args, "Version 2") // for focus color
}

return fmt.Sprintf(format, args...)
}

// Init initializes git module
func Init(ctx context.Context) error {
DefaultContext = ctx

DefaultCommandExecutionTimeout = time.Duration(setting.Git.Timeout.Default) * time.Second

if err := SetExecutablePath(setting.Git.Path); err != nil {
return err
}

// force cleanup args
GlobalCommandArgs = []string{}

if CheckGitVersionAtLeast("2.9") == nil {
// Explicitly disable credential helper, otherwise Git credentials might leak
GlobalCommandArgs = append(GlobalCommandArgs, "-c", "credential.helper=")
}

// Since git wire protocol has been released from git v2.18
if setting.Git.EnableAutoGitWireProtocol && CheckGitVersionAtLeast("2.18") == nil {
GlobalCommandArgs = append(GlobalCommandArgs, "-c", "protocol.version=2")
}

CommitsRangeSize = setting.Git.CommitsRangeSize
BranchesRangeSize = setting.Git.BranchesRangeSize

// Save current git version on init to gitVersion otherwise it would require an RWMutex
if err := LoadGitVersion(); err != nil {
return err
Expand Down
Loading