Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTPClient大量请求没有释放 #1822

Closed
xxxlzj520 opened this issue May 11, 2022 · 3 comments
Closed

HTTPClient大量请求没有释放 #1822

xxxlzj520 opened this issue May 11, 2022 · 3 comments

Comments

@xxxlzj520
Copy link

c := g.Client()
defer c.CloseIdleConnections()
url:=fmt.Sprintf("http://xxxx/inner/books/owned/list?uid=%d",i)
r, err := c.Get(gctx.New(), url)
defer r.Close()
if err != nil{
	fmt.Println(err)
	
}else {
	fmt.Println(r.StatusCode)
	
}

gf版本:v2.0.6
现像描述:当for循环调用请求时,数据过多会导致请求没有释放,本地打开文件数过多

@huangqian1985
Copy link
Member

huangqian1985 commented May 11, 2022

g.Client() 底层是对原生http.Client的封装, 代码见

c := &Client{
Client: http.Client{
Transport: &http.Transport{
// No validation for https certification of the server in default.
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
DisableKeepAlives: true,
},
},
header: make(map[string]string),
cookies: make(map[string]string),
}

http.Client原生是支持连接池的处理的, http.Transport的MaxConnsPerHost和MaxIdleConnsPerHost属性控制了每个Client最大支持的请求对象的数量, MaxIdleConns属性控制了保持多少个活跃连接.

所以其实在for循环里一直创建gClient对象不是一个好的处理方案.

但是我看了gClient的方法,并没有自定义http.Client的方式, 是否考虑加一个相关的方法?可以开一个help wanted 我来操作一波 @gqcn

@xxxlzj520 如果你比较着急的话, 我建议你使用gpool的方式来做实现. 文档见 https://goframe.org/pages/viewpage.action?pageId=1114377

@gqcn gqcn added the question label May 11, 2022
@gqcn gqcn closed this as completed May 11, 2022
@gqcn gqcn reopened this May 11, 2022
@gqcn
Copy link
Member

gqcn commented May 11, 2022

@huangqian1985 @xxxlzj520 是的,本来http.Client就有连接池控制的,不过我试了下,似乎r.Close并没有把链接归还到连接池,应该是有什么设置?
image

加了MaxConnsPerHost连接池参数设置,就阻塞了:

package main

import (
	"fmt"
	"net/http"

	"github.com/gogf/gf/v2/frame/g"
	"github.com/gogf/gf/v2/os/gctx"
)

func main() {
	c := g.Client()
	defer c.CloseIdleConnections()
	c.Transport.(*http.Transport).MaxConnsPerHost = 3
	for {
		r, err := c.Get(gctx.New(), `https://baidu.com`)
		if err != nil {
			fmt.Println(err)
		} else {
			fmt.Println(r.StatusCode)
			_ = r.Close()
		}
	}
}

image

@huangqian1985
Copy link
Member

[WeOpen Star]I would like to help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants