Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

netpoll should handle error with backoff #230

Closed
ganlvtech opened this issue Jan 12, 2023 · 4 comments
Closed

netpoll should handle error with backoff #230

ganlvtech opened this issue Jan 12, 2023 · 4 comments
Labels

Comments

@ganlvtech
Copy link

Describe the bug

netpoll should handle error with backoff

To Reproduce

mkdir hello
cd hello
go mod init hello
curl -L -O https://github.com/cloudwego/hertz-examples/raw/main/hello/main.go
go mod tidy
go build .
ulimit -n 10
./hello

Open another terminal and telnet several times.

telnet 127.0.0.1 8888 &
telnet 127.0.0.1 8888 &

Now there are tons of logs. It may be 10MB per second.

2023/01/13 04:16:02 accept conn failed: too many open files
2023/01/13 04:16:02 accept conn failed: too many open files
......

Expected behavior

Don't print so many logs.

Screenshots

netpoll server
image
go net http server
image

Server:

  • OS: Ubuntu 20.04 Linux 5.4.0-47-generic
  • go: 1.19.5
  • hertz: v0.5.0
  • netpoll: v0.3.1

Additional context

netpoll server implement

netpoll/netpoll_server.go

Lines 96 to 103 in 6a5a4f7

// shut down
if strings.Contains(err.Error(), "closed") {
s.operator.Control(PollDetach)
s.onQuit(err)
return err
}
logger.Println("NETPOLL: accept conn failed:", err.Error())
return err

go net http server implement

https://github.com/golang/go/blob/245e95dfabd77f337373bf2d6bb47cd353ad8d74/src/net/http/server.go#L3064-L3076

@joway
Copy link
Member

joway commented Jan 13, 2023

I think the true problem is that your dont have enough max_open_files.

try to use ulimit -n 1024000 on your linux os and try again.

@ganlvtech
Copy link
Author

ulimit is a way that system provided to prevent a process from using too many resources. accept syscall returns EMFILE or ENFILE when it reaches the limit.
The limit could be very large (eg. 1024000) or be the default 65535, and it may also be limit to 1024 by the system admin on purpose.
Maybe some other code exactly needs to open many files to search for some thing. So less file descriptors for web service.

go net http server handled it. It won't cry for giving it larger fd limit. It just wait once per second for other connection closed.

@joway
Copy link
Member

joway commented Feb 13, 2023

go net http server handled it. It won't cry for giving it larger fd limit.

It's a interesting suggestion, will reconsider it. thanks!

@joway joway added the wanted label Feb 13, 2023
@joway joway added feature and removed wanted labels Mar 28, 2023
@joway
Copy link
Member

joway commented Feb 22, 2024

@ganlvtech #311 already find a way to fix it

@joway joway closed this as completed Feb 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

2 participants