Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add max_connections to TCP input #12691

Merged
merged 5 commits into from
Jul 1, 2019
Merged

Add max_connections to TCP input #12691

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
564563a
add max_connections to TCP input of Filebeat
kvch Jun 26, 2019
107c58f
add to reference config
kvch Jun 26, 2019
75ca612
when 0 is configured do not limit number of connections
kvch Jun 27, 2019
13741d2
update x-pack filebeat config
kvch Jun 28, 2019
2ec998d
fix configuration
kvch Jul 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions filebeat/_meta/common.reference.inputs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,9 @@ filebeat.inputs:
# Maximum size in bytes of the message received over TCP
#max_message_size: 20MiB

# Maximum number of connections
#max_connections: 10

# The number of seconds of inactivity before a remote connection is closed.
#timeout: 300s

Expand Down
3 changes: 3 additions & 0 deletions filebeat/filebeat.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -640,6 +640,9 @@ filebeat.inputs:
# Maximum size in bytes of the message received over TCP
#max_message_size: 20MiB

# Maximum number of connections
#max_connections: 10

# The number of seconds of inactivity before a remote connection is closed.
#timeout: 300s

Expand Down
1 change: 1 addition & 0 deletions filebeat/input/tcp/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ var defaultConfig = config{
Config: tcp.Config{
Timeout: time.Minute * 5,
MaxMessageSize: 20 * humanize.MiByte,
MaxConnections: 10,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the effects of this default value on a typical node? Are we confident that this is high enough not to artificially bottleneck a node that was previously handling a higher load? I'd be happier if this value was higher, or if the default was "no maximum"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! Updated the configuration to be unlimited by default.

},
LineDelimiter: "\n",
}
1 change: 1 addition & 0 deletions filebeat/inputsource/tcp/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ type Config struct {
Host string `config:"host"`
Timeout time.Duration `config:"timeout" validate:"nonzero,positive"`
MaxMessageSize cfgtype.ByteSize `config:"max_message_size" validate:"nonzero,positive"`
MaxConnections int `config:"max_connections" validate:"nonzero,positive"`
TLS *tlscommon.ServerConfig `config:"ssl"`
}

Expand Down
17 changes: 15 additions & 2 deletions filebeat/inputsource/tcp/server.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ import (
"net"
"sync"

"golang.org/x/net/netutil"

"github.com/elastic/beats/filebeat/inputsource"
"github.com/elastic/beats/libbeat/common/transport/tlscommon"
"github.com/elastic/beats/libbeat/logp"
Expand Down Expand Up @@ -175,12 +177,23 @@ func (s *Server) allClients() []*client {
}

func (s *Server) createServer() (net.Listener, error) {
var l net.Listener
var err error
if s.tlsConfig != nil {
t := s.tlsConfig.BuildModuleConfig(s.config.Host)
s.log.Info("Listening over TLS")
return tls.Listen("tcp", s.config.Host, t)
l, err = tls.Listen("tcp", s.config.Host, t)
if err != nil {
return nil, err
}
} else {
l, err = net.Listen("tcp", s.config.Host)
if err != nil {
return nil, err
}
}
return net.Listen("tcp", s.config.Host)

return netutil.LimitListener(l, s.config.MaxConnections), nil
}

func (s *Server) clientsCount() int {
Expand Down
74 changes: 74 additions & 0 deletions vendor/golang.org/x/net/netutil/listen.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions vendor/vendor.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2345,6 +2345,12 @@
"version": "release-branch.go1.9",
"versionExact": "release-branch.go1.9"
},
{
"checksumSHA1": "CkkyCjLPBm0OGzfMuTfaDsd45X4=",
"path": "golang.org/x/net/netutil",
"revision": "3b0461eec859c4b73bb64fdc8285971fd33e3938",
"revisionTime": "2019-06-20T19:42:36Z"
},
{
"checksumSHA1": "QEm/dePZ0lOnyOs+m22KjXfJ/IU=",
"path": "golang.org/x/net/proxy",
Expand Down
3 changes: 3 additions & 0 deletions x-pack/filebeat/filebeat.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -759,6 +759,9 @@ filebeat.inputs:
# Maximum size in bytes of the message received over TCP
#max_message_size: 20MiB

# Maximum number of connections
#max_connections: 10

# The number of seconds of inactivity before a remote connection is closed.
#timeout: 300s

Expand Down