Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase #1

Merged
merged 1,117 commits into from
Apr 17, 2019
Merged

Rebase #1

merged 1,117 commits into from
Apr 17, 2019

Conversation

felix-lessoer
Copy link
Owner

No description provided.

Steffen Siering and others added 30 commits March 8, 2019 20:30
Fix an issue with a partial enroll, when a user refused to overrides a
local configuration actually the enroll command did already used the
token on the ES cluster, this commit move the confirm in the CM instead
of having it in the Enroll's function and is executed by sending the
token or creating any files on disk.

Fixes: #10150
Moved the definition of `agent.hostname` to fields.common.yml
Missing changelog for #11067
…2 interface (#10962)

This is the first Kubernetes modules which uses the new testing interface too
Fixes a number of issues in the `user` dataset discovered while testing on Fedora 29:

* When calling C functions via cgo, don't rely on errno as an indicator of success - check the return value first.
* Set errno to 0 before calling getpwent(3) or getspent(3).
* Make C function calls thread-safe.
* Address a systemd bug where it can return ENOENT even when there is no error.
* Don't fail on every error.
* Fail system test on WARN/ERROR messages in the log.
These dashboards are now maintained in the new elastic/uptime-contrib repo, so the docs should point there.
TCP checks are not adding URL fields on NXDOMAIN endpoints. This fixes that issue.

It does so by ensuring that URL metadata is added before executing the check, and not during, as done previously.

A side effect of this is that we now perform DNS lookups once per `{hostname,port}` instead of once per `{hostname}`. This is worth the increased simplicity however, as the code would be quite convoluted otherwise, which would put us at risk for more bugs.

Related (but different) 6.x issue: #10777

Fix import formatting
This PR is part of  #10071.

Co-Authored-By: Stamatis Katsaounis <katsaouniss@gmail.com>
Co-Authored-By: Michael Katsoulis <michaelkatsoulis88@gmail.com>
* Import updated ECS 1.0.0 go library.
* Adjust Packetbeat to use int64 for URL ports.
* Re-generate golden files for filebeat and x-pack/filebeat
* Fix fields.yml for sqs from message.* to messages.* and field type
* Change type back to long
* Add format: bytes
After repeatedly running the Elasticsearch module integration test in Metricbeat, I found that sometimes Elasticsearch doesn't get enough time to perform CCR and generate CCR stats. This causes the following error, but only some times:

```
--- FAIL: TestFetch (2.44s)
    --- FAIL: TestFetch/ccr (0.08s)
        elasticsearch_integration_test.go:92:
                Error Trace:    elasticsearch_integration_test.go:92
                Error:          Should NOT be empty, but was []
                Test:           TestFetch/ccr
```

So this PR adds a 300ms sleep to give Elasticsearch enough time to perform CCR and generate CCR stats. After testing various sleep durations, I found that 300ms seemed to be the lowest (round) value I could use that consistently passed this test.

Possibly related: #10866
The Monitoring Bulk API (`POST _xpack/monitoring/_bulk`) does not interpret `_type` in the bulk metadata the same was as the regular Bulk API (`POST _bulk`). In the case of the Monitoring Bulk API, `_type` has a special meaning. It does not correspond to the Elasticsearch document `_type` but rather a `type` field within monitoring documents. The `_type` of those monitoring documents gets automatically set to `_doc` by the Monitoring Bulk API.

Therefore, the Beats monitoring code should always send `_type` to the Monitoring Bulk API.
* Add instance.state.name into fields.yml

* Rebase and regenerate fields.yml

* Update changelog
* Fix sqs queue name
* Add function to get queueUrls
* Fix field names from message to messages
* Move looping through each queueURL into a separate function
* Update aws documentation
* Add checks for instance fields in TestFetch
* Add tests for CheckTimestampInArray and FindTimestamp
* Update comment for CheckTimestampInArray
* Change test to table driven
* Adopt CheckTimestampInArray and FindTimestamp in S3 and SQS
This updates the ingest pipeline loading to look at the
pipeline definition file extension and use a JSON or YAML
decoder depending on the type.
The structures `Field` and `Fields` are moved from `common` to the package `mapping`.
This is a step towards moving things from the abused package `common`.
I would like to move fields and other asset related functionality to this package in the future.
At the moment, docker containers are shared between tests. Password test
changes the password as part of its checks, what can affect the
connection of other tests using the same instance, skip this test by
now to avoid flakiness in the rest of tests.
SIGHUP is sent to a process if the user's terminal is disconnected. It is one of the five termination signals. Currently, Beats is going to terminate suddenly, i.e. the log will cut off, no shutdown actions are going to be run, etc.

This change catches SIGHUP as it already does SIGINT/SIGTERM and runs the usual shutdown actions. This can include waiting for a period of time specified using the shutdown_timeout option Filebeat and Packetbeat support.
ycombinator and others added 29 commits April 13, 2019 05:33
We're starting to see errors like this quite frequently (but not all the time) in our CI:

```
Creating metricbeat91226fb6ffe6d06676cfc1383d0a3ffb19f00db7_kibana_1 ... 
    stats_integration_test.go:60: 
        	Error Trace:	stats_integration_test.go:60
        	Error:      	Should be empty, but was [error making http request: Get http://kibana:5601/api/stats?extended=true: net/http: request canceled (Client.Timeout exceeded while awaiting headers)]
        	Test:       	TestFetch
    stats_integration_test.go:61: 
        	Error Trace:	stats_integration_test.go:61
        	Error:      	Should NOT be empty, but was []
        	Test:       	TestFetch
```

So this PR skips this test suite for now, while we try to debug and fix the root cause in #11380.
* Fix flaky service_integration_windows_test

Some services can change state before reader.Read() call is executed which will result in a test faillure.
Fix test by introducing a confidence factor and enriching the error message with the state-changed service details.

Fixes #8880 and #7977

* Update metricbeat/module/windows/service/service_integration_windows_test.go

Co-Authored-By: narph <mariana@elastic.co>
* Add option to set rcv_buffer for UDP inputs

* Missed err handling

* Import style fix

* Implement review feedback

* add documentation

* Add option to set rcv_buffer for UDP inputs

* Missed err handling

* Import style fix

* Implement review feedback

* add documentation

* Correct changelog link

* Apply gofmt suggestion

* Apply indentation recommendation from gofmt

* Newline for linter

* Autogen reference file

* x-pack fmt update

* Restore import order that was modified by fmt update
* migrate redis to reporter V2 with error return
This PR add a new input configuration option named `line_terminator`:

```
# Characters which separate the lines. Valid values: auto, line_feed, vertical_tab, form_feed,
# carriage_return, carriage_return_line_feed, next_line, line_separator, paragraph_separator.
#line_terminator: auto
```
The option `auto` tells Filebeat to use our current hybrid new line finder approach. Thus, we can avoid introducing a breaking change.

It also contains a minor refactoring in `readfile` package. I have created a new type `Config` which stores the configuration of the readers of the package. This eliminates a long list of parameters in the constructors of `EncodeReader` and `LineReader`.


Closes #5500
* migrate kvm to reporterv2 with error handling
* Fix TODO in Suricata doc

Add config example to replace TODO in Suricata module documentation.
…#11822)

* add line about ptrace blocks in the system metricset
* Un-skip flaky tests

* Making healthcheck more robust
…ay can be freed (#11524)

In `LimitReader` when truncating the line if the configured limit was reached, the message was sliced. Slicing does not touch the underlying array. To make sure the long line can be truncated and freed, the reader copies the contents to a new array. Thus, freeing the data.
Add CoreDNS metrics and adapt some existing names
* Add filebeat module envoyproxy for Envoy access logs
@felix-lessoer felix-lessoer merged commit 132d6c3 into felix-lessoer:master Apr 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.