Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to set just monitoring.cluster_uuid #14338

Merged
merged 10 commits into from
Nov 20, 2019

Conversation

ycombinator
Copy link
Contributor

@ycombinator ycombinator commented Oct 30, 2019

Currently there is a bug in libbeat which allows users to set monitoring.cluster_uuid only if there is also a monitoring reporter configured (either via output.elasticsearch.* settings or monitoring.elasticsearch.* settings). If there are no monitoring reporter settings (but monitoring.cluster_uuid is set), the value of monitoring.cluster_uuid is not exposed via the GET state API. This further does not allow the monitoring.cluster_uuidMetricbeat's beat module to monitor a beat that has specified the monitoring.cluster_uuid setting in it's configuration. Ultimately this results in the monitored beat being shown under a "Standalone Cluster" in the Stack Monitoring UI.

Therefore, we should allow users to configure the monitoring.cluster_uuid setting even when no other settings under the monitoring namespace are configured.

This PR fixes this bug by:

  • parsing out the monitoring.cluster_uuid setting before the reporter configuration is done;
  • exposing the parsed out value via the monitoring registry, regardless of whether monitoring is enabled or not. This allows the GET state API to use, so Metricbeat's beat module may be able to fetch it; and
  • considering monitoring as disabled if the only setting under monitoring is cluster_uuid, as we expect the other settings to be related to the reporter (monitoring.elasticsearch.*).

This PR also does a bit of cleanup by extracting the monitoring setup code out of (*Beat).launch and into its own method.

Testing this PR

  1. Build Filebeat (or any other Beat) with this PR.
    cd $GOPATH/src/github.com/elastic/beats/filebeat
    mage build
    
  2. Disable the elasticsearch output in your Beat and enable some other output.
    output.elasticsearch:
      enabled: false
    output.console:
      enabled: true
    
  3. Start the beat while setting http.enabled=true and monitoring.cluster_uuid=<some value>.
    ./filebeat -e -E http.enabled=true -E monitoring.cluster_uuid=foobar
    
  4. Call the State API for your Beat and check that the monitoring.cluster_uuid value is reported.
    curl -s 'http://localhost:5066/state' | jq .monitoring.cluster_uuid
    

@elasticmachine
Copy link
Collaborator

Pinging @elastic/stack-monitoring (Stack monitoring)

libbeat/monitoring/monitoring.go Show resolved Hide resolved
default:
return nil, nil, nil
}
}

func GetMonitoringClusterUUID(monitoringCfg *common.Config) (string, error) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exported function GetMonitoringClusterUUID should have comment or be unexported

@ycombinator ycombinator changed the title Allow users to just set monitoring.cluster_uuid Allow users to set just monitoring.cluster_uuid Oct 30, 2019
@ycombinator ycombinator force-pushed the lb-bugfix-mon-cluster-uuid branch 3 times, most recently from 0c97867 to baa4771 Compare November 12, 2019 18:56
Copy link
Contributor

@cachedout cachedout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know the libbeat codebase well enough to leave a thorough review on the code changes but I tested this functionally and it looks good to me.

Copy link
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cachedout
Copy link
Contributor

@ycombinator Was this a bad rebase? It looks like this suddenly has many unrelated changes.

@ycombinator
Copy link
Contributor Author

@cachedout Thanks, indeed, I did a bad rebase 🤦‍♂. Fixing now...

@ycombinator
Copy link
Contributor Author

Bad rebase is fixed.

@ycombinator
Copy link
Contributor Author

Travis CI is green and Jenkins CI failures are unrelated. Merging.

@ycombinator ycombinator merged commit a75e3e5 into elastic:master Nov 20, 2019
@ycombinator ycombinator deleted the lb-bugfix-mon-cluster-uuid branch November 20, 2019 01:11
ycombinator added a commit to ycombinator/beats that referenced this pull request Nov 20, 2019
* Allow users to just set monitoring.cluster_uuid

* Adding back comment lost in refactoring

* Adds godoc comments for functions

* Removes stutter in function name

* Adding system test

* Adding CHANGELOG entry

* Fixing formatting

* Move defer to right scope

* Adding system test

* Fixing formatting
ycombinator added a commit to ycombinator/beats that referenced this pull request Nov 20, 2019
* Allow users to just set monitoring.cluster_uuid

* Adding back comment lost in refactoring

* Adds godoc comments for functions

* Removes stutter in function name

* Adding system test

* Adding CHANGELOG entry

* Fixing formatting

* Move defer to right scope

* Adding system test

* Fixing formatting
ycombinator added a commit that referenced this pull request Nov 20, 2019
* Allow users to set just monitoring.cluster_uuid (#14338)

* Allow users to just set monitoring.cluster_uuid

* Adding back comment lost in refactoring

* Adds godoc comments for functions

* Removes stutter in function name

* Adding system test

* Adding CHANGELOG entry

* Fixing formatting

* Move defer to right scope

* Adding system test

* Fixing formatting

* Fixing up CHANGELOG
ycombinator added a commit that referenced this pull request Nov 21, 2019
* Allow users to just set monitoring.cluster_uuid

* Adding back comment lost in refactoring

* Adds godoc comments for functions

* Removes stutter in function name

* Adding system test

* Adding CHANGELOG entry

* Fixing formatting

* Move defer to right scope

* Adding system test

* Fixing formatting
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…elastic#14633)

* Allow users to set just monitoring.cluster_uuid (elastic#14338)

* Allow users to just set monitoring.cluster_uuid

* Adding back comment lost in refactoring

* Adds godoc comments for functions

* Removes stutter in function name

* Adding system test

* Adding CHANGELOG entry

* Fixing formatting

* Move defer to right scope

* Adding system test

* Fixing formatting

* Fixing up CHANGELOG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants