-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove remote store attributes from DiscoveryNode toString() #10810
Conversation
Signed-off-by: Dhwanil Patel <dhwanip@amazon.com>
Compatibility status:Checks if related components are compatible with change f50377f Incompatible componentsIncompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git] |
Gradle Check (Jenkins) Run Completed with:
|
Unrelated flaky test is failing : Created issue for it : #10813 |
Signed-off-by: Dhwanil Patel <dhwanip@amazon.com> (cherry picked from commit 911afc4) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this change!!
It would be good to have a cluster setting to control this as there might be cases where we want to see the remote store attributes, particularly when the node join validation fails. |
attributes.entrySet() | ||
.stream() | ||
.filter(entry -> !entry.getKey().startsWith(REMOTE_STORE_NODE_ATTRIBUTE_KEY_PREFIX)) // filter remote_store attributes | ||
// from logging to reduce noise. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please add details why it creates lot of noise.
For redacting the APIs response, we can do it based on cluster setting. For logging one option could be to append all the remote store attributes based if tracing is enabled. |
Another option could be to add summarized view of repos which can be enabled always. |
…rch-project#10810) Signed-off-by: Dhwanil Patel <dhwanip@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Description
Remove remote store attributes from discovery node toString method to reduce the noise in logs. With this change, it will not log the remote store attributes when it logs the node's information.
Log before fix:
[2023-10-21T11:59:37,563][INFO ][o.o.c.s.MasterService ] [a483e760c3eb.ant.amazon.com] elected-as-cluster-manager ([1] nodes joined)[{a483e760c3eb.ant.amazon.com}{h32zPMOiSdi2-bPknzppAg}{-TZ4rc9URAqRxMMrQ7urhQ}{127.0.0.1}{127.0.0.1:9300}{dimr}{remote_store.repository.my-repo-1.settings.region=XXXXX, remote_store.repository.my-repo-1.settings.bucket=XXXXXX, remote_store.repository.my-repo-1.type=XXXX, remote_store.state.repository=my-repo-1, remote_store.repository.my-repo-1.settings.base_path=XXXXX, shard_indexing_pressure_enabled=true, remote_store.segment.repository=my-repo-1, remote_store.translog.repository=my-repo-1} elect leader, _BECOME_CLUSTER_MANAGER_TASK_, _FINISH_ELECTION_], term: 6, version: 20, delta: cluster-manager node changed {previous [], current [{a483e760c3eb.ant.amazon.com}{h32zPMOiSdi2-bPknzppAg}{-TZ4rc9URAqRxMMrQ7urhQ}{127.0.0.1}{127.0.0.1:9300}{dimr}{remote_store.repository.my-repo-1.settings.region=XXXX, remote_store.repository.my-repo-1.settings.bucket=XXXXX, remote_store.repository.my-repo-1.type=XXXX, remote_store.state.repository=my-repo-1, remote_store.repository.my-repo-1.settings.base_path=XXXXXXX, shard_indexing_pressure_enabled=true, remote_store.segment.repository=my-repo-1, remote_store.translog.repository=my-repo-1}]}
Log after fix:
[2023-10-21T12:28:24,448][INFO ][o.o.c.s.MasterService ] [a483e760c3eb.ant.amazon.com] elected-as-cluster-manager ([1] nodes joined)[{a483e760c3eb.ant.amazon.com}{h32zPMOiSdi2-bPknzppAg}{HrkFLotDS1KSN6EeQdBK0Q}{127.0.0.1}{127.0.0.1:9300}{dimr}{shard_indexing_pressure_enabled=true} elect leader, _BECOME_CLUSTER_MANAGER_TASK_, _FINISH_ELECTION_], term: 8, version: 28, delta: cluster-manager node changed {previous [], current [{a483e760c3eb.ant.amazon.com}{h32zPMOiSdi2-bPknzppAg}{HrkFLotDS1KSN6EeQdBK0Q}{127.0.0.1}{127.0.0.1:9300}{dimr}{shard_indexing_pressure_enabled=true}]}
Related Issues
#10809
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.