Use config defaults with a custom config #392

chuckha · 2018-03-23T22:21:25Z

Start with sensible defaults instead of zero values. If a config
is passed in it will merge into the default config instead of
a zero-value config.

Fixes #390

Signed-off-by: Chuck Ha chuck@heptio.com

What this PR does / why we need it:
This uses the default config (not the zero value config) as the base config. If no config is supplied, we end up with default values, if a config is supplied we end up with the default values overwritten by any supplied config fields.

Which issue(s) this PR fixes

Fixes Problems running systemd-logs plugin #390

Special notes for your reviewer:

We already have mergo in our dependency list!

Release note:

release-note

chuckha · 2018-03-23T22:29:00Z

Here are the logs:

$ cat cfg.json |jq
{
  "plugins": [
    {
      "name": "systemd-logs"
    }
  ]
}

test salazar:sonobuoy cha$ go run main.go run --config cfg.json 
Running plugins: systemd-logs
INFO[0027] created object                                name=heptio-sonobuoy namespace= resource=namespaces
INFO[0028] created object                                name=sonobuoy-serviceaccount namespace=heptio-sonobuoy resource=serviceaccounts
INFO[0029] created object                                name=sonobuoy-serviceaccount-heptio-sonobuoy namespace= resource=clusterrolebindings
INFO[0030] created object                                name=sonobuoy-serviceaccount namespace= resource=clusterroles
INFO[0031] created object                                name=sonobuoy-config-cm namespace=heptio-sonobuoy resource=configmaps
INFO[0032] created object                                name=sonobuoy-plugins-cm namespace=heptio-sonobuoy resource=configmaps
INFO[0033] created object                                name=sonobuoy namespace=heptio-sonobuoy resource=pods
INFO[0033] created object                                name=sonobuoy-master namespace=heptio-sonobuoy resource=services
test salazar:sonobuoy cha$ go run main.go logs -f
namespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-m5bhv" container="sonobuoy-worker"
time="2018-03-23T22:27:20Z" level=info msg="Waiting for waitfile" waitfile=/tmp/results/done
time="2018-03-23T22:27:21Z" level=info msg="Detected done file, transmitting result file" resultFile=/tmp/results/systemd_logs
namespace="heptio-sonobuoy" pod="sonobuoy" container="kube-sonobuoy"
time="2018-03-23T22:27:18Z" level=info msg="Scanning plugins in ./plugins.d (pwd: /)"
time="2018-03-23T22:27:18Z" level=info msg="Scanning plugins in /etc/sonobuoy/plugins.d (pwd: /)"
time="2018-03-23T22:27:18Z" level=info msg="Directory (/etc/sonobuoy/plugins.d) does not exist"
time="2018-03-23T22:27:18Z" level=info msg="Scanning plugins in ~/sonobuoy/plugins.d (pwd: /)"
time="2018-03-23T22:27:18Z" level=info msg="Directory (~/sonobuoy/plugins.d) does not exist"
time="2018-03-23T22:27:18Z" level=info msg="Filtering namespaces based on the following regex:.*|heptio-sonobuoy"
time="2018-03-23T22:27:18Z" level=info msg="Namespace default Matched=true"
time="2018-03-23T22:27:18Z" level=info msg="Namespace heptio-sonobuoy Matched=true"
time="2018-03-23T22:27:18Z" level=info msg="Namespace kube-public Matched=true"
time="2018-03-23T22:27:18Z" level=info msg="Namespace kube-system Matched=true"
time="2018-03-23T22:27:18Z" level=info msg="Starting server Expected Results: [{ip-10-0-18-149.us-west-2.compute.internal systemd_logs} {ip-10-0-20-112.us-west-2.compute.internal systemd_logs} {ip-10-0-22-78.us-west-2.compute.internal systemd_logs}]"
time="2018-03-23T22:27:18Z" level=info msg="starting aggregation server" address=0.0.0.0 port=8080
time="2018-03-23T22:27:18Z" level=info msg="Running plugin" plugin=systemd-logs
time="2018-03-23T22:27:21Z" level=info msg="received aggregator request" client_cert=systemd-logs node=ip-10-0-20-112.us-west-2.compute.internal plugin_name=systemd_logs
time="2018-03-23T22:27:21Z" level=info msg="received aggregator request" client_cert=systemd-logs node=ip-10-0-22-78.us-west-2.compute.internal plugin_name=systemd_logs
time="2018-03-23T22:27:21Z" level=info msg="received aggregator request" client_cert=systemd-logs node=ip-10-0-18-149.us-west-2.compute.internal plugin_name=systemd_logs
time="2018-03-23T22:27:21Z" level=info msg="Running non-ns query"
time="2018-03-23T22:27:21Z" level=error msg="error querying CustomResourceDefinitions: the server could not find the requested resource"
time="2018-03-23T22:27:21Z" level=info msg="Collecting Node Configuration and Health..."
time="2018-03-23T22:27:21Z" level=info msg="Creating host results for ip-10-0-18-149.us-west-2.compute.internal under /tmp/sonobuoy/5e2de22a-87b1-40bf-97e0-66def9e9a045/hosts/ip-10-0-18-149.us-west-2.compute.internal\n"
time="2018-03-23T22:27:21Z" level=info msg="Creating host results for ip-10-0-20-112.us-west-2.compute.internal under /tmp/sonobuoy/5e2de22a-87b1-40bf-97e0-66def9e9a045/hosts/ip-10-0-20-112.us-west-2.compute.internal\n"
time="2018-03-23T22:27:21Z" level=info msg="Creating host results for ip-10-0-22-78.us-west-2.compute.internal under /tmp/sonobuoy/5e2de22a-87b1-40bf-97e0-66def9e9a045/hosts/ip-10-0-22-78.us-west-2.compute.internal\n"
time="2018-03-23T22:27:21Z" level=error msg="error querying ThirdPartyResources: don't know how to handle non-namespaced resource ThirdPartyResources"
time="2018-03-23T22:27:21Z" level=info msg="Running ns query (default)"
time="2018-03-23T22:27:22Z" level=info msg="Collecting Pod Logs..."
time="2018-03-23T22:27:22Z" level=error msg="error querying PodPresets: the server could not find the requested resource (get podpresets.settings.k8s.io)"
time="2018-03-23T22:27:24Z" level=info msg="Running ns query (heptio-sonobuoy)"
time="2018-03-23T22:27:25Z" level=info msg="Collecting Pod Logs..."
time="2018-03-23T22:27:27Z" level=error msg="error querying PodPresets: the server could not find the requested resource (get podpresets.settings.k8s.io)"
time="2018-03-23T22:27:28Z" level=info msg="Running ns query (kube-public)"
time="2018-03-23T22:27:29Z" level=info msg="Collecting Pod Logs..."
time="2018-03-23T22:27:29Z" level=error msg="error querying PodPresets: the server could not find the requested resource (get podpresets.settings.k8s.io)"
time="2018-03-23T22:27:31Z" level=info msg="Running ns query (kube-system)"
time="2018-03-23T22:27:32Z" level=info msg="Collecting Pod Logs..."
namespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-fphx5" container="sonobuoy-worker"
time="2018-03-23T22:27:20Z" level=info msg="Waiting for waitfile" waitfile=/tmp/results/done
time="2018-03-23T22:27:21Z" level=info msg="Detected done file, transmitting result file" resultFile=/tmp/results/systemd_logs
namespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-qj96m" container="sonobuoy-worker"
time="2018-03-23T22:27:20Z" level=info msg="Waiting for waitfile" waitfile=/tmp/results/done
time="2018-03-23T22:27:21Z" level=info msg="Detected done file, transmitting result file" resultFile=/tmp/results/systemd_logs
namespace="heptio-sonobuoy" pod="sonobuoy" container="kube-sonobuoy"
time="2018-03-23T22:28:38Z" level=error msg="error querying PodPresets: the server could not find the requested resource (get podpresets.settings.k8s.io)"
namespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-m5bhv" container="sonobuoy-worker"
rpc error: code = Unknown desc = Error: No such container: df929664274ef4115efe5262f1f00f525e20e2c90a2427545ff976424558dc15namespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-fphx5" container="sonobuoy-worker"
rpc error: code = Unknown desc = Error: No such container: 5d7418834e264150ee71639b9176491ca830e6068abfd32d9a0ea62cadba1e69namespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-qj96m" container="sonobuoy-worker"
rpc error: code = Unknown desc = Error: No such container: 4880d63b7cc77659db5eddc67e6b750fd47f6a303b1b7408ae41fd80907d8a0bnamespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-m5bhv" container="sonobuoy-systemd-logs-config"
rpc error: code = Unknown desc = Error: No such container: 31985d8a29e9a0386b78a4f883bce603545c754c880aa403bf2e0035169eab23namespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-fphx5" container="sonobuoy-systemd-logs-config"
rpc error: code = Unknown desc = Error: No such container: 171b1c531535817f69e0651037748f37ba75ac68b06a7c082f90631105ea3e17namespace="heptio-sonobuoy" pod="sonobuoy-systemd-logs-daemon-set-35cab9120c4e46c3-qj96m" container="sonobuoy-systemd-logs-config"
rpc error: code = Unknown desc = Error: No such container: ea3fc98ce01dd87d2f3f64db95d3943120f39d621f4dff2d284484806b839175namespace="heptio-sonobuoy" pod="sonobuoy" container="kube-sonobuoy"
time="2018-03-23T22:28:43Z" level=info msg="Results available at /tmp/sonobuoy/201803232227_sonobuoy_5e2de22a-87b1-40bf-97e0-66def9e9a045.tar.gz"
time="2018-03-23T22:28:43Z" level=info msg="no-exit was specified, sonobuoy is now blocking"

chuckha · 2018-03-23T22:31:38Z

also of note, retrieve works and there is data despite the errors in the logs. I'll file an issue to look into it.

timothysc

I think we may want to just merge this and follow on with an issue + PR to add a test to ensure this behavior.

timothysc · 2018-03-24T00:53:34Z

pkg/client/gen.go

- }
-
+ // Set defaults for the config.Config and overwrite fields with the passed in config.
+ mergo.MergeWithOverwrite(cfg.Config, config.New())


Given that this was inbound from the customer I think we should write some tests to ensure behavior.

Start with sensible defaults instead of zero values. If a config is passed in it will merge into the default config instead of a zero-value config. Fixes vmware-tanzu#390 Signed-off-by: Chuck Ha <chuck@heptio.com>

chuckha · 2018-03-24T04:31:48Z

Adding tests, will push up another pr in a few. yay jetlag -_-

chuckha · 2018-03-24T04:38:38Z

(I closed this because I had discovered a bug while writing tests and didn't want an accidental merge 🙃)

chuckha requested review from timothysc and liztio March 23, 2018 22:21

chuckha mentioned this pull request Mar 23, 2018

Errors in sonobuoy logs -f #393

Closed

timothysc approved these changes Mar 24, 2018

View reviewed changes

Use config defaults with a custom config

826ea6e

Start with sensible defaults instead of zero values. If a config is passed in it will merge into the default config instead of a zero-value config. Fixes vmware-tanzu#390 Signed-off-by: Chuck Ha <chuck@heptio.com>

chuckha force-pushed the issue-390 branch from f6dff32 to 826ea6e Compare March 24, 2018 03:30

chuckha closed this Mar 24, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use config defaults with a custom config #392

Use config defaults with a custom config #392

chuckha commented Mar 23, 2018

chuckha commented Mar 23, 2018

chuckha commented Mar 23, 2018

timothysc left a comment

timothysc Mar 24, 2018

chuckha commented Mar 24, 2018

chuckha commented Mar 24, 2018

Use config defaults with a custom config #392

Use config defaults with a custom config #392

Conversation

chuckha commented Mar 23, 2018

chuckha commented Mar 23, 2018

chuckha commented Mar 23, 2018

timothysc left a comment

Choose a reason for hiding this comment

timothysc Mar 24, 2018

Choose a reason for hiding this comment

chuckha commented Mar 24, 2018

chuckha commented Mar 24, 2018