Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SIGSEGV in kube.GetResourcesWithLabel #587

Closed
jessesuen opened this issue Sep 12, 2018 · 1 comment
Closed

SIGSEGV in kube.GetResourcesWithLabel #587

jessesuen opened this issue Sep 12, 2018 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@jessesuen
Copy link
Member

Noticed that an application controller had exited with the following error:

time="2018-08-21T20:02:20Z" level=info msg="cache miss: apires|https://CLUSTER-URL-REDACTED"
time="2018-08-21T20:02:20Z" level=warning msg="Failed to watch app resources on https://CLUSTER-URL-REDACTED: Get https://CLUSTER-URL-REDACTED/api: dial tcp: lookup CLUSTER-URL-REDACTED on 100.64.0.10:53: no such host, retrying in 10s"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x45676c]

goroutine 7342065 [running]:
github.com/argoproj/argo-cd/util/kube.GetResourcesWithLabel.func1(0xc4213abe80, 0x1a57840, 0xc42106fcb0, 0x1572e47, 0x21, 0xc420dcb670, 0xe, 0xc4211174d0, 0xc420c53700)
        /root/go/src/github.com/argoproj/argo-cd/util/kube/kube.go:331 +0x3a3
created by github.com/argoproj/argo-cd/util/kube.GetResourcesWithLabel
        /root/go/src/github.com/argoproj/argo-cd/util/kube/kube.go:316 +0x695
@jessesuen jessesuen added the bug Something isn't working label Sep 12, 2018
@jessesuen
Copy link
Member Author

This was on v0.7.1. The issue is with concurrent appends to a list:

	var result []*unstructured.Unstructured

	var wg sync.WaitGroup
	wg.Add(len(resourceInterfaces))
	for i := range resourceInterfaces {
		client := resourceInterfaces[i]
		go func() {
			defer wg.Done()
			list, err := client.List(metav1.ListOptions{
				LabelSelector: fmt.Sprintf("%s=%s", labelName, labelValue),
			})
			if err != nil {
				asyncErr = err
				return
			}
			// apply client side filtering since not every kubernetes API supports label filtering
			for i := range list.(*unstructured.UnstructuredList).Items {
				item := list.(*unstructured.UnstructuredList).Items[i]
				labels := item.GetLabels()
				if labels != nil {
					if value, ok := labels[labelName]; ok && value == labelValue {
						result = append(result, &item)            <<<<< this append is not thread safe
					}
				}
			}
		}()
	}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant