Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

controllers panic at machineScript #318

Closed
hchenxa opened this issue Apr 22, 2019 · 12 comments
Closed

controllers panic at machineScript #318

hchenxa opened this issue Apr 22, 2019 · 12 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@hchenxa
Copy link
Contributor

hchenxa commented Apr 22, 2019

/kind bug

What steps did you take and what happened:
[A clear and concise description of what the bug is.]

What did you expect to happen:

I create the cluster and the controller panic with below:

I0422 08:24:54.757650       1 controller.go:184] Reconciling machine object hchenxa-master-qrpt6 triggers idempotent create.
E0422 08:24:57.238797       1 runtime.go:66] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:72
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/panic.go:522
/usr/local/go/src/runtime/panic.go:82
/usr/local/go/src/runtime/signal_unix.go:390
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/pkg/cloud/openstack/machine/machineScript.go:54
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/pkg/cloud/openstack/machine/actuator.go:126
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/cluster-api/pkg/controller/machine/controller.go:185
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:207
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:157
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88
/usr/local/go/src/runtime/asm_amd64.s:1337
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x120 pc=0xfb525b]

goroutine 156 [running]:
sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x105
panic(0x10fcc00, 0x1eeee20)
	/usr/local/go/src/runtime/panic.go:522 +0x1b5
sigs.k8s.io/cluster-api-provider-openstack/pkg/cloud/openstack/machine.masterStartupScript(0x0, 0xc0000f02c0, 0xc0004f8a00, 0x1436, 0xc0004f8a00, 0x1436, 0x0, 0x0)
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/pkg/cloud/openstack/machine/machineScript.go:54 +0x9b
sigs.k8s.io/cluster-api-provider-openstack/pkg/cloud/openstack/machine.(*OpenstackClient).Create(0xc000610420, 0x1409d40, 0xc000042060, 0x0, 0xc0000f02c0, 0x1, 0x1)
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/pkg/cloud/openstack/machine/actuator.go:126 +0xd1a
sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/cluster-api/pkg/controller/machine.(*ReconcileMachine).Reconcile(0xc0003832c0, 0xc0003d0b99, 0x7, 0xc0005899e0, 0x14, 0x4, 0x0, 0x0, 0x0)
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/cluster-api/pkg/controller/machine/controller.go:185 +0x932
sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000444140, 0x0)
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:207 +0x17d
sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1()
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:157 +0x36
sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc000258040)
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x54
sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000258040, 0x3b9aca00, 0x0, 0x1, 0xc000414420)
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134 +0xf8
sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0xc000258040, 0x3b9aca00, 0xc000414420)
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88 +0x4d
created by sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start
	/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:156 +0x33e

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

  • Cluster-api version:
  • Minikube/KIND version:
  • Kubernetes version: (use kubectl version):
  • OS (e.g. from /etc/os-release):
@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Apr 22, 2019
@jichenjc
Copy link
Contributor

maybe provide some additional info will be helpful including the yaml files you used etc,

looks like nil reference here...
/root/gocode/src/sigs.k8s.io/cluster-api-provider-openstack/pkg/cloud/openstack/machine/machineScript.go:54

@gyliu513
Copy link
Contributor

@hchenxa I used KIND and created two clusters, it works fine. +1 to @jichenjc would love to see more reproduce steps for this issue.

@chrigl
Copy link

chrigl commented Apr 26, 2019

The same error happens when there is no cluster passed by cluster-api. Maybe #317 resolves this.

@hchenxa
Copy link
Contributor Author

hchenxa commented Apr 26, 2019

yes, I add some debug info in controller and the null point was caused by the value of cluster was nil.

@jichenjc
Copy link
Contributor

yes , I think I can work on #317 and let's see whether this still occurs..

@hchenxa
Copy link
Contributor Author

hchenxa commented Apr 28, 2019

the issue can be reproduced when using the existing the cluster as bootstrap nodes.

@jichenjc
Copy link
Contributor

you mean the cluster is existing? @hchenxa
then it might be a different issue to #317

I thought I tried this before and at that time it works (with existing cluster),so what's your command looks like?

@jichenjc
Copy link
Contributor

jichenjc commented May 5, 2019

#317 merged, will this fix your problem?
@hchenxa

@chrigl
Copy link

chrigl commented May 6, 2019

Just a side-note. Seems we don't have automated builds of master any more. So please test with an own image of openstack-cluster-api-controller:latest

@jichenjc
Copy link
Contributor

@hchenxa can you help to check whether your problem is still there or fixed by latest update?

@hchenxa
Copy link
Contributor Author

hchenxa commented May 21, 2019

sure, will let u know the result then.

@hchenxa
Copy link
Contributor Author

hchenxa commented May 23, 2019

looks like the issue was gone.

@hchenxa hchenxa closed this as completed May 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

5 participants