Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud integration tests are failing #679

Open
imilchev opened this issue Nov 11, 2022 · 6 comments · Fixed by #683
Open

Cloud integration tests are failing #679

imilchev opened this issue Nov 11, 2022 · 6 comments · Fixed by #683
Assignees
Labels
bug Something isn't working ci testing

Comments

@imilchev
Copy link
Member

The cloud integration tests have been failing for a while now. We should investigate the root cause and fix them https://github.com/mondoohq/mondoo-operator/actions/runs/3406390108/jobs/5665044460

@czunker czunker added bug Something isn't working ci testing labels Nov 15, 2022
@czunker
Copy link
Contributor

czunker commented Nov 15, 2022

From the output, two possible problems:

Messages:   	Couldn't find expected version in MondooAuditConfig.Status.ReconciledByOperatorVersion

or

integration/audit_config_base_suite.go:801	Webhook is not available yet: Post "https://127.0.0.1:18443/validate-k8s-mondoo-com": dial tcp 127.0.0.1:18443: connect: connection refused

Bot were already addressed in previous PRs: #648 and #660

@czunker czunker self-assigned this Nov 15, 2022
@czunker
Copy link
Contributor

czunker commented Nov 15, 2022

The VERSION is missing for the integration tests:
image

That's because the docker step to fetch the version failed:
image

The docker step is missing ${{ env.MONDOO_OPERATOR_IMAGE_TAG }}:
image

czunker added a commit that referenced this issue Nov 15, 2022
Fixes #679

Signed-off-by: Christian Zunker <christian@mondoo.com>
czunker added a commit that referenced this issue Nov 16, 2022
Fixes #679

Signed-off-by: Christian Zunker <christian@mondoo.com>
@czunker
Copy link
Contributor

czunker commented Nov 21, 2022

They still fail, but with a different error: https://github.com/mondoohq/mondoo-operator/actions/runs/3510168153

@czunker czunker reopened this Nov 21, 2022
@czunker
Copy link
Contributor

czunker commented Nov 21, 2022

Different errors:

In the webhook:

2022-11-20T23:18:56Z	ERROR	webhook-validator	failed to set labels for incoming request	{"error": "object does not implement the Object interfaces"}
go.mondoo.com/mondoo-operator/pkg/webhooks/handler.(*webhookValidator).Handle
	/home/runner/work/mondoo-operator/mondoo-operator/pkg/webhooks/handler/webhook.go:113
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).Handle
	/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/webhook/admission/webhook.go:169
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).ServeHTTP
	/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/webhook/admission/http.go:98
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1
	/home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.12.2/prometheus/promhttp/instrument_server.go:40
net/http.HandlerFunc.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2109
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1
	/home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.12.2/prometheus/promhttp/instrument_server.go:117
net/http.HandlerFunc.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2109
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2
	/home/runner/go/pkg/mod/github.com/prometheus/client_golang@v1.12.2/prometheus/promhttp/instrument_server.go:84
net/http.HandlerFunc.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2109
net/http.(*ServeMux).ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2487
net/http.serverHandler.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2947
net/http.initALPNRequest.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:3556
net/http.(*http2serverConn).runHandler
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/h2_bundle.go:5911

And:

2022-11-20T23:18:56Z	INFO	webhook-validator	Webhook triggered	{"kind": "Deployment", "resource": "mondoo-operator/passing-deployment"}
2022-11-20T23:18:56Z	INFO	webhook-validator	Scan result	{"shouldAdmit": false, "kind": "Deployment", "resource": "kube-system/coredns-autoscaler", "worstscore": {"value":100}}
2022-11-20T23:18:56Z	DEBUG	controller-runtime.webhook.webhooks	wrote response	{"webhook": "/validate-k8s-mondoo-com", "code": 403, "reason": "FAILED MONDOO SCAN", "UID": "1e7b33f6-7702-4966-944e-b1a2ab94b009", "allowed": false}

From the scan API logs:

�[90m2022-11-21T10:21:59Z�[0m �[32mINF�[0m discover related assets for 1 asset(s)
�[90m2022-11-21T10:21:59Z�[0m �[32mINF�[0m resolved assets �[36mresolved-assets=�[0m0
�[90m2022-11-21T10:21:59Z�[0m �[32mINF�[0m scan took 5.8µs

This seems fine.

@czunker
Copy link
Contributor

czunker commented Dec 5, 2022

@czunker
Copy link
Contributor

czunker commented Dec 12, 2022

This time they failed because of missing protc: https://github.com/mondoohq/mondoo-operator/actions/runs/3671233346/jobs/6206299007

Fix PR is up: #723

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ci testing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants