✨ LFX: Extend use-case of detecting deprecated Kubernetes API usage #441

Parthiba-Hazra · 2023-11-28T18:08:25Z

Added 'yq' provider.
Added rules to detect use of deprecated APIs in k8s yaml files.

Parthiba-Hazra · 2023-11-28T18:11:03Z

@JonahSussman @eemcmullan still need to add the test for those new function. Will try to complete that in few days.

JonahSussman

Looks great, thanks for submitting a contribution! Just a few changes

JonahSussman · 2023-11-29T15:37:17Z

Dockerfile

@@ -27,9 +30,12 @@ RUN python3 -m pip install python-lsp-server

 COPY --from=jaeger-builder /go/bin/all-in-one-linux /usr/bin/

+COPY --from=yq-builder /usr/bin/yq /usr/bin/yq


Thinking out loud here, but I think there was talk of doing some dependency management for external providers a while back. It's fine for now, but this method could bloat the Dockerfile when adding more providers.

It will for sure, I think we should ask @dymurray to prioritize this work for the next version.

JonahSussman · 2023-11-29T15:38:54Z

cmd/analyzer/main.go

@@ -217,7 +217,7 @@ func main() {
 	if err != nil {
 		log.Error(err, "error writing output file", "file", outputViolations)
 		os.Exit(1) // Treat the error as a fatal error
-	}	
+	}


This extra space has been bugging me for a while now, thanks! 😂

external-providers/yq-external-provider/pkg/generic/dependency.go

external-providers/yq-external-provider/pkg/generic/provider.go

external-providers/yq-external-provider/pkg/generic/service_client.go

external-providers/yq-external-provider/pkg/generic/tables.go

JonahSussman · 2023-11-29T15:58:16Z

k8s-rule.yaml

Unsure if this is where we want to put these k8s rules. Perhaps we should add it to the https://github.com/konveyor/rulesets/ repo instead? WDYT @shawn-hurley

eemcmullan · 2023-11-29T17:30:23Z

external-providers/yq-external-provider/pkg/yq_provider/provider.go

+
+	lspServerPath, ok := c.ProviderSpecificConfig[provider.LspServerPathConfigKey].(string)
+	if !ok || lspServerPath == "" {
+		return nil, fmt.Errorf("invalid lspServerPath provided, unable to init go provider")


this log mentions a go provider

eemcmullan · 2023-11-29T17:31:20Z

external-providers/yq-external-provider/pkg/yq_provider/service_client.go

+const KIND = "kind"
+const GITHUB_K8S_API_URL = "https://api.github.com/repos/kubernetes/kubernetes/releases"
+
+var default_k8s_version = "v1.28.4"


may want to think of a way to make this more dynamic

Should I add a config struct and then assign those values form a json file or yaml file?

eemcmullan · 2023-11-29T17:38:16Z

k8s-rule.yaml

@@ -0,0 +1,1034 @@
+- message: Deprecated/removed Kubernetes API version 'extensions/v1beta1' is used for 'Deployment'. Consider using 'apps/v1'.


Could we maybe get these rules separated into rulesets by the api group? i.e. rules for apps, rules for networking, etc..

will look after it. need some time

eemcmullan · 2023-11-29T17:38:56Z

k8s-rule.yaml

+        removedIn: "v1.16.0"
+        replacementAPI: "apps/v1"
+- message: Deprecated/removed Kubernetes API version 'apps/v1beta2' is used for 'Deployment'. Consider using 'apps/v1'.
+  ruleID: k8s-deprecated-api-002


If the above is done, we could then have more descriptive ruleIDs

shawn-hurley · 2023-11-29T18:22:37Z

Dockerfile


 RUN make build

+# Add yq to the build stage
+FROM docker.io/mikefarah/yq as yq-builder


note to us, we probably need to fork this and build it out of konveyor in the future

shawn-hurley · 2023-11-29T18:28:56Z

k8s-rule.yaml

@@ -0,0 +1,1034 @@
+- message: Deprecated/removed Kubernetes API version 'extensions/v1beta1' is used for 'Deployment'. Consider using 'apps/v1'.


Can these rules be generated or did you do this by hand?

nah it's not generated

I think we can find a way to get this automated.

Wow, that must have taken a long time, thank you. This is going to be a nice feature!

trust me, that didn't take much time 😅

shawn-hurley

I think this makes a lot of sense; I wonder if we could make it an external provider in its own repo, though maybe in the konveyor-externsions and make a flag to build an image with it built in or something?

Just wondering, if it is too much work right now I am cool with adding it and doing the messy work after :)

shawn-hurley · 2023-11-29T18:43:31Z

@Parthiba-Hazra This is awesome work! Thank you so much for taking the time to work on this and contributing!

shawn-hurley · 2023-12-04T18:36:59Z

@Parthiba-Hazra can you rebase then I think we can merge

JonahSussman

LGTM!

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

Signed-off-by: Shawn Hurley <shawn@hurley.page>

eemcmullan · 2024-01-03T15:59:45Z

@Parthiba-Hazra Do you mind updating the demo testing output?

Parthiba-Hazra · 2024-01-03T16:28:39Z

@Parthiba-Hazra Do you mind updating the demo testing output?

yes sure,

Signed-off-by: Parthiba-Hazra <parthibahazra@gmail.com>

k8s-rule.yaml

JonahSussman requested review from JonahSussman, eemcmullan, shawn-hurley and fabianvf November 29, 2023 15:09

JonahSussman requested changes Nov 29, 2023

View reviewed changes

eemcmullan requested changes Nov 29, 2023

View reviewed changes

shawn-hurley reviewed Nov 29, 2023

View reviewed changes

JonahSussman self-requested a review December 4, 2023 19:09

JonahSussman reviewed Dec 4, 2023

View reviewed changes

JonahSussman approved these changes Dec 4, 2023

View reviewed changes

Parthiba-Hazra added 10 commits December 5, 2023 18:43

add yq provider

4806d4e

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

trying yq-provider

8179f04

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

trying yq-provider

e398533

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

fix minor issues

7091145

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

add all the k8s deprecated rules

1b5870a

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

some minor changes

a93b128

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

some minor changes

dd2d201

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

add requested changes

52075f7

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

some minor changes

06ce2c9

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

resolve merge conflict

eae64fa

Signed-off-by: Parthiba-Hazra <parthibahazra02@gmail.com>

Parthiba-Hazra force-pushed the yq-proivider branch from a58211b to eae64fa Compare December 5, 2023 13:32

shawn-hurley approved these changes Dec 5, 2023

View reviewed changes

eemcmullan approved these changes Dec 6, 2023

View reviewed changes

Merge branch 'main' into yq-proivider

1bfe5a6

Signed-off-by: Shawn Hurley <shawn@hurley.page>

update the demo testing output

7086df4

Signed-off-by: Parthiba-Hazra <parthibahazra@gmail.com>

pranavgaikwad requested changes Jan 10, 2024

View reviewed changes

k8s-rule.yaml Show resolved Hide resolved

pranavgaikwad approved these changes Jan 11, 2024

View reviewed changes

shawn-hurley merged commit f6cdf40 into konveyor:main Jan 12, 2024
6 checks passed

jwmatthews mentioned this pull request Feb 14, 2024

[LFX Fall '23 Mentorship]: Extend use-case of detecting deprecated Kubernetes API usage konveyor/operator#251

Closed

This was referenced May 8, 2024

operator konveyor-operator (0.4.0-alpha.1) k8s-operatorhub/community-operators#4324

Merged

operator konveyor-operator (0.4.0-alpha.1) redhat-openshift-ecosystem/community-operators-prod#4509

Merged

This was referenced Jun 5, 2024

operator konveyor-operator (0.4.0) k8s-operatorhub/community-operators#4455

Merged

operator konveyor-operator (0.4.0) redhat-openshift-ecosystem/community-operators-prod#4623

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ LFX: Extend use-case of detecting deprecated Kubernetes API usage #441

✨ LFX: Extend use-case of detecting deprecated Kubernetes API usage #441

Parthiba-Hazra commented Nov 28, 2023

Parthiba-Hazra commented Nov 28, 2023

JonahSussman left a comment

JonahSussman Nov 29, 2023

shawn-hurley Nov 29, 2023

JonahSussman Nov 29, 2023

JonahSussman Nov 29, 2023

eemcmullan Nov 29, 2023

eemcmullan Nov 29, 2023 •

edited

Loading

Parthiba-Hazra Nov 30, 2023

eemcmullan Nov 29, 2023

Parthiba-Hazra Nov 30, 2023

eemcmullan Nov 29, 2023

shawn-hurley Nov 29, 2023

shawn-hurley Nov 29, 2023

Parthiba-Hazra Nov 30, 2023

Parthiba-Hazra Nov 30, 2023

shawn-hurley Dec 1, 2023

Parthiba-Hazra Dec 2, 2023

shawn-hurley left a comment

shawn-hurley commented Nov 29, 2023

shawn-hurley commented Dec 4, 2023

JonahSussman left a comment

eemcmullan commented Jan 3, 2024

Parthiba-Hazra commented Jan 3, 2024

		@@ -27,9 +30,12 @@ RUN python3 -m pip install python-lsp-server

		COPY --from=jaeger-builder /go/bin/all-in-one-linux /usr/bin/

		COPY --from=yq-builder /usr/bin/yq /usr/bin/yq

		@@ -0,0 +1,1034 @@
		- message: Deprecated/removed Kubernetes API version 'extensions/v1beta1' is used for 'Deployment'. Consider using 'apps/v1'.

✨ LFX: Extend use-case of detecting deprecated Kubernetes API usage #441

✨ LFX: Extend use-case of detecting deprecated Kubernetes API usage #441

Conversation

Parthiba-Hazra commented Nov 28, 2023

Parthiba-Hazra commented Nov 28, 2023

JonahSussman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eemcmullan Nov 29, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shawn-hurley left a comment

Choose a reason for hiding this comment

shawn-hurley commented Nov 29, 2023

shawn-hurley commented Dec 4, 2023

JonahSussman left a comment

Choose a reason for hiding this comment

eemcmullan commented Jan 3, 2024

Parthiba-Hazra commented Jan 3, 2024

eemcmullan Nov 29, 2023 •

edited

Loading