Problems while trying to use kubeconform

[onepushmain]

Hello!

I'm having some problems while using megalinter into my kubernetes manifest repository. Seem like the megalinter don't have the option to run some kubernetes linteres/checkers

my .mega-linter.yml:

APPLY_FIXES: all # all, none, or list of linter keys
ENABLE: ["KUBERNETES"] # If you use ENABLE variable, all other languages/formats/tooling-formats will be disabled by default
SHOW_ELAPSED_TIME: true
FILEIO_REPORTER: false

logs that I get while running with this config:

----------------------------------------------------------------------------------------------------
------------------------------------ MegaLinter, by OX Security ------------------------------------
----------------------------------------------------------------------------------------------------
 - Image Creation Date: 2023-07-26T20:35:14Z
 - Image Revision: 63776c4389c474e1a6b9faa27bc8b9500fdff95a
 - Image Version: v7.2.1
----------------------------------------------------------------------------------------------------
The MegaLinter documentation can be found at:
 - https://megalinter.io/7.2.1
----------------------------------------------------------------------------------------------------
MegaLinter initialization
MegaLinter will analyze workspace [/tmp/lint]

[Activation] KUBERNETES_HELM has been set inactive, as none of these files has been found: ['Chart.yml', 'Chart.yaml']
[Activation] KUBERNETES_KUBESCAPE has been set inactive, as none of these files has been found: ['Chart.yml', 'Chart.yaml']
MARKDOWN_REMARK_LINT has been temporary disabled in MegaLinter, please use a previous MegaLinter version or wait for the next one !
Skipped linters: ACTION_ACTIONLINT, ANSIBLE_ANSIBLE_LINT, ARM_ARM_TTK, BASH_EXEC, BASH_SHELLCHECK, BASH_SHFMT, BICEP_BICEP_LINTER, CLOJURE_CLJSTYLE, CLOJURE_CLJ_KONDO, CLOUDFORMATION_CFN_LINT, COFFEE_COFFEELINT, COPYPASTE_JSCPD, CPP_CPPLINT, CSHARP_CSHARPIER, CSHARP_DOTNET_FORMAT, CSS_SCSS_LINT, CSS_STYLELINT, C_CPPLINT, DART_DARTANALYZER, DOCKERFILE_HADOLINT, EDITORCONFIG_EDITORCONFIG_CHECKER, ENV_DOTENV_LINTER, GHERKIN_GHERKIN_LINT, GO_GOLANGCI_LINT, GO_REVIVE, GRAPHQL_GRAPHQL_SCHEMA_LINTER, GROOVY_NPM_GROOVY_LINT, HTML_DJLINT, HTML_HTMLHINT, JAVASCRIPT_ES, JAVASCRIPT_PRETTIER, JAVASCRIPT_STANDARD, JAVA_CHECKSTYLE, JAVA_PMD, JSON_ESLINT_PLUGIN_JSONC, JSON_JSONLINT, JSON_NPM_PACKAGE_JSON_LINT, JSON_PRETTIER, JSON_V8R, JSX_ESLINT, KOTLIN_KTLINT, KUBERNETES_HELM, KUBERNETES_KUBECONFORM, KUBERNETES_KUBESCAPE, LATEX_CHKTEX, LUA_LUACHECK, MAKEFILE_CHECKMAKE, MARKDOWN_MARKDOWNLINT, MARKDOWN_MARKDOWN_LINK_CHECK, MARKDOWN_MARKDOWN_TABLE_FORMATTER, MARKDOWN_REMARK_LINT, OPENAPI_SPECTRAL, PERL_PERLCRITIC, PHP_PHPCS, PHP_PHPLINT, PHP_PHPSTAN, PHP_PSALM, POWERSHELL_POWERSHELL, POWERSHELL_POWERSHELL_FORMATTER, PROTOBUF_PROTOLINT, PUPPET_PUPPET_LINT, PYTHON_BANDIT, PYTHON_BLACK, PYTHON_FLAKE8, PYTHON_ISORT, PYTHON_MYPY, PYTHON_PYLINT, PYTHON_PYRIGHT, PYTHON_RUFF, RAKU_RAKU, REPOSITORY_CHECKOV, REPOSITORY_DEVSKIM, REPOSITORY_DUSTILOCK, REPOSITORY_GITLEAKS, REPOSITORY_GIT_DIFF, REPOSITORY_GRYPE, REPOSITORY_KICS, REPOSITORY_SECRETLINT, REPOSITORY_SEMGREP, REPOSITORY_SYFT, REPOSITORY_TRIVY, REPOSITORY_TRIVY_SBOM, REPOSITORY_TRUFFLEHOG, RST_RSTCHECK, RST_RSTFMT, RST_RST_LINT, RUBY_RUBOCOP, RUST_CLIPPY, R_LINTR, SALESFORCE_SFDX_SCANNER_APEX, SALESFORCE_SFDX_SCANNER_AURA, SALESFORCE_SFDX_SCANNER_LWC, SCALA_SCALAFIX, SNAKEMAKE_LINT, SNAKEMAKE_SNAKEFMT, SPELL_CSPELL, SPELL_LYCHEE, SPELL_PROSELINT, SPELL_VALE, SQL_SQLFLUFF, SQL_SQL_LINT, SQL_TSQLLINT, SWIFT_SWIFTLINT, TEKTON_TEKTON_LINT, TERRAFORM_TERRAFORM_FMT, TERRAFORM_TERRAGRUNT, TERRAFORM_TERRASCAN, TERRAFORM_TFLINT, TSX_ESLINT, TYPESCRIPT_ES, TYPESCRIPT_PRETTIER, TYPESCRIPT_STANDARD, VBDOTNET_DOTNET_FORMAT, XML_XMLLINT, YAML_PRETTIER, YAML_V8R, YAML_YAMLLINT
To receive reports as email, please set variable EMAIL_REPORTER_EMAIL

MegaLinter now collects the files to analyse
Listing all files in directory [/tmp/lint], then filter with:
- Excluding .gitignored files [16]: /tmp/lint/megalinter-reports/copy-paste/html/index.html, /tmp/lint/megalinter-reports/copy-paste/html/js/prism.js, /tmp/lint/megalinter-reports/copy-paste/html/jscpd-report.json, /tmp/lint/megalinter-reports/copy-paste/html/styles/prism.css, /tmp/lint/megalinter-reports/copy-paste/html/styles/tailwind.css, /tmp/lint/megalinter-reports/linters_logs/ERROR-COPYPASTE_JSCPD.log, /tmp/lint/megalinter-reports/linters_logs/ERROR-REPOSITORY_DEVSKIM.log, /tmp/lint/megalinter-reports/linters_logs/SUCCESS-JSON_ESLINT_PLUGIN_JSONC.log, /tmp/lint/megalinter-reports/linters_logs/SUCCESS-JSON_PRETTIER.log, /tmp/lint/megalinter-reports/linters_logs/SUCCESS-JSON_V8R.log,…(full list in DEBUG)
Kept [0] files on [104] found files

+----MATCHING LINTERS-+----------+----------------+------------+
| Descriptor | Linter | Criteria | Matching files | Format/Fix |
+------------+--------+----------+----------------+------------+


+----SUMMARY-+--------+------+-------+-------+--------+--------------+
| Descriptor | Linter | Mode | Files | Fixed | Errors | Elapsed time |
+------------+--------+------+-------+-------+--------+--------------+

Am I doing something wrong?

thanks in advance for the support and, I apologize if this is too trivial, it's my first time using this tool

[nvuillam]

@onepushmain Kubeconform looks for specific file extensions & content to be activated

Activated only if sub-directory kubernetes is found. (directory name can be overridden with KUBERNETES_DIRECTORY)
File extensions: .yml, .yaml, .json
Detected file content (regex): apiVersion:, kustomize\.config\.k8s\.io, tekton

Source: https://megalinter.io/beta/descriptors/kubernetes_kubeconform/#how-are-identified-applicable-files

How is structured your repository ? do you have a kubernetes directory ?

[TimothyEarley]

Similar issue here, at least for the KUBERNETES_HELM linter:
We have our Helm charts in its own dedicated subdirectory, i.e.

project-root/
 ├─ subdir/
 │  ├─ our-chart/
 │  │  ├─ Chart.yml
 │  │  ├─ templates/

Since the descriptor sets active_only_if_file_found the linter is skipped.

There is a variable KUBERNETES_DIRECTORY, but I could not figure out how to use it properly.

At the moment my only workaround is adding a dummy Chart.yaml file at the root and then passing the actual subdirectory with KUBERNETES_HELM_ARGUMENTS: "subdir/our-chart". Setting this config is fine, adding a dummy Chart.yaml is not.

Are there any solutions to disable the active_only_if_file_found check? Can one override it?

Thanks!

[nvuillam]

What if you try KUBERNETES_DIRECTORY: subdir ?

[TimothyEarley]

The same thing happens when KUBERNETES_DIRECTORY is set: The check for activation does not pass.

Looking at the code there are only two places that are looked at to do the check (directly in the workspace or under linter_rules_path). This linter_rules_path is global, right? So any other linters would be affected as well. I have nonetheless tried setting it and found that the workspace is in the path twice (in line 351, since in my test linter_rules_path included the absolute path). Removing the workspace from the front allows the check to go through (but using this global LINTER_RULES_PATH variable).

megalinter/megalinter/Linter.py

Lines 341 to 373 in da6a41e

	if len(self.active_only_if_file_found) > 0:
	is_found = False
	for file_to_check in self.active_only_if_file_found:
	found_file = None
	prop = None
	if ":" in file_to_check:
	file_to_check, prop = file_to_check.split(":")
	if os.path.isfile(f"{self.workspace}{os.path.sep}{file_to_check}"):
	found_file = f"{self.workspace}{os.path.sep}{file_to_check}"
	if os.path.isfile(
	f"{self.workspace}{os.path.sep}{self.linter_rules_path}{os.path.sep}{file_to_check}"
	):
	found_file = (
	f"{self.workspace}{os.path.sep}{self.linter_rules_path}"
	+ f"{os.path.sep}{file_to_check}"
	)
	# filename case
	if found_file is not None and prop is None:
	is_found = True
	break
	# filename + prop case
	if found_file is not None and prop is not None:
	with open(found_file, "r", encoding="utf-8") as json_file:
	found_file_content = json.load(json_file)
	if prop in found_file_content:
	is_found = True
	break
	if is_found is False:
	self.is_active = False
	logging.info(
	f"[Activation] {self.name} has been set inactive, as none of these files has been found:"
	f" {str(self.active_only_if_file_found)}"
	)

My config for testing was (in various variations):

ENABLE_LINTERS:
  - KUBERNETES_HELM
LINTER_RULES_PATH: subdir/our-chart

KUBERNETES_DIRECTORY: subdir/our-chart
KUBERNETES_HELM_ARGUMENTS: "subdir/our-chart"

[nvuillam]

mmmm what if we always activate the linter if KUBERNETES_DIRECTORY: any is defined ?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[Kurt-von-Laven]

Yeah, that seems like a logical solution.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[nvuillam]

A million years later, PR is on the way :)

[lukaalba]

I think this issue isn't resolved yet. With KUBERNETES_HELM and KUBERNETES_KUBESCAPE linters enabled and with the KUBERNETES_DIRECTORY environment variable set to any, I get the following log output:

[Activation] KUBERNETES_HELM skip check of directory as value set to "any"
[Activation] KUBERNETES_HELM has been set inactive, as none of these files has been found: ['Chart.yml', 'Chart.yaml']
[Activation] KUBERNETES_KUBESCAPE skip check of directory as value set to "any"
[Activation] KUBERNETES_KUBESCAPE has been set inactive, as none of these files has been found: ['Chart.yml', 'Chart.yaml']

[nvuillam]

@lukaalba it is another problem ^^

Isn't there a Chart.yml in your repo ?
Many you need us to provide a variable allowing to override it ? It's just that we can't run KUBE linters on any repo everytime there is any YML file found :/

https://megalinter.io/latest/descriptors/kubernetes_helm/#how-are-identified-applicable-files

[lukaalba]

Thanks for your reply! But I'm not sure if I fully get you here.
I have a similar project structure as @TimothyEarley with the Chart.yml placed not in the root-path of the project, but in a subfolder chart.

Reading the previous comments I would have expected that setting the KUBERNETES_DIRECTORY to any the whole project will be scanned even if a Chart.yaml is not placed at root-level. Am I mistaken here?

Setting KUBERNETES_DIRECTORY to chart doesn't work neither.

I'm not sure but to me it looks like self.files_sub_directory which is filled with the KUBERNETES_DIRECTORY value is not respected in the actual file existence check.

megalinter/megalinter/Linter.py

Lines 347 to 361 in 8fc6cb2

	for file_to_check in self.active_only_if_file_found:
	found_file = None
	prop = None
	if ":" in file_to_check:
	file_to_check, prop = file_to_check.split(":")
	if os.path.isfile(f"{self.workspace}{os.path.sep}{file_to_check}"):
	found_file = f"{self.workspace}{os.path.sep}{file_to_check}"
	if os.path.isfile(
	f"{self.workspace}{os.path.sep}{self.linter_rules_path}{os.path.sep}{file_to_check}"
	):
	found_file = (
	f"{self.workspace}{os.path.sep}{self.linter_rules_path}"
	+ f"{os.path.sep}{file_to_check}"
	)
	# filename case

[nvuillam]

@lukaalba MegaLinter will look in KUBERNETES_DIRECTORY + "/Chart.yml"

If KUBERNETES_DIRECTORY is any, it will look for Chart.yml at the root

Do you have one or multiple Chart.yml files ? ( I'm not a K8 expert ^^ )
IF you have one, you could define KUBERNETES_DIRECTORY=path/to/folder/ where Chart.yml can be found

If you have multiple, we'll have to make some enhancements ^^

[lukaalba]

I have only one Chart.yml, so this shouldn't be an issue :D

Just for clarification. With this project structure:

project-root/
 ├─ chart/
 │ ├─ Chart.yml
 │ ├─ templates/

Setting KUBERNETES_DIRECTORY to chart should work, right?

[ghost]

Hi,
first of all thank for this tool @nvuillam.

I am encountering the same issue as @lukaalba . Even if I set KUBERNETES_DIRECTORY to "chart" or "chart/" it will be ignored and i get the following message:

[Activation] KUBERNETES_HELM has been set inactive, as none of these files has been found: ['Chart.yml', 'Chart.yaml']

The directory structure is equal to the one provided by @lukaalba.

I "discovered" that setting KUBERNETES_DIRECTORY doesn't affect the "helm" and "kubescape" linter, but only kubeconform. Starting megalinter inside the chart-directory is working as expected.

Megalinter recognizes that this directory indeed exists, but it's maybe checking the wrong directory for the Chart.yaml ?
If I set KUBERNETES_DIRECTORY to something "random" you'll receive the following output for KUBERNETES_HELM:

[Activation] KUBERNETES_HELM has been set inactive, as subdirectory has not been found: someotherfolder (set value "any" to always activate)
[Activation] KUBERNETES_HELM has been set inactive, as none of these files has been found: ['Chart.yml', 'Chart.yaml']

[nvuillam]

Megalinter recognizes that this directory indeed exists, but it's maybe checking the wrong directory for the Chart.yaml ?
If I set KUBERNETES_DIRECTORY to something "random" you'll receive the following output for KUBERNETES_HELM:

I think you're right... let's provide a variable that will force KUBE linters if set to true ?

[lukaalba]

Thanks for reopening this issue!

From the user point of perspective it would be clearer how to use those linters if the helm and kubescape one respect the KUBERNETES_DIRECTORY, too. Any idea if this is possible?
The helm lint command has a path argument by default

[ghost]

Megalinter recognizes that this directory indeed exists, but it's maybe checking the wrong directory for the Chart.yaml ?
If I set KUBERNETES_DIRECTORY to something "random" you'll receive the following output for KUBERNETES_HELM:

I think you're right... let's provide a variable that will force KUBE linters if set to true ?

Sry for the late reply, i've been sick.
I'm not sure if a force-variable is really needed here. Why not just always check "force" megalinter, to look inside this directory ? Are there any known side-effects ?

[nvuillam]

@dennishoffmann-edu because the detection rules would make MegaLinter read the content of all YML files everytime... that's not fair for other MegaLinter usages performances ;)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[sstrullmyer]

@lukaalba MegaLinter will look in KUBERNETES_DIRECTORY + "/Chart.yml"

If KUBERNETES_DIRECTORY is any, it will look for Chart.yml at the root

Do you have one or multiple Chart.yml files ? ( I'm not a K8 expert ^^ ) IF you have one, you could define KUBERNETES_DIRECTORY=path/to/folder/ where Chart.yml can be found

If you have multiple, we'll have to make some enhancements ^^

Hi - I just ran into this issue as well (having a Chart.yaml file located in <workspace>/charts and Megalinter not detecting the Chart.yaml file). I tried setting KUBERNETES_DIRECTORY, similarly without success as the other reports

Assuming I'm looking in the correct location in the source code: https://github.com/oxsecurity/megalinter/blob/main/megalinter/Linter.py#L346-L379, would it be correct to say that MegaLinter isn't incorporating self.files_sub_directory to extend the checked path(s) with the KUBERNETES_DIRECTORY value?

From cursory testing (adding additional logging to Linter.py), it appears only ./Chart.yml and ././.github/linters/Chart.yml (and the .yaml equivalents) are being checked, even while KUBERNETES_DIRECTORY is being properly identified

[Kryan90]

Is there a general timeline for when the next release will be? Ran into this same issue today and realized the fix isn't present in v7.8.0

[nvuillam]

@Kryan90 there is no pre-written timeline for new releases as it depends from content and maintainers availability, but there is almost never more than one month between 2 minor releases :)

Meanwhile, you can use beta version if you are in a hurry to benefit from a features/fix :)