Unable to get number of errors with regex_number

[alexanderbazhenoff]

Hi, not so serious bug, but...

Describe the bug
Here is a actions log where trufflehog pass, but still trying to capture number of errors.

 ##[group]�[32m✅ Linted [REPOSITORY] files with [trufflehog] successfully - (10.5s)�[0m (expand for details)
 - Using [trufflehog v3.71.0] https://megalinter.io/7.10.0/descriptors/repository_trufflehog
 - MegaLinter key: [REPOSITORY_TRUFFLEHOG]
 - Rules config: identified by [trufflehog]
 ##[endgroup]
 Unable to get number of errors with regex_count and [0-9]+:[0-9]+.*error

To Reproduce
I think any success result, but you can try my working branch.

Expected behavior
Expected no tries of getting number of errors :)

Screenshots

[echoix]

I see how it can happen, but it's kinda weird. Do you know if it happened with a previous version? Did the output change, or a new text is printed breaking it?

[alexanderbazhenoff]

@echoix Don't know, just moved to ML from SL.

[echoix]

But just to make sure of correctly assessing the importance here, does it happen only when there are no problems for real, and is still able to fail when the linter finds errors? (Is the green check correct ?)

[alexanderbazhenoff]

If I knew what it depends on... I never had trufflehog errors in this repository, but next run there was no fail tries of regex.

I'll try to see when.

[alexanderbazhenoff]

Well, now I am not sure that it was trufflehog. Can you see this log? Now it's after kics.

What debug variable I should set to see who's output this?

[echoix]

It isn't kicks, but more likely ansible:

megalinter/megalinter/descriptors/ansible.megalinter-descriptor.yml

Line 34 in 0909ca8

cli_lint_errors_regex: ": ([0-9]+) failure\\(s\\), .* warning\\(s\\) on .* files"

I'm looking a bit, but not on how to fix your code, on why is the output not matching the regex. But at the linter reports that it is a failure. But the .nice-to-have" error count is not correct

[echoix]

I wonder if it is ansible/ansible-lint#4007, that was fixed in ansible/ansible-lint#4026 and released in 24.2.1, the same day as our last release, so it wasn't included in yet. Ansible 24.2.1 is in our beta version though (beta is the current build of the main branch). You might want to try it, But you still have 678 fatal violations in the meantime : https://github.com/alexanderbazhenoff/ansible-collection-linux/actions/runs/8410583902/job/23029212065#step:4:1570

[echoix]

The warning seems to be hit when preparing the output for the next tool.

[alexanderbazhenoff]

Please check this log, there was no ansible lint.

Most of ansible-lint violation comes because these roles are made for Ansible 2.9 and linted for ansible 2.10 max (ansible-lint 5.4.0 using ansible 2.10.17). Since 2022 they add a lot of new ansible-lint rules. I need to lint and test if first on my local installed ansible-core 2.15. So I think that while I'm doing it, there's will be a new version of ML with latest ansible :) Anyway thanks for the quick reaction.

[echoix]

Ok, that's a different one. It seems that the output format switches to GitHub Actions automatically since 2.16.0, after that format being introduced in 2.15.0 (September 2020). I think that we never caught it since the tests run inside docker, and probably the detection doesn't go through. In your only error, there isn't anything after the line:column that has the text "error", so it never matches.
Here's how they create the output:
https://github.com/adrienverge/yamllint/blob/81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6/yamllint/cli.py#L80-L89

    @staticmethod
    def github(problem, filename):
        line = f'::{problem.level} file={filename},' \
               f'line={problem.line},col={problem.column}' \
               f'::{problem.line}:{problem.column} '
        if problem.rule:
            line += f'[{problem.rule}] '
        line += problem.desc
        return line

The problem level never appears after the line:column text.

The other output formats are created here:
https://github.com/adrienverge/yamllint/blob/81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6/yamllint/cli.py#L49-L88

class Format:
    @staticmethod
    def parsable(problem, filename):
        return (f'{filename}:{problem.line}:{problem.column}: '
                f'[{problem.level}] {problem.message}')

    @staticmethod
    def standard(problem, filename):
        line = f'  {problem.line}:{problem.column}'
        line += max(12 - len(line), 0) * ' '
        line += problem.level
        line += max(21 - len(line), 0) * ' '
        line += problem.desc
        if problem.rule:
            line += f'  ({problem.rule})'
        return line

    @staticmethod
    def standard_color(problem, filename):
        line = f'  \033[2m{problem.line}:{problem.column}\033[0m'
        line += max(20 - len(line), 0) * ' '
        if problem.level == 'warning':
            line += f'\033[33m{problem.level}\033[0m'
        else:
            line += f'\033[31m{problem.level}\033[0m'
        line += max(38 - len(line), 0) * ' '
        line += problem.desc
        if problem.rule:
            line += f'  \033[2m({problem.rule})\033[0m'
        return line

    @staticmethod
    def github(problem, filename):
        line = f'::{problem.level} file={filename},' \
               f'line={problem.line},col={problem.column}' \
               f'::{problem.line}:{problem.column} '
        if problem.rule:
            line += f'[{problem.rule}] '
        line += problem.desc
        return line

So until we either rewrite the regex to accept all their output formats, or impose using a certain output format, you could use either the standard, colored (standard, but with colors), or parsable output format with their -f or --format cli option.
It could be set as YAML_YAMLLINT_ARGUMENTS: "--format parsable" in your config file.

Are you fluent enough in regex to suggest one that fits the multiple output formats?

[echoix]

The regex to change is:

megalinter/megalinter/descriptors/yaml.megalinter-descriptor.yml

Line 77 in 0909ca8

cli_lint_errors_regex: "[0-9]+:[0-9]+.*error"

What we are testing for:
https://github.com/oxsecurity/megalinter/blob/0909ca81b33bcfa31c49413f0778dc6d6126f850/.automation/test/yaml/reports/ERROR-YAML_YAMLLINT.txt
https://github.com/oxsecurity/megalinter/blob/0909ca81b33bcfa31c49413f0778dc6d6126f850/.automation/test/yaml/reports/SUCCESS-YAML_YAMLLINT.txt
(as the output of the files _good and _bad in https://github.com/oxsecurity/megalinter/tree/0909ca81b33bcfa31c49413f0778dc6d6126f850/.automation/test/yaml

[alexanderbazhenoff]

The regex to change is:

megalinter/megalinter/descriptors/yaml.megalinter-descriptor.yml

Line 77 in 0909ca8

cli_lint_errors_regex: "[0-9]+:[0-9]+.*error"

Where I can replace this file before linting in my CI? May be path inside of repo or docker, right? It's a good idea to experiment with this regex when I patch all my ten millions code violations :)

PS:

❌ Linted [REPOSITORY] files with [kics]: Found 1 error(s) - (37.01s) (expand for details)
Unable to get number of errors with regex_number and : ([0-9]+) failure\(s\), .* warning\(s\) on .* files

This time after kics... 👽

[echoix]

The regex to change is:

megalinter/megalinter/descriptors/yaml.megalinter-descriptor.yml

Line 77 in 0909ca8

cli_lint_errors_regex: "[0-9]+:[0-9]+.*error"

Where I can replace this file before linting in my CI? May be path inside of repo or docker, right? It's a good idea to experiment with this regex when I patch all my ten millions code violations :)

The way we manage to use all the linters in a standard way, while they all work differently, is through the descriptors. They are copied into the docker images built. So your approach is quite clever, since they aren't really used to build the containers, they are parsed inside. So it will work to edit them inside the containers.

The full dockerfile is here:

megalinter/Dockerfile

Lines 771 to 776 in 0909ca8

	#######################################
	# Copy scripts and rules to container #
	#######################################
	COPY megalinter/descriptors /megalinter-descriptors
	COPY TEMPLATES /action/lib/.automation

But we have flavors that are trimmed down, and even individual linters published as images, like this one with just yamllint:

megalinter/linters/yaml_yamllint/Dockerfile

Lines 139 to 143 in 0909ca8

	#######################################
	# Copy scripts and rules to container #
	#######################################
	COPY megalinter/descriptors /megalinter-descriptors
	COPY TEMPLATES /action/lib/.automation

So inside the containers, the folder containing the files to change is /megalinter-descriptors

For speeding up your debugging of just one tool (the startup and all), you could use an image with only one linter. If you run locally with mega-linter-runner (a tool that calls docker with all the arguments more easily), for this image:
https://hub.docker.com/r/oxsecurity/megalinter-only-yaml_yamllint, it would be the only-yaml_yamllint flavor.
So any of these should work

npx mega-linter-runner --flavor only-yaml_yamllint
npx mega-linter-runner --flavor only-yaml_yamllint --release beta
npx mega-linter-runner --image 'oxsecurity/megalinter-only-yaml_yamllint:beta'
npx mega-linter-runner --image 'ghcr.io/oxsecurity/megalinter-only-yaml_yamllint:beta'
npx mega-linter-runner --image 'ghcr.io/oxsecurity/megalinter-only-yaml_yamllint:beta_20240319-2208'

PS:

❌ Linted [REPOSITORY] files with [kics]: Found 1 error(s) - (37.01s) (expand for details)
Unable to get number of errors with regex_number and : ([0-9]+) failure\(s\), .* warning\(s\) on .* files

This time after kics... 👽

It's not after kicks, it's before ansible-lint. The error occurs when building the summary output, so the extra warning will be before the tool. So the problem is ansible-lint again.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[alexanderbazhenoff]

Sorry, not enough time to see this. I'll be back a bit later.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.