trufflehog linter raises spurious false positives

[quotidian-ennui]

Describe the bug

New 'trufflehog' linter causes issues out of the box since it detects potential secrets inside the ".git" directory if you run megalinter as a github action.

To Reproduce
Steps to reproduce the behavior:

1. Install and use megalinter as a github action; with pull_request: write and contents:write permissions
2. Trigger it.

You will intermittently get things like

❌ [ERROR] for workspace /github/workspace
Linter raw log:
2023-07-25T16:20:29Z	info-0	trufflehog	loaded decoders	{"count": 3}
2023-07-25T16:20:29Z	info-0	trufflehog	loaded detectors	{"total": 748, "verification_enabled": 748, "verification_disabled": 0}
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

Found unverified result 🐷🔑❓
Detector Type: Github
Decoder Type: BASE64
Raw result: ghs_xxxxxxx
File: .git/config
Line: 11

• This is potentially correct but still a false positive because there's not an awful lot you can do about it... (since I imageine that this is under the control of actions/checkout)

Results of trufflehog linter (version 3.45.0)
See documentation on https://megalinter.io/7.2.0/descriptors/repository_trufflehog/
-----------------------------------------------

❌ [ERROR] for workspace /tmp/lint
Linter raw log:
2023-07-25T15:03:12Z	info-0	trufflehog	loaded decoders	{"count": 3}
2023-07-25T15:03:12Z	info-0	trufflehog	loaded detectors	{"total": 748, "verification_enabled": 748, "verification_disabled": 0}
🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

Found unverified result 🐷🔑❓
Detector Type: Github
Decoder Type: PLAIN
Raw result: -some-hash-here
File: .git/logs/HEAD
Line: 29

Found unverified result 🐷🔑❓
Detector Type: Github
Decoder Type: PLAIN
Raw result: -some-hash-here
File: .git/logs/HEAD
Line: 62

• Which is quite obviously a false positive.

[nvuillam]

Indeed that's strange...

For the moment I didn't find a way to ignore false positives in trufflehog, but I'll check again !

Meanwhile you can just disable errors by adding

DISABLE_ERRORS_LINTERS:
  - REPOSITORY_TRUFFLEHOG

[iggy]

Not sure if this is useful (at least it's not a 👍). Got the same except my raw result line is different. Here is a public repo run if it helps https://github.com/iggy/govern/actions/runs/5663001741/job/15343965208?pr=121
Raw result: ***

[dochang]

This is my workaround:

REPOSITORY_TRUFFLEHOG_ARGUMENTS: "--only-verified"

[quotidian-ennui]

DISABLE_ERRORS_LINTERS:
  - REPOSITORY_TRUFFLEHOG

I did this after I looked at your release notes, and saw the trufflehog was new in 7.2.

The issue is of course that it doesn't happen when you run trufflehog with no additional args manually after installing it on your local machine; so developers could never have remediated this.

If you look at the trace logging coming out from something like

• trufflehog --trace filesystem . 2>tlog1.txt (which I guess is how megalinter must run it, since it is scanning .git)
• trufflehog --trace git ssh://git@github.com/the/same/repo 2>tlog2.txt then you can see the difference in operation modes.
• trufflehog --trace git file://. --since-commit main seems like it's doing the right thing by checking the commits

I'm personally not entirely convinced that trufflehog is doing anything useful in the context of

2023-07-26T08:26:12+01:00	info-3	trufflehog	scanning file	{"source_type": "SOURCE_TYPE_FILESYSTEM", "source_name": "filesystem", "path": ".git\\objects\\ff\\a6331a22476f7c96d65d4785d553df773b9594"}

If I'm running megalinter on pull requests then I should be running trufflehog --trace git file://. --since-commit main?

.git could be added to ADDITIONAL_EXCLUDED_DIRECTORIES; but I feel that has unknowable downstream impact as a global flag so perhaps you might want to think about changing your commandline for execution to be something like this

bsh ❯ echo .git >th-exclude
bsh ❯ trufflehog --trace filesystem . --exclude-paths=./th-exclude 2> exclude.txt

That's a different workaround to the the --only-verified, potentially more lenient than you want, but it doesn't feel like trufflehog is actually parsing git history when you run it in filesystem mode.

Making that change would make the default behaviour out-of-the-box more friendly, but leave you in a state of having to detect if someone has included --exclude-paths in their commandline override.

which leaves us with 2 potential workarounds (horses for courses here, but some additional documentation in megalinter is probably useful, since all of these are sane changes that any user might want to make).

• If you want a more general workaround as per @dochang REPOSITORY_TRUFFLEHOG_ARGUMENTS: "--only-verified"

• Add an exclusion file that explicitly ignores the .git directory and then use REPOSITORY_TRUFFLEHOG_ARGUMENTS: --exclude-paths=path/to/exclusion

• I'd like to be able to do REPOSITORY_TRUFFLEHOG_ARGUMENTS: "git file://. --since-commit main" but this is broken because of how megalinter is running trufflehog.

[nvuillam]

Thanks to all of you for your analysis, I'll publish a patch with --only-verified called by default :)

[nvuillam]

Fix released in https://github.com/oxsecurity/megalinter/releases/tag/v7.2.1 :)