Unable to suppress the pem file during scan

[prathamesh199728]

• I'm submitting a ...
  As the project requirement we need to suppress the pem file for testing purpose. but no option allowed as such.

• What is the current behavior?
  We dont want to balckilist on file base using paremeters like --exclude-files --exclude-secrets because its hectic and developer can misuse by including sensitive data in this files. Its a compliance issue.

• What is the expected behavior?
  We want to use the --word-list flag for this requirement but its only working while single string in one line like AWS secrets. When we are trying to use this for PEM key and read it in output its considering every line of PEM key as new one hence it not suppressed.

Need some solutions for such cases.

[prathamesh199728]

Any update here team?

[lorenzodb1]

Hi @prathamesh199728, thank you for opening this issue!

I'm trying to understand what the use-case would be here. Based on

As the project requirement we need to suppress the pem file for testing purpose.

and

We dont want to balckilist on file base using paremeters like --exclude-files --exclude-secrets because its hectic and developer can misuse by including sensitive data in this files.

I suppose you'd like detect-secrets to check PEM files in prod, but not in your testing env? Otherwise, could you tell us a bit more about what you're trying to do? I admit I'm a bit confused at the moment.

[prathamesh199728]

For testing purpose some PEM key required which we don't want to highlight during scan. Whitelist this test pem and provide in some file using extension --exclude-files is not feasible solution for us because dev team can pass sensitive material other than PEM also.

To avoid this case we need to control those file at command level. Where regex pattern will work for each line of PEM. For example in regex pattern will mention first 2 line of PEM and pass using --exclude-secrets or using --words-list extension where will mention whole file. I tried this at my side but its not working. I am more prefer to go with --words-list extension.

[prathamesh199728]

--words-list this extension only working for single line secret and not for PEM file.

[lorenzodb1]

Have you considered splitting the secrets in two different files and excluding the one used specifically for testing purposes?

[prathamesh199728]

No I guess some confusion here.

I have PEM file with below content:

--pemfile--
ok I am test PEM data
----end----

Now I want to ignore above PEM file while scanning so I created one file same as wordlist and put above data in it

wordlist.txt

--pemfile--
ok I am test PEM data
----end----

I use this command now:-
detect-secrets scan --word-list wordlist.txt

But above command is not working because logic return in code is scan happened on the basis of each line not for whole content. So need any alternate solution to suppress test pem file content while scanning.

[lorenzodb1]

So you'd like to ignore all data that matches the content of that PEM file?

[prathamesh199728]

correct.

[lorenzodb1]

Sorry, I just would like to make sure I understood this correctly. You have a PEM file containing multiple keys and you'd like to ignore a specific one, thus you created a file containing the key that you ignored and want to pass that file to filter out those keys from the results. Correct?

[prathamesh199728]

Yes this is correct.

[lorenzodb1]

Would the keys you'd like to ignore be present in multiple files, or there'd be just one file containing those keys?

[prathamesh199728]

its multiple files (with same PEM file content) present in different projects and we integrated this tool in CI/CD pipeline so during git push all code get scanned.

[lorenzodb1]

Thank you for the clarification! We'll keep you posted on this issue.

[prathamesh199728]

@lorenzodb1 any update here?

[lorenzodb1]

@prathamesh199728 sorry for the delay in replying. We have accumulated a bit of a backlog and we're trying to get through it. You're more than welcome to open a PR and propose a solution yourself in the meantime :)