OSSF Scorecards

[Molkree]

What do you think about using Open Source Security Foundation' Scorecards (repo)?
They check quite a long list of things, including branch protection rules, fuzzing, pinned dependencies, signed releases, etc (list of checks).
There is a GitHub Action.
I found out about this project from the recent GitHub blog post announcing V4 and GitHub Action.

[anthonyharrison]

@Molkree I think this is a good idea but @terriko is already looking at how the tool can be made OSSF compliant. There are a number of features which aren't currently done, including some of the new features. Maybe this would make a suitable project for GSOC?

[terriko]

I'm also on the openSSF mailing lists and learned about the new scorecard stuff! I think yes, this is a thing we want to do eventually. I'm not sure we're actually ready for it yet, or maybe it's more accurate to say that I haven't gotten to that step in the process yet.

Currently, I'm working on the best practices badge (was CII, now OpenSSF best practices): https://bestpractices.coreinfrastructure.org/en/projects/5380

To finish that I still need to

• Get CI: Add bandit to pre-commit (fixes #1110) #1523 merged so we're scanning using bandit on every commit (It's waiting on code review from someone who isn't me)
• Possibly get us in an automated setup with scan.coverity.com (I've got us set up with manual scans right now, so it's just CI enablement left)
• Add dynamic code analysis. This is likely going to be fuzzing. I may apply to see if we qualify for OSS Fuzz, if not there's plenty of other options.

I want the best practices badge at 100% first before we start enabling the scorecard, but I'm happy to get it set up after that. If anyone wants to help get us to 100%, setting up weekly fuzzing runs in CI would be a good place to contribute right now.

Not sure that this it a viable gsoc project since it looks like a lot of these tasks are going to have to be done by someone with admin access to the repo, but I'd be willing to entertain a project idea if someone can clearly divide out the parts that can be done by someone who isn't me.

[terriko]

And yes, I realize that you don't have to have the Best Practices Badge in order to enable the scorecard, this is just a priority list for me.

[terriko]

Update here: when we do the next release we will have completed the basic practices badge (I'm waiting on release so we can say that we did the fuzzing pre-release). I'm going to tag this with the "future" milestone so we can look into doing more after 3.2 is out.

[terriko]

Update: We've finished the "passing" basic badge and 3.2 is out, so it's probably time to work on enabling the automatic scorecard. We may also want to look into the "silver" or "gold" levels.

[terriko]

I've done a quick pass on the next two levels in case anyone's interested:

silver: https://bestpractices.coreinfrastructure.org/en/projects/5380?criteria_level=1
gold: https://bestpractices.coreinfrastructure.org/en/projects/5380?criteria_level=2

We're around 3/4 of the way there even on those. Some of the remaining issues are governance-type stuff that can really only be done by me, but some is things like handling warnings, improving test coverage, verifying that urllib3/requests are using only good crypto algorithms, looking at accessibility issues in cve-bin-tool, etc. and could be done by anyone who wants to spend the time. If anyone works on any of those and wants me to update the score card, let me know.