Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Document whether scorecard should be used as a requirement for organizations consuming OSS #4219

Open
sudo-bmitch opened this issue Jul 5, 2024 · 2 comments · May be fixed by #4318
Assignees
Labels
kind/docs Improvements or additions to documentation

Comments

@sudo-bmitch
Copy link

Is your feature request related to a problem? Please describe.

There have been requests on whether companies should require a scorecard score, and what the score should be, of the OSS software they consume.

Describe the solution you'd like

Add a FAQ entry or clearly identify on the main readme whether scorecard should be used this way, and if so, what a minimum required score should be. If it should not be used that way, describe why the maintainers recommend against it.

Describe alternatives you've considered

Repeated answers in slack threads.

Additional context

N/A.

@sudo-bmitch sudo-bmitch added the kind/enhancement New feature or request label Jul 5, 2024
@spencerschrock
Copy link
Contributor

There have been requests on whether companies should require a scorecard score, and what the score should be, of the OSS software they consume.

I think the answer should be no for a few reasons:

  1. With 19 checks, there's multiple different ways to arrive at score X/10.
  2. Scores change as we add new heuristics
  3. Every OSS consumer has their own risk tolerances
  4. Scorecard isn't perfect at detecting behaviors

I think a more reasonable question is: "Should OSS consumers require certain behaviors?"

With the v5 release, we see Structured Results as a way of doing this. Instead of relying on an overall Scorecard of X, or a Maintained score of Y, an OSS consumer may want to ensure the repo they're depending on isn't archived (which is covered by the archived probe).

@spencerschrock spencerschrock added kind/docs Improvements or additions to documentation and removed kind/enhancement New feature or request labels Jul 19, 2024
@sudo-bmitch
Copy link
Author

I tend to agree that Scorecard should not be used by consumers. An example attack I can think of would be to find popular projects with low scorecard scores, perhaps because maintainers use tools that are not recognized by the scanner, don't have co-maintainers, or similar challenges. The attacker could fork the project, use sock puppets for co-maintainers, and add files that trick the scorecard scanner to give a false positive score (e.g. an empty fuzzing test). Then with a higher scorecard score, they could go around with other sock puppet accounts and force projects to change their dependency to the malicious attacker controlled project citing the better scorecard score.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/docs Improvements or additions to documentation
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

2 participants