Scorecards indicators in Thoth recommendations [Tracker]

[pacospace]

GOAL: Provide Code Review justifications for each dependency used
thoth-station/thamos#868

To include these indicators in the recommendations, these data needs to be collected for each pypi project linked to GitHub repo, in the best case this can be done per tag.

Steps

Part 1

• RFE: scorecard on tag/commit, Open tag feature upstream;
• document disclaimer about timely relation/close proximity of upstream release and scorecards master-only behavior;
• Decide which metrics to include for security recommendations, let's start with Security-Policy;
• New github token app to run scorecards (https://github.com/thoth-station/thoth-application/blob/7fd94e543c8df48611045c63e8deb4d38c332aee/baboon/base/secrets.enc.yaml#L2);
• Create/Take container image to run scorecards (similar to the docker one provided by upstream ;https://github.com/ossf/scorecard#docker);
• create a notebook that show how many project have source relation or not;
• Create logic to have candidate github URLs from solver document https://github.com/thoth-station/solver-project-url-job (done by @bjoernh2000 ;

Part 2

• 1. Create a new handler for Ceph for this new data;
• 2. Create new table to sync scorecards;
• 3. Create new sync logic in database to sync new data (update logic);
• 4. new thoth-storages release
• 5. create a new workflow ScoreCardAnalysis workflow with tasks:
  5.1. Run scorecards and get json data. ( ./scorecard --repo https://github.com/thoth-station/adviser --checks=CI-Tests,Code-Review --format json --show-details)
  5.2. graph sync score cards reports
• 6. create new common methods to schedule the new workflow;
• 7. new thoth-common release
• 8. Add step to solver workflow: Run solver project url job to find candidates URL from solver document;
• 9.create message in thoth-messaging ScoreCardAnalysisMessage
• 10. new thoth-messaging release
• 11. Add task in solver to trigger ScoreCardAnalysisMessage after URLs candidates are discovered (if any)
• 12. Add consumer and metrics to investigator for new message that is used to schedule ScoreCardAnalysis workflow
• 13. Add env variable to disable/enable this workflow in investigator.
• 14. new investigator release
• 15. Restart security indicators;

Part 3

• Create logic in adviser to provide justification for security recommendation that consider scorecards;

Enhancments: ossf/scorecard#575

[fridex]

What's the priority of this? CC @goern

[goern]

I created a new GitHub app https://github.com/apps/baboon-the-sourcerer and put the secrets at https://github.com/thoth-station/thoth-application/blob/master/baboon/base/secrets.enc.yaml

[goern]

/priority important-soon

[fridex]

I created a new GitHub app https://github.com/apps/baboon-the-sourcerer and put the secrets at https://github.com/thoth-station/thoth-application/blob/master/baboon/base/secrets.enc.yaml

Do we have a higher priority work than this?

[goern]

ja, there are 8 critical-urgent and 80 important-soon issues in our org.

[fridex]

ja, there are 8 critical-urgent and 80 important-soon issues in our org.

So why there is time spent on this? What is the value of this?

[goern]

good point, a) I'll keep an eye on the critical issues b) @riekrh asked if we can have a look at more security-related indicators bringing more Sec in DevOps. Given the software infrastructure we have for bandit-based SI, I put a soon'ish prior on it for us to explore how to add scorecard-related indicators. c) always feel free to re-prioritize, it's in important sentiment we should share!

[fridex]

IMHO computing and ingesting dependency information data should be prioritized now + have the data lifecycle management in prod done. That way we can keep up with rhel/ubi/fedora releases to support thoth users = we will be able to give recommendations on these runtime environments. Once this will be in place, adding additional information to the recommendations is a manner of evaluating suitability of data and eventually plugging in other data sources. Unless we have dependency data (which btw https://deps.dev/ do not have as of now) any other data are not directly valuable to recommendations we do.

[pacospace]

Related-To: ossf/scorecard#575

[sesheta]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

[sesheta]

@sesheta: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[goern]

/triage accepted

Let's put this for Q4 as a key result, syn?

[codificat]

The solution currently proposed is to use prescriptions for this. See a detailed explanation
here