✨ Give low importance to github-owned actions #906

nanikjava · 2021-08-25T10:45:15Z

#802

Please check if the PR fulfills these requirements

Tests for the changes have been added (for bug fixes / features)
PR title follows the guidelines defined in https://github.com/ossf/scorecard/blob/main/CONTRIBUTING.md#pr-process

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

feature update

Other information:
close Give low importance to github-owned actions #802

laurentsimon · 2021-08-25T16:49:14Z

checks/pinned_dependencies.go

 for jobName, job := range workflow.Jobs {
 if len(job.Name) > 0 {
 jobName = job.Name
 }
 for _, step := range job.Steps {
 if len(step.Uses) > 0 {
+ // check whether we have github related action


nit: start comments with capital letter and ends with full dot.

laurentsimon · 2021-08-25T16:49:48Z

checks/pinned_dependencies.go

 for jobName, job := range workflow.Jobs {
 if len(job.Name) > 0 {
 jobName = job.Name
 }
 for _, step := range job.Steps {
 if len(step.Uses) > 0 {
+ // check whether we have github related action
+ // either action/ or github/
+ ghaction := regexp.MustCompile(`(actions/)`)


do we need a regex or can we simplify by using strings.HasPrefix()?

laurentsimon · 2021-08-25T16:50:28Z

checks/pinned_dependencies.go

 for jobName, job := range workflow.Jobs {
 if len(job.Name) > 0 {
 jobName = job.Name
 }
 for _, step := range job.Steps {
 if len(step.Uses) > 0 {
+ // check whether we have github related action
+ // either action/ or github/
+ ghaction := regexp.MustCompile(`(actions/)`)


create a dedicated function, maybe isGitHubOwnedAction()

laurentsimon · 2021-08-25T17:01:50Z

checks/pinned_dependencies.go

@@ -534,7 +547,7 @@ func validateGitHubActionWorkflow(pathfn string, content []byte,
 }
 }

- addPinnedResult(pdata, ret)


the dependency check is the most complicated to work with. I think what you need is the following:

create new structure to store workflow pinned results, something like

type worklowPinningResult struct { thirdParties pinnedResult gitHubOwned pinnedResult }

Create a function addPinnedWorkflowResult() to use for this case (here)

You'll need a dataAsWorkflowResultPointer() to replace dataAsResultPointer() for this use case (here)

Update createReturnForIsGitHubActionsWorkflowPinned to calculate the score as we discussed earlier.

Let me know if this clarifies things.

Thanks again for taking the time to contribute!

Thank you @laurentsimon for the comments and pointer on what to do.

I've updated the PR based from my understanding after reading your comments. Kindly give me your feedback 👍

laurentsimon

once you've updated your code, add some unit tests to verify your code does what we expect it to do.

laurentsimon · 2021-08-26T16:19:17Z

checks/pinned_dependencies.go

@@ -52,6 +51,13 @@ type gitHubActionWorkflowConfig struct {
 Name string `yaml:"name"`
 }

+// Structure to host information about pinned github
+// or third party dependencies


nit: end comment with full stop.

laurentsimon · 2021-08-26T16:19:48Z

checks/pinned_dependencies.go

@@ -153,6 +159,16 @@ func dataAsResultPointer(data FileCbData) *pinnedResult {
 return pdata
 }

+func dataAsWorkflowPointer(data FileCbData) *worklowPinningResult {


nitL rename to dataAsWorkflowResultPointer

laurentsimon · 2021-08-26T16:21:37Z

checks/pinned_dependencies.go

-func createReturnForIsGitHubActionsWorkflowPinned(r pinnedResult, dl checker.DetailLogger, err error) (int, error) {
- return createReturnValues(r,
+func createReturnForIsGitHubActionsWorkflowPinned(r worklowPinningResult, dl checker.DetailLogger, err error) (int, error) {
+ return createReturnValues(r.gitHubOwned,


you need a new function to return the values. The function needs to look at both gitHubOwned and thirdParties to calculate the score.

laurentsimon · 2021-08-26T16:23:23Z

checks/pinned_dependencies.go


 if !CheckFileContainsCommands(content, "#") {
- addPinnedResult(pdata, true)
+ addPinnedResult(&pdata.gitHubOwned, true)


you need a new function addWorkflowPinnedResult that takes * worklowPinningResult as input. You can access and set gitHubOwned and thirdParties accordingly inside that function.

laurentsimon · 2021-08-26T16:24:28Z

checks/pinned_dependencies.go

+ mgithub := ghgithub.Match([]byte(step.Uses))
+
+
+ if (!maction) && (!mgithub) {


this is where you should call addWorkflowPinnedResult to set gitHubOwned and/or thirdParties

nanikjava · 2021-08-27T10:59:18Z

Thank you @laurentsimon

I've updated the code based on your feedback but I'm still uncertain about the following:

If in the workflow file there is following uses

uses: actions/checkout@daadedc81d5f9d3c06d2c92f49202a3cc2b919ba
uses: github/codeql-action/init@v2

does this mean that the gitHubOwned will have value pinned because one of the uses has a full hash ?

If in the workflow file there is following uses

uses: no-github/checkout@daadedc81d5f9d3c06d2c92f49202a3cc2b919ba
uses: no-github/codeql-action/init@v2

does this mean that the thirdParties will have value pinned because one of the uses has a full hash ?

Added a separate function to do the calculation but unsure if this is the correct way to calculate ? can you please let me know.

Apology for going back and forth as I'm still trying to nail the logic down and understand it properly.

Once I get the logic correct will add more test cases and fix any broken fix test cases.

laurentsimon

we're in the right direction. Add the unit tests and I'll review everything.
Note: I'm OOO from 01-Sept to 07-Sept so expect some delays for me to LGTM the PR.
Sorry about that!

laurentsimon · 2021-08-30T17:47:44Z

checks/pinned_dependencies.go

+ return checker.InconclusiveResultScore, err
+ }
+
+ score := 0


initialize to checker.MinResultScore instead of 0

laurentsimon · 2021-08-30T17:50:11Z

checks/pinned_dependencies.go

+ switch r.gitHubOwned {
+ case pinned:
+ score += 2
+ case notPinned:


this case not needed since checker.MinResultScore is 0. I think we can simplify the switch and use a single if as:

if r.gitHubOwned != notPinned { score += 2 }

his will also take care of the case when r.gitHubOwned is pinnedUndefined which means we found no GitHub-owned actions in the workflow.

laurentsimon · 2021-08-30T17:50:25Z

checks/pinned_dependencies.go

+ score += checker.MinResultScore
+ }
+
+ switch r.thirdParties {


same comment as above.

laurentsimon · 2021-08-30T17:59:00Z

checks/pinned_dependencies.go

+ maction := strings.HasPrefix(step.Uses, "actions/")
+ mgithub := strings.HasPrefix(step.Uses, "github/")
+
+ if (match) && ((maction) || (mgithub)) {


We can simplify addWorkflowPinnedResult by adding 1 additionalbool: isGitHub. The function can then internally call addPinnedResult(). Something like:

githubOwned := isGitHubOwnedAction(step.Uses) addWorkflowPinnedResult(pdata, match, githubOwned)

func addWorkflowPinnedResult(r * worklowPinningResult, to, isGitHub bool) { if isGitHub { addPinnedResult(&r.gitHubOwned, to) } else { addPinnedResult(&r.thirdParties, to) }

laurentsimon · 2021-08-30T18:03:13Z

checks/pinned_dependencies.go

 return true, nil
 }

+func addWorkflowPinnedResult(w *pinnedResult, to bool) {


see comment above to simplify

nanikjava · 2021-08-31T12:44:26Z

@laurentsimon PR has been modified per your suggestion and test has been added. Thank you.

laurentsimon

Thank you for the hard work, this is great!
I think the next PR will be perfect and we can merge. I'm OOO until 07 Sept so expect some delay for my last LGTM. Sorry about that.

laurentsimon · 2021-08-31T17:47:28Z

checks/pinned_dependencies.go

+ score += 8
+ }
+
+ if r.gitHubOwned == pinned {


move the Info() inside the if r.gitHubOwned != notPinned.
Also add a call to Info() inside the if r.thirdParties != notPinned.

Use a different message for the call to Call(), and use Info3() as in https://github.com/ossf/scorecard/blob/main/checks/permissions.go#L66. Say

Info3(&checker.LogMessage{ Path: PATH2FILE, Type: checker.FileTypeSource, Offset: LINE_NO, Snippet: SNIPPET, Text: fmt.Sprintf("%s %s"), "GitHub-owned", infoMsg, }) Info3(&checker.LogMessage{ Path: PATH2FILE, Type: checker.FileTypeSource, Offset: LINE_NO, Snippet: SNIPPET, Text: fmt.Sprintf("%s %s"), "Third-party", infoMsg, })

If you're not sure how to get the line numbers and snippet, omit them and just add a //TODO: set Snippet and line numbers.

Last point: call createReturnValuesForGitHubActionsWorkflowPinned() with a generic message "actions are pinned"

laurentsimon · 2021-08-31T17:52:38Z

checks/pinned_dependencies.go

- return createReturnValues(r,
+func createReturnForIsGitHubActionsWorkflowPinned(r worklowPinningResult, dl checker.DetailLogger,
+ err error) (int, error) {
+ return createReturnValuesForGitHubActionsWorkflowPinned(r,
 "GitHub actions are pinned",


update this message as I said in comment above.

laurentsimon · 2021-08-31T17:54:13Z

checks/testdata/workflow-non-github-not-pinned.yaml

@@ -0,0 +1,48 @@
+# Copyright 2021 Security Scorecard Authors


should this file be renamed to workflow-github-pinned.yaml since it only contains pinned GitHub-owned actions?

laurentsimon · 2021-08-31T17:57:23Z

checks/testdata/workflow-non-github-not-pinned.yaml

@@ -0,0 +1,48 @@
+# Copyright 2021 Security Scorecard Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");


please add additional unit tests:

mix of pinned and non-pinned GitHub actions. score should be 8

mixed of pinned and non-pinned non-GitHub dependencies. score should be 2

mixed of the above. score should be 0

all github and 3p actions pinned. score should be 10. For this one, you can simply update an existing workflow instead of creating a new file. Or you can create a new file: up to you.

laurentsimon · 2021-08-31T18:00:30Z

@inferno-chromium please LGTM and merge this PR once the changes above have been made. Otherwise it will have to wait until I'm back from OOO on 08 Sept.

nanikjava · 2021-09-05T07:59:57Z

@inferno-chromium please LGTM and merge this PR once the changes above have been made. Otherwise it will have to wait until I'm back from OOO on 08 Sept.

@inferno-chromium PR has been updated based on @laurentsimon feedback. Thanks.

The build/license boilerplate check (pull_request) failed due to some issue with protoc, don't have permission to re-run it.

checks/pinned_dependencies.go

laurentsimon

LGTM. I Left a single comment in the unit test. Just address it and update your branch.
I'll check what''s wrong with the protobuf issue.

checks/pinned_dependencies_test.go

laurentsimon · 2021-09-08T23:15:34Z

the protobuf problem in the check may be due to your branch being out-of-date. The problem was solved recently and hopefully will go away when you update your fork.

* Different calculation between github and non-github actions * Add test case for different kind of github and non-github action * Modify existing test as score calculation has changed

nanikjava changed the title ~~wip: fix for #802~~ [WIP]: fix for #802 Aug 25, 2021

laurentsimon changed the title ~~[WIP]: fix for #802~~ [WIP]: Give low importance to github-owned actions Aug 25, 2021

laurentsimon reviewed Aug 25, 2021

View reviewed changes

nanikjava force-pushed the f-fix-802 branch from 05d3cc2 to e05fc7f Compare August 26, 2021 08:50

laurentsimon reviewed Aug 26, 2021

View reviewed changes

nanikjava force-pushed the f-fix-802 branch from e05fc7f to 8350946 Compare August 27, 2021 10:57

nanikjava requested a review from laurentsimon August 27, 2021 23:36

laurentsimon reviewed Aug 30, 2021

View reviewed changes

nanikjava force-pushed the f-fix-802 branch from 8350946 to 52142d3 Compare August 31, 2021 11:59

nanikjava changed the title ~~[WIP]: Give low importance to github-owned actions~~ Give low importance to github-owned actions Aug 31, 2021

nanikjava changed the title ~~Give low importance to github-owned actions~~ ✨ Give low importance to github-owned actions Aug 31, 2021

nanikjava force-pushed the f-fix-802 branch 2 times, most recently from 35fe3bc to 552949c Compare August 31, 2021 12:30

laurentsimon reviewed Aug 31, 2021

View reviewed changes

laurentsimon requested a review from inferno-chromium August 31, 2021 17:59

nanikjava force-pushed the f-fix-802 branch 2 times, most recently from 63b9117 to d007f73 Compare September 5, 2021 07:47

nanikjava requested a review from laurentsimon September 5, 2021 08:00

naveensrinivasan reviewed Sep 7, 2021

View reviewed changes

checks/pinned_dependencies.go Show resolved Hide resolved

laurentsimon approved these changes Sep 8, 2021

View reviewed changes

checks/pinned_dependencies_test.go Outdated Show resolved Hide resolved

nanikjava force-pushed the f-fix-802 branch from d007f73 to 7021cfe Compare September 9, 2021 09:17

✨ Give low importance to github-owned actions (ossf#802)

8ff671c

* Different calculation between github and non-github actions * Add test case for different kind of github and non-github action * Modify existing test as score calculation has changed

nanikjava force-pushed the f-fix-802 branch from 7021cfe to 8ff671c Compare September 9, 2021 12:56

laurentsimon merged commit 1da121d into ossf:main Sep 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Give low importance to github-owned actions #906

✨ Give low importance to github-owned actions #906

nanikjava commented Aug 25, 2021 •

edited

Loading

laurentsimon Aug 25, 2021

laurentsimon Aug 25, 2021

laurentsimon Aug 25, 2021

laurentsimon Aug 25, 2021

nanikjava Aug 26, 2021

laurentsimon left a comment

laurentsimon Aug 26, 2021

laurentsimon Aug 26, 2021

laurentsimon Aug 26, 2021

laurentsimon Aug 26, 2021

laurentsimon Aug 26, 2021

nanikjava commented Aug 27, 2021 •

edited

Loading

laurentsimon left a comment

laurentsimon Aug 30, 2021

laurentsimon Aug 30, 2021

laurentsimon Aug 30, 2021

laurentsimon Aug 30, 2021

laurentsimon Aug 30, 2021

nanikjava commented Aug 31, 2021 •

edited

Loading

laurentsimon left a comment

laurentsimon Aug 31, 2021

laurentsimon Aug 31, 2021

laurentsimon Aug 31, 2021

laurentsimon Aug 31, 2021

laurentsimon commented Aug 31, 2021

nanikjava commented Sep 5, 2021

laurentsimon left a comment

laurentsimon commented Sep 8, 2021

		mgithub := ghgithub.Match([]byte(step.Uses))


		if (!maction) && (!mgithub) {

✨ Give low importance to github-owned actions #906

✨ Give low importance to github-owned actions #906

Conversation

nanikjava commented Aug 25, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

laurentsimon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nanikjava commented Aug 27, 2021 • edited Loading

laurentsimon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nanikjava commented Aug 31, 2021 • edited Loading

laurentsimon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

laurentsimon commented Aug 31, 2021

nanikjava commented Sep 5, 2021

laurentsimon left a comment

Choose a reason for hiding this comment

laurentsimon commented Sep 8, 2021

nanikjava commented Aug 25, 2021 •

edited

Loading

nanikjava commented Aug 27, 2021 •

edited

Loading

nanikjava commented Aug 31, 2021 •

edited

Loading