🐛 fix Docker remediations for unpinned GHA dependencies

[raboof]

What kind of change does this PR introduce?

Bug fix

• PR title follows the guidelines defined in our pull request documentation

What is the current behavior?

As both the check for unpinned dependencies in GitHub Actions and the check for unpinned Docker dependencies contribute to d.Dependencies, the loop that created remediations for Docker dependencies would also create try to create Docker remediations for the unpinned GitHub Actions dependencies.

This could get really slow, especially when scanning a repo with many GitHub Actions such as https://github.com/apache/beam.

What is the new behavior (if this is a feature change)?**

Only create Docker remediations for Docker dependencies (and similar for GitHub workflow remediations)

• Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

Improve Pinned-Dependencies remediation creation

[spencerschrock]

In the context of how the code works now, I think this change seems reasonable.

How did the speed-up look on your end? Is there still need for improvement, we can try to get rid of the quadratic behavior currently exhibited as the code iterates over dependencies multiple times, which for large projects like the ASF may take a lot of time still.

These functions are all called from

scorecard/checks/raw/pinned_dependencies.go

Lines 36 to 63 in 5447253

	func PinningDependencies(c *checker.CheckRequest) (checker.PinningDependenciesData, error) {
	var results checker.PinningDependenciesData
	// GitHub actions.
	if err := collectGitHubActionsWorkflowPinning(c, &results); err != nil {
	return checker.PinningDependenciesData{}, err
	}
	// // Docker files.
	if err := collectDockerfilePinning(c, &results); err != nil {
	return checker.PinningDependenciesData{}, err
	}
	// Docker downloads.
	if err := collectDockerfileInsecureDownloads(c, &results); err != nil {
	return checker.PinningDependenciesData{}, err
	}
	// Script downloads.
	if err := collectShellScriptInsecureDownloads(c, &results); err != nil {
	return checker.PinningDependenciesData{}, err
	}
	// Action script downloads.
	if err := collectGitHubWorkflowScriptInsecureDownloads(c, &results); err != nil {
	return checker.PinningDependenciesData{}, err
	}

So collectGitHubActionsWorkflowPinning identifies workflow deps W1, W2, and W3 via validateGitHubActionWorkflow and then cycles through them.

Then collectDockerfilePinning identifies docker images D1, D2, and D3, and via validateDockerfilesPinning, and then cycles through W1, W2, W3, AND D1, D2, and D3.

This continues with collectDockerfileInsecureDownloads, collectShellScriptInsecureDownloads, and collectGitHubWorkflowScriptInsecureDownloads.

This could be fixed by changing where we append to the the final dep slice, and iterating over temporary slices, but if its not worth the effort we don't need to do it yet.

[spencerschrock]

/scdiff generate Pinned-Dependencies

[github-actions]

Here's a link to the scdiff run

[raboof]

How did the speed-up look on your end?

scorecard --repo=https://github.com/apache/beam --checks Pinned-Dependencies went from 16 minutes to 24 seconds on my machine ;)

Is there still need for improvement

Faster is always better, and I haven't done any further profiling, but I suspect the bottlenecks will be elsewhere now.

This continues with collectDockerfileInsecureDownloads, collectShellScriptInsecureDownloads, and collectGitHubWorkflowScriptInsecureDownloads.

These don't iterate over the Dependencies, though, as far as I noticed

[spencerschrock]

scorecard --repo=https://github.com/apache/beam --checks Pinned-Dependencies went from 16 minutes to 24 seconds on my machine ;)

Fantastic. Thanks for catching this!

Faster is always better, and I haven't done any further profiling, but I suspect the bottlenecks will be elsewhere now.

This continues with collectDockerfileInsecureDownloads, collectShellScriptInsecureDownloads, and collectGitHubWorkflowScriptInsecureDownloads.

These don't iterate over the Dependencies, though, as far as I noticed

I did a quick and dirty test to avoid going over the GHA deps again in collectDockerfilePinning and got another 20% speedup: 24s to 20s on my machine forgithub.com/apache/beam, which is better but like you said probably no longer a bottleneck.

[spencerschrock]

/scdiff generate Pinned-Dependencies

[github-actions]

Here's a link to the scdiff run