go modules: Allowlist specific hashicorp modules #1595

oshoval · 2023-08-14T13:20:46Z

What this PR does / why we need it:
Due to [1] we need to make sure not to use BSL modules.

Luckily the current we use have not changed.
The ones that are not changed are SDK/API and general Go libraries.
"HashiCorp APIs, SDKs, and almost all other libraries will remain MPL 2.0." [1] [2]

This commit creates a github action which whitelists them.
Any other module of hashicorp will be rejected,
and will need to be manually examined if it uses MPL (or other non restrictive license) or BSL.

[1] https://www.hashicorp.com/blog/hashicorp-adopts-business-source-license
[2] cncf/foundation#617 (comment)

Special notes for your reviewer:

Release note:

None

oshoval · 2023-08-14T13:26:27Z

/cc @maya-r

maya-r · 2023-08-14T20:05:00Z

I wonder what happened to our fossa lanes, which seem to be deactivated for kubevirt/kubevirt, which served this purpose but more widely - they tested the licenses of all our modules. (However, I am not opposed to a list of allowed modules)

oshoval · 2023-08-15T11:04:23Z

Print all errors at once
Remove IFS because lines dont start / end with whitespace, and even if so, it won't affect

phoracek · 2023-08-23T11:41:33Z

.github/workflows/check_hashicorp_modules.yaml

+            module=$(echo "$line" | cut -d ' ' -f 1)
+            if [[ $module == github.com/hashicorp/* ]]; then
+                if ! [[ " ${allowed_hashicorp_modules[*]} " == *" $module "* ]]; then
+                    echo "found non whitelisted hashicorp module: $module"


Suggested change

echo "found non whitelisted hashicorp module: $module"

echo "found non allowlisted hashicorp module: $module"

phoracek · 2023-08-23T11:42:35Z

.github/workflows/check_hashicorp_modules.yaml

+        done < go.sum
+
+        if [[ $error_found == true ]]; then
+            echo "non whitelisted hashicorp modules found. exiting with error"


Suggested change

echo "non whitelisted hashicorp modules found. exiting with error"

echo "non 03589504 hashicorp modules found. exiting with error"

Please include reasoning why we cannot allow any unlisted modules. And also what should be done to resolve the issue (review the licensing and either add it to the list or use a different library)

It is in the commit message, self explanatory isn't it ?
Will add if you prefer of course.

This commit creates a github action which allowlists them. Any other module of hashicorp will be rejected, and will need to be manually examined if it uses MPL (or other non restrictive license) or BSL.

When the check fails on my PR, i will not be looking for the PR that added the check. I would like the message on my PR to tell me what I should do to get it merged

phoracek · 2023-08-23T11:43:00Z

.github/workflows/check_hashicorp_modules.yaml

+            echo "non whitelisted hashicorp modules found. exiting with error"
+            exit 1
+        fi
+        echo "All hashicorp modules are whitelisted"


Suggested change

echo "All hashicorp modules are whitelisted"

echo "All included hashicorp modules are allowlisted"

Due to [1] We need to make sure not to use BSL modules. Luckily the current we use have not changed. The ones that are not changed are SDK/API and general Go libraries. "HashiCorp APIs, SDKs, and almost all other libraries will remain MPL 2.0." [1] [2] This commit creates a github action which allowlists them. Any other module of hashicorp will be rejected, and will need to be manually examined if it uses MPL (or other non restrictive license) or BSL. [1] https://www.hashicorp.com/blog/hashicorp-adopts-business-source-license [2] cncf/foundation#617 (comment) Signed-off-by: Or Shoval <oshoval@redhat.com>

oshoval · 2023-08-23T12:50:25Z

Thanks
Addressed comments

sonarcloud · 2023-08-23T12:50:41Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

kubevirt-bot · 2023-08-23T13:03:51Z

@oshoval: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-e2e-cnao-multus-dynamic-networks-functests	`6dd00f5`	link	false	`/test pull-e2e-cnao-multus-dynamic-networks-functests`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

phoracek

/lgtm
/approve

kubevirt-bot · 2023-08-23T13:54:11Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: phoracek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [phoracek]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

oshoval · 2023-08-24T06:29:06Z

Thanks,
Since we want to use it on all of our repos, should we just copy the file all over?
or

Have the other repos fetch the list from CNAO (the list will resides on a dedicated file)
Have CNAO run those checks when it clones the components during e2e, it will detect problems on component bumping, not on component pushes, but maybe it is good enough?

Not sure it worth it, but it will allow central changes, which is nice

phoracek · 2023-08-24T16:28:00Z

I would vote to just copy it. It would be simpler and it would keep those repos independent

oshoval · 2023-09-07T11:17:49Z

It seems MPL2 need to be in exception list [1], [2]

but we need to do so only for the direct dependencies (BenTheElder raised it, and also on [1])

still not sure about those, monitoring the decisions of kubevirt maintainers

[1] cncf/foundation#617 (comment)
[2] https://github.com/cncf/foundation/blob/main/license-exceptions/cncf-exceptions-2023-06-27.json

kubevirt-bot added release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Aug 14, 2023

kubevirt-bot requested review from RamLavi and rhrazdil August 14, 2023 13:20

kubevirt-bot added the size/M label Aug 14, 2023

oshoval force-pushed the hashicorp branch from 50bf3f1 to 745c651 Compare August 14, 2023 13:23

kubevirt-bot requested a review from maya-r August 14, 2023 13:26

oshoval force-pushed the hashicorp branch from 745c651 to 692661d Compare August 15, 2023 11:03

oshoval force-pushed the hashicorp branch from 692661d to dcc498f Compare August 15, 2023 11:15

oshoval mentioned this pull request Aug 15, 2023

Investigate MPL -> BUSL Changes/Impact cncf/foundation#617

Closed

oshoval changed the title ~~go mod: Whitelist specific hashicorp modules~~ go modules: Whitelist specific hashicorp modules Aug 15, 2023

phoracek requested changes Aug 23, 2023

View reviewed changes

kubevirt-bot assigned phoracek Aug 23, 2023

oshoval changed the title ~~go modules: Whitelist specific hashicorp modules~~ go modules: Allowlist specific hashicorp modules Aug 23, 2023

oshoval force-pushed the hashicorp branch from dcc498f to 6dd00f5 Compare August 23, 2023 12:50

phoracek approved these changes Aug 23, 2023

View reviewed changes

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Aug 23, 2023

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 23, 2023

kubevirt-bot merged commit de53805 into kubevirt:main Aug 23, 2023
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

go modules: Allowlist specific hashicorp modules #1595

go modules: Allowlist specific hashicorp modules #1595

oshoval commented Aug 14, 2023 •

edited

Loading

oshoval commented Aug 14, 2023

maya-r commented Aug 14, 2023 •

edited

Loading

oshoval commented Aug 15, 2023

phoracek Aug 23, 2023

phoracek Aug 23, 2023

oshoval Aug 23, 2023

phoracek Aug 23, 2023

oshoval Aug 23, 2023

phoracek Aug 23, 2023

oshoval commented Aug 23, 2023

sonarcloud bot commented Aug 23, 2023

kubevirt-bot commented Aug 23, 2023

phoracek left a comment

kubevirt-bot commented Aug 23, 2023

oshoval commented Aug 24, 2023

phoracek commented Aug 24, 2023

oshoval commented Sep 7, 2023

	echo "found non whitelisted hashicorp module: $module"
	echo "found non allowlisted hashicorp module: $module"

	echo "non whitelisted hashicorp modules found. exiting with error"
	echo "non 03589504 hashicorp modules found. exiting with error"

	echo "All hashicorp modules are whitelisted"
	echo "All included hashicorp modules are allowlisted"

go modules: Allowlist specific hashicorp modules #1595

go modules: Allowlist specific hashicorp modules #1595

Conversation

oshoval commented Aug 14, 2023 • edited Loading

oshoval commented Aug 14, 2023

maya-r commented Aug 14, 2023 • edited Loading

oshoval commented Aug 15, 2023

phoracek Aug 23, 2023

Choose a reason for hiding this comment

phoracek Aug 23, 2023

Choose a reason for hiding this comment

oshoval Aug 23, 2023

Choose a reason for hiding this comment

phoracek Aug 23, 2023

Choose a reason for hiding this comment

oshoval Aug 23, 2023

Choose a reason for hiding this comment

phoracek Aug 23, 2023

Choose a reason for hiding this comment

oshoval commented Aug 23, 2023

sonarcloud bot commented Aug 23, 2023

kubevirt-bot commented Aug 23, 2023

phoracek left a comment

Choose a reason for hiding this comment

kubevirt-bot commented Aug 23, 2023

oshoval commented Aug 24, 2023

phoracek commented Aug 24, 2023

oshoval commented Sep 7, 2023

oshoval commented Aug 14, 2023 •

edited

Loading

maya-r commented Aug 14, 2023 •

edited

Loading