Hosting instrumenta/conftest under OPA

[tsandall]

The https://github.com/instrumenta/conftest project was started by @garethr to provide a simple way to test/validate configuration files using OPA/Rego. In the last year, Conftest adoption has grown and the project now has multiple active contributors. Conftest is licensed under ALv2.

Conftest implements several interesting ideas/features on top of OPA. After chatting with @garethr and the other OPA maintainers we felt it made sense to host Conftest as an OPA subproject. Over time, we also expect to extend core OPA with features initially implemented inside of Conftest. For example:

• Support for parsing of new file formats
• Support for additional bundle download protocols
• Support for an opinionated runner for testing data/config

For more information on the thought process here, see this document.

To move forward we need to:

• Hold a vote amongst the OPA maintainers (the process is outlined in the GOVERNANCE.md file)
• Create a conftest-maintainers team on GitHub and add @garethr @Blokje5 @jpreese @boranx and @tsandall
• Transfer the instrumenta/conftest repo into the open-policy-agent organization
• Ensure that DCO check is enabled on the repo
• Transfer ownership of the conftest.dev domain to the Linux Foundation (@garethr can you work with @caniszczyk on this?)
• Update the Conftest Code Of Conduct to use the CNCF's Code Of Conduct (they're based on the same original document AFAIK)
• Add a security notice in Conftest pointing back to SECURITY.md

In the short-term @garethr will manage the project infrastructure dependencies (currently these are: Netlify, CircleCI, and GolangCI.) Eventually we will migrate to shared accounts.

@garethr can you confirm that there are no registered trademark or logo assets for Conftest?

@ritazh and @maxsmythe PTAL and share any thougths/comments and indicate your vote in a comment.

[garethr]

I can confirm there are no registered trademark or logo assets for Conftest.

Just to confirm publicly as well that we've spoken about this and I'm definitely supportive. Conftest is increasingly popular, and it forming part of the main Open Policy Agent project makes lots of sense to me and hopefully to current and future users.

[caniszczyk]

Awesome, let me know when you want to do the domain transfer dance.

I'm excited to see this happen.

[ritazh]

+1 🎉

Excited to see the move!

[maxsmythe]

+1 I'm unable to comment on the doc, so adding a +1 here :) It is indeed exciting! I do have some conceptual nits on the document (nothing that affects the overall decision, IMO) - "Conftest produces pass/fail results for data": this is not strictly true, as in the doc there are at least 3 states: pass/fail/warn, which impacts this statement: "does not conflict with OPA's domain-agnosticism", as different result verbs may make sense in different domains. OPA itself makes no assumptions on rule names. Adding explicit results creates a coupling (albeit a loose one), with being run on a platform where the available rules make sense. - "OPA is intended to be able to consume arbitrary hierarchical structured data" this may be true in sentiment, but currently everything it ingests must be expressible as JSON, as that is the internal storage format OPA uses. I'm not sure how much difference this distinction makes in a practical sense as I'm unsure if there are data representations that don't meet this criteria. Ironically JSON schema cannot be processed by OPA as $ref references can be recursive, but is JSON schema a storage format in and of itself? Or is it JSON + some third party code that gives it meaning? Also "$ref" might break the "hierarchical" qualifier. None of this changes the sentiment that "adding additional representations that are expressible as JSON" is consistent with OPA's goals. Like I said, none of this affects whether integration is a good idea, but the first bullet in particular may be useful in figuring out what exactly that integration looks like.

…

-Max

On Fri, May 8, 2020 at 2:42 PM Rita Zhang ***@***.***> wrote: +1 🎉 Excited to see the move! — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#2387 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNYNZUM3T6ADZA3GLLM3MDRQR4CPANCNFSM4M4BONIA> .

[timothyhinrichs]

+1 from me too. Looking forward to making this happen!

…

On Fri, May 8, 2020 at 4:03 PM Max Smythe ***@***.***> wrote: +1 I'm unable to comment on the doc, so adding a +1 here :) It is indeed exciting! I do have some conceptual nits on the document (nothing that affects the overall decision, IMO) - "Conftest produces pass/fail results for data": this is not strictly true, as in the doc there are at least 3 states: pass/fail/warn, which impacts this statement: "does not conflict with OPA's domain-agnosticism", as different result verbs may make sense in different domains. OPA itself makes no assumptions on rule names. Adding explicit results creates a coupling (albeit a loose one), with being run on a platform where the available rules make sense. - "OPA is intended to be able to consume arbitrary hierarchical structured data" this may be true in sentiment, but currently everything it ingests must be expressible as JSON, as that is the internal storage format OPA uses. I'm not sure how much difference this distinction makes in a practical sense as I'm unsure if there are data representations that don't meet this criteria. Ironically JSON schema cannot be processed by OPA as $ref references can be recursive, but is JSON schema a storage format in and of itself? Or is it JSON + some third party code that gives it meaning? Also "$ref" might break the "hierarchical" qualifier. None of this changes the sentiment that "adding additional representations that are expressible as JSON" is consistent with OPA's goals. Like I said, none of this affects whether integration is a good idea, but the first bullet in particular may be useful in figuring out what exactly that integration looks like. -Max On Fri, May 8, 2020 at 2:42 PM Rita Zhang ***@***.***> wrote: > +1 🎉 > > Excited to see the move! > > — > You are receiving this because you were assigned. > Reply to this email directly, view it on GitHub > < #2387 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/ABNYNZUM3T6ADZA3GLLM3MDRQR4CPANCNFSM4M4BONIA > > . > — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#2387 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACMVQK2SFL3FJGUF6VND5U3RQSFUFANCNFSM4M4BONIA> .

[garethr]

The last two items are in open-policy-agent/conftest#302, for the SECURITY and CODE_OF_CONDUCT files.

Note I think that means, based on https://github.com/open-policy-agent/opa/blob/master/SECURITY.md#membership-criteria, once we have the conftest-maintainers team setup some or all of us should be added to the security mailing list?

[garethr]

@caniszczyk I'll drop you a line on Slack.

[garethr]

@Blokje5 @jpreese @boranx @tsandall one of thing, that I'm unsre about, is changing the package paths on Conftest. I'm not sure the implications or best approach to that, as part of moving the repository. Any thoughts or prior art?

[tsandall]

I'm not sure the implications or best approach to that, as part of moving the repository. Any thoughts or prior art?

When we do the repo transfer github.com/instrumenta/conftest will redirect to github.com/open-policy-agent/conftest. Git operations against github.com/instrumenta/conftest should continue to work though GitHub will return messages that tell you about the move.

I guess there are a few things to consider here:

1. If backwards compatibility is not a hard requirement, we could just transfer the repo, update the Go module and rewrite the package/import paths. I tried this out with github.com/ksubedi/gomove and it just worked (had to use tip of master from that tool). I was able to produce a build of conftest from scratch in a few minutes.

2. If we go ahead with (1) what's the migration path for library users? Specifically, how do they upgrade to new versions of Conftest? It doesn't seem like Go modules replace works. I tried this out with a fork of conftest and ended up with this error after trying to upgrade (related: golang/go#26904):

$ go get -u
------8< SNIP 8<------
go: github.com/tsandall/test-conftest-transfer@v0.22.0 used for two different module paths (github.com/instrumenta/conftest and github.com/tsandall/test-conftest-transfer)

3. Alternatively we could proceed with (1) and just show how library users how to update manually. E.g., (1) update go.mod to point at github.com/open-policy-agent/conftest and (2) rewrite import paths from github.com/instrumenta/conftest to github.com/open-policy-agent/conftest. Depending on how many people have to do this and how many dependencies they have on exported symbols in conftest, this may just be the simplest approach.

4. OTOH, if backwards compatibility is a hard requirement, I suppose conftest could keep the github.com/instrumenta/conftest module name. I was curious whether this would just work so I did a small experiment. It seems to require replace in the dependent Go module. I'm of two minds here. Reducing the impact on users is nice--but requiring existing and new users rely on replace going forward feels suboptimal.

Go modules experiment

I have less experience with Go modules so I did a small experiment to see what would happen to users embedding Conftest as a library. It looks like go get operations on old versions will continue to work. For example, you should be able to add a dependency on github.com/tsandall/test-repo-transfer@v0.1.0 even though I have transferred it into the open-policy-agent organization:

# create a simple program that calls github.com/tsandall/test-repo-transfer/api.Foo, then...
go get github.com/tsandall/test-repo-transfer@v0.1.0
go mod vendor
go build -mod=vendor

However, versions created after the transfer are not accessible. I'm not sure there is a way around this:

$ go get -u github.com/tsandall/test-repo-transfer@v0.2.0
go get: github.com/tsandall/test-repo-transfer@v0.2.0/go.mod: verifying module: github.com/tsandall/test-repo-transfer@v0.2.0/go.mod: reading https://sum.golang.org/lookup/github.com/tsandall/test-repo-transfer@v0.2.0: 410 Gone
	server response: not found: github.com/tsandall/test-repo-transfer@v0.2.0: invalid version: unknown revision v0.2.0

To obtain the latest version, you can use the replace directive. Go will fetch the new version and your code will continue to work. You can see an example here:

replace github.com/tsandall/test-repo-transfer => github.com/open-policy-agent/test-repo-transfer v0.2.0

This approach seems to work however it's a bit more complicated for users embedding as a library since they need to use replace.

[jpreese]

@tsandall I would take approach #1 that you have outlined.

The consumers of conftest as a library are going to be small, and we don't expose many packages:

/policy
https://pkg.go.dev/github.com/instrumenta/conftest/policy?tab=importedby

/plugin (..and all packages under it)
https://pkg.go.dev/github.com/instrumenta/conftest/plugin?tab=importedby

/downloader
https://pkg.go.dev/github.com/instrumenta/conftest/downloader?tab=importedby

Maintaining backwards compatibility isn't a hard requirement for me in this case, and performing the move/updating paths is going to be the simplest approach.

[garethr]

I agree with @jpreese on this.

[tsandall]

@garethr @jpreese great! Transferring the repo and rewriting the import paths will be pretty simple (I think 🤞). The conftest-maintainers team has been created and everyone has been added. As soon as the repo transfer is opened, I'll accept it and associate the team to the repo.

[garethr]

I think the best approach is probably to prep a PR with the path changes, then move the repo and merge. I probably won't have time to put up the PR but I can test, and then change the various bits of infrastructure over when we tranfer.

[tsandall]

@garethr since I had the changes partially made on from last week during that experiment, I just went ahead and finished them off. PTAL.

[garethr]

I've transferred the repository. https://github.com/open-policy-agent/conftest

I'll ping Chris now about the domain name.

[garethr]

@tsandall for moving over the CircleCI build, I've requested CircleCI access to the open-policy-agent org. The admins for the org should have received an email.

[tsandall]

@garethr fantastic. I've approved the CircleCI app for the org and the DCO check is working. Last thing on the checklist is the domain transfer.

[tsandall]

@garethr @caniszczyk any update on the last outstanding item from the checklist (transfering domain ownership)?

[garethr]

@tsandall ah! I need to send Chris a code. I'll try and code next week.

[garethr]

I had to do some domain name juggling, but I've just send @caniszczyk the code.

[tsandall]

@caniszczyk has filed a support ticket with LF to get it done. Hopefully will finished soon.

[tsandall]

@garethr @caniszczyk any movement? I'd love to close this :D

[garethr]

I believe this is done? I think it's good to close.

[tsandall]

Excellent. 🎉