Add .spec.version #142

perdasilva · 2023-03-28T20:35:37Z

Adds .spec.version property to the Operator CRD. Some small changes were made to the resolver and top level variable sources to accommodate for this change and, hopefully, future ones. When reviewing, we should be asking ourselves how easy it would be to add the next property.

This PR also adds a small admissions unit test, so we can validate the OpenApi validation rules we apply using the kubebuilder annotations.

I've also nerfed the stackdumps down to panic. Otherwise we just get logs pointing to internal controller-runtime code...

joelanford · 2023-03-29T17:52:53Z

api/v1alpha1/operator_types.go

@@ -27,6 +27,11 @@ type OperatorSpec struct {
 	//+kubebuilder:validation:MaxLength:=48
 	//+kubebuilder:validation:Pattern:=^[a-z0-9]+(-[a-z0-9]+)*$
 	PackageName string `json:"packageName"`
+
+	//+kubebuilder:validation:MaxLength:=64
+	//+kubebuilder:validation:Pattern:=^(\|\||\s+)?([\s~^><=]*)v?(\d+)(\.(\d+))?(\.(\d+))?(\-(.+))?$


Will this match the full set of (but not more than) version ranges supported by semver.ParseRange?

Hoping for two things:

We support the full syntax of semver.ParseRange

We block invalid ranges at admission time, and we never return an error from semver.ParseRange at reconcile time.

AFACT, this doesn't support the full syntax.

The semver package explicitly mentions no regex... Where did that one come from?

https://ihateregex.io/expr/semver/ then add the comparison operators

I believe the "official" semver regex is here: https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string

Either way, it's not correct as-is, and needs to account for the range prefix elements.

Well, based on the milestone 3 consensus in #162, it sounds like we're going with spec.version and a single explicit version for now. So I think we can literally use the linked regex?

Based on our upstream convo yesterday, I've left the name as "version" (I'm also happy with PackageVersion, if that's prefered - I personally like the succinctness of version. I've added some comments with examples and a link to the semver.org site.

api/v1alpha1/operator_types.go

joelanford · 2023-03-29T17:56:24Z

controllers/operator_controller_test.go

+			var pkgName string
+			BeforeEach(func() {
+				By("initializing cluster state")
+				pkgName = fmt.Sprintf("non-existent-%s", rand.String(6))


Should this be a package name that does exist, but one that doesn't have a bundle in this range?

I need to address this before we can merge. I'll come back to this in a bit. I need to pick up my son from footy camp =P

Gotcha now. I've addressed it. Changed the name of the package and put a comment to say the version of the package is doesn't exist

joelanford · 2023-03-29T18:05:49Z

I've also nerfed the stackdumps down to panic. Otherwise we just get logs pointing to internal controller-runtime code...

Typically those logs contain the error message our reconciler returns (if we return a non-nil error). Do those errors still come through, and we're just no longer also barfing out the stack trace? If so, 👍

Signed-off-by: perdasilva <perdasilva@redhat.com>

joelanford · 2023-04-11T18:06:42Z

Just as an update, we discussed the tradeoffs of versionRange, version, versionConstraint and -- for now -- landed on spec.version and it being optional and a single version (no range). The semantics are hopefully implicitly defined by the demo script in #162.

joelanford · 2023-04-12T12:36:32Z

internal/resolution/variable_sources/olm/olm.go

+func (o *OLMVariableSource) requiredPackageFromOperator(operator *operatorsv1alpha1.Operator) (*required_package.RequiredPackageVariableSource, error) {
+	var opts []required_package.RequiredPackageOption
+	if operator.Spec.Version != "" {
+		opts = append(opts, required_package.InVersionRange(operator.Spec.Version))


@perdasilva given the direction set out in #162, we're saying that only individual versions will be allowed in the API. However, an individual version is a valid range, and I am still of the belief that we'll want to give users the ability to specify a range at some point.

So my advice is:

Let's make the spec.version regex permit only individual versions

Let's keep InVersionRange as the under-the-hood implementation of the constraint.

Yeah, I agree with everything you said. The original regex was for a single version (got my wires crossed - managed to get a pretty decent one for ranges, but I don't think there's a regex that matches all cases - it could be that we'll need to match most but still verify under the hood. ChatGPT ftw XD

I've also nerfed the stackdumps down to panic. Otherwise we just get logs pointing to internal controller-runtime code...

Typically those logs contain the error message our reconciler returns (if we return a non-nil error). Do those errors still come through, and we're just no longer also barfing out the stack trace? If so, +1

Just to sanity check, I reverted the changes locally to test, before:

1.6813110527752454e+09 ERROR Reconciler error {"controller": "operator", "controllerGroup": "operators.operatorframework.io", "controllerKind": "Operator", "Operator": {"name":"operator-sample"}, "namespace": "", "name": "operator-sample", "reconcileID": "995fa1ab-8119-4059-9e3c-7ccfe06fbedf", "error": "package 'prometheus' at version '0.33.0' not found"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/perdasilva/repos/perdasilva/operator-controller/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:326 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/perdasilva/repos/perdasilva/operator-controller/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/perdasilva/repos/perdasilva/operator-controller/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:234

After:

1.6813111185853214e+09 ERROR Reconciler error {"controller": "operator", "controllerGroup": "operators.operatorframework.io", "controllerKind": "Operator", "Operator": {"name":"operator-sample"}, "namespace": "", "name": "operator-sample", "reconcileID": "c4b4e0ff-a639-440e-bc95-94d21675aca1", "error": "package 'prometheus' at version '0.33.0' not found"}

So, the error is still there. But, the one with the stacktrace isn't particularly helpful. Should I keep this change? We can always change it back later. Rn, this feels much cleaner.

… annotations Signed-off-by: perdasilva <perdasilva@redhat.com>

Signed-off-by: perdasilva <perdasilva@redhat.com>

perdasilva · 2023-04-13T10:05:53Z

I've gone ahead and made the semver validation more robust. I also decided to add validation in the reconciler in case the regex fails. I haven't managed to find a combination that does. But, as soon as we release someone will XD. I've made the validation additive, so we can just add new validation functions.

tmshort · 2023-04-13T13:43:12Z

api/v1alpha1/operator_types.go

+	//+kubebuilder:validation:MaxLength:=64
+	//+kubebuilder:validation:Pattern=^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(-(0|[1-9]\d*|[0-9]*[a-zA-Z-][0-9a-zA-Z-]*)(\.(0|[1-9]\d*|[0-9]*[a-zA-Z-][0-9a-zA-Z-]*))*)?(\+([0-9a-zA-Z-]+(\.[0-9a-zA-Z-]+)*))?$
+	//+kubebuilder:Optional
+	// Version is an optional is semver constraint on the package version. If not specified, the latest version available of the package will be installed.


Nit: grammar:
Version is an optional is semver constraint

tmshort · 2023-04-13T13:43:32Z

config/crd/bases/operators.operatorframework.io_operators.yaml

@@ -39,6 +39,16 @@ spec:
                maxLength: 48
                pattern: ^[a-z0-9]+(-[a-z0-9]+)*$
                type: string
+              version:
+                description: "Version is an optional is semver constraint on the package


Same grammar nit.

fixed ^^ nice catch. I also modified the "Example" because the rendering in the description looked funky

Signed-off-by: perdasilva <perdasilva@redhat.com>

m1kola

Still reviewing this one. Just a few questions so far

m1kola · 2023-04-13T16:01:14Z

controllers/validators/validators.go

+	}
+	for _, validator := range validators {
+		if err := validator(operator); err != nil {
+			return err


For now it is ok as there is only one field which requires validation. Just curious how you see UX for it once we have more validators - Will we bereturning one error at a time or use some sort of aggregated error (so we surface all issues at once)?

That's a great call out. Personally, I think we should try as much as possible to add all of the validation errors as conditions. It would suck to keep going back and forth trying to get it right. What I'm not sure of right now is if it's "acceptable" to add multiple conditions of the same type (I don't see why not, but I don't know what the best practice is here). According to my good friend ChatGPT:

It is generally best to keep the status of a Kubernetes resource concise and clear. If you have multiple conditions of the same type, it might be better to consolidate them into a single condition that provides an accurate representation of the overall state of the resource. Here are some guidelines to consider when working with conditions in the status of a Kubernetes resource: Each condition should have a distinct type that describes the aspect of the resource being observed. Conditions should be used to provide insights into the state of the resource and any issues that might be affecting it. Use the Reason and Message fields within a condition to provide more detailed information about the state of the resource and any issues. Update conditions as the state of the resource changes, ensuring that the status stays up-to-date and accurate. By following these guidelines, you can maintain a clear and concise status that communicates the state of the resource effectively. In cases where you need to represent multiple aspects of the resource's state, it's better to use separate conditions with different types. However, if you have multiple instances of the same type of condition, consider consolidating them into a single condition with a clear reason and message.

It seems to suggest we should try to group everything into a single condition. But that seems ugly at first sight. Another option from the top of my head might be to create a condition for each validation error. That also seems ugly. Personally, I'm in favor of just having multiple conditions of the same type each describing its own reason for failure.

I suggest we leave it as it is for now. I will add a comment and a ticket so we can follow up on this after a more thorough discussion upstream. wdyt?

I have definitevely seen conditions like ValidSomething true/false with message containing concatenated errors. I think it is very common in OCP. But I also seen condition per issue approach.

IMO - if condition is meant to be consumed by humans - we might just want to concatenate errors. We will later be able to split them into separate conditions. I personally not a fan of scrolling through 10s of conditions: I would rather see all validation errors in one place.

However if we want to programmatically consume these conditions or want to give our users this ability - then it will definitevely be easier to consume if we have conditions for each issue. But in this specific case I do not see a lot of value in this feature (but I'm happy to be proven wrong).

I suggest we leave it as it is for now. I will add a comment and a ticket so we can follow up on this after a more thorough discussion upstream. wdyt?

Agreed

The condition type needs to be unique in the list of conditions.

Type and reason are part of the API so once we GA, those can't be removed or changed.

My recommendation is generally to keep the number of types and reasons as small as possible based on what clients specifically require.

In this case, I'd start with whatever type we're already using, maybe even use the existing failure reason, and then put the validation errors concatenated in the message.

But +1 to capturing this in a separate issue.

m1kola · 2023-04-13T16:03:34Z

controllers/validators/validators_test.go

+	"github.com/operator-framework/operator-controller/controllers/validators"
+)
+
+var _ = Describe("Validators", func() {


Maybe I'm missing something - why not to use simple unit tests here? Looks like a classic use case for a table driven test.

I think we decided to go for ginkgo for the unit tests. However, I'm starting to think it might have been a mistake. They aren't as easy to execute as go tests, and carry the cruft of the *_suite.go files. From my perspective at the moment, I'm not a huge fan of table driven tests. I think they do have a nice additive property to them and can make it easier to change many tests at once. But from previous experience (which is just anecdotal at best) they carry a higher complexity in trying to understand whats going later on in the project. Especially if you have massive test case structures with huge inputs. On the other hand, singular tests are easier to wrap your head around later on. However, they can carry a higher maintenance burden during refactoring/maintenance. So, I find it to be a tradeoff between readability and and maintainability (which are also interrelated XD). Now with copilot and chatGPT it's easier to write singular tests hahahah. On a serious side, I'm easy either way. I'd say we should do the following:

I'll create a github discussion around this so we can decide what we should do (I think it's important to have some consistency)

Let's leave it as it is for now, make a join decision, and we can always refactor out ginkgo (at least for the unit tests) in favor of gotests and make a decision on whether we want to do tabular or singular tests and keep that decision as a compass bearing

wdyt?

Ginkgo supports table driven test fwiw. Maybe not quite as straightforward as a hand-written one, but maybe a reasonable compromise?

tmshort · 2023-04-13T13:57:37Z

controllers/validators/validators.go

+// this validation should already be happening at the CRD level. But, it depends
+// on a regex that could possibly fail to validate a valid SemVer. This is added as an
+// extra measure to ensure a valid spec before the CR is processed for resolution
+func validateSemver(operator *operatorsv1alpha1.Operator) error {


If the CRD regex validation fails, will the value even be set? So, is this necessary? Are you concerned about false positives?

For false positives. It's in case a bad semver somehow slips through the regex. It's so complicated and hard to parse I couldn't guarantee that nothing would slip through. So, I thought it might be a good idea to be defensive, even if 99.999% of the time this code won't probably be executed.

If the CRD regex validation fails, will the value even be set?

If I understood the question correctly (and please forgive me if I'm going over stuff you already know), my understanding of the way it works is: there's both client-side and server-side validation of the field value against the regex before it reaches the controller. The intent of putting that the validation is to automatically reject anything that isn't a semver so it doesn't even make it to the controller. But, after looking at the regex, I couldn't get any confidence that it would work 100% of the time. I tried to validate with different negative cases, but I couldn't convince myself that it will definitely catch everything. It could be possible that an invalid spec slips through, in which case it would be set. So, I thought it best to handle that case in code (even if it's unlikely that it will ever be executed). Hopefully, for most of the fields that come along in the API we'll be able to get away with relatively simple regexs and length limits that should be sufficient enough for us use of this controller validation sparingly.

Since I'm rubber ducking this with you now, it occured to me that should a false positive slip through, we'd get a resolution failure on trying to parse the semver. So, maybe I'm being overzealous? Though a part of me thinks that ensuring that the input is clean before reaching the resolver might be worthwhile.

I'm totally open for suggestions here. If we feel we're adding more complexity for relatively little gain, I can remove this 2nd layer of validation. wdyt?

It indeed feels like belt and braces, but I think I'm in favour of keeping this validation as it prevents us hittng rest of the code in case of discrepancies between the regex and the library (e.g. bugs in library, for example).

Assuming you got the regex from a reliable source, this scenario should never be hit, but better safe than sorry. Even regex's and libraries can have bugs.

tmshort · 2023-04-13T15:11:42Z

config/samples/operators_v1alpha1_operator.yaml

@@ -11,3 +11,4 @@ metadata:
 spec:
  # TODO(user): Add fields here
  packageName: prometheus
+  version: 3.0.0


Do we need/want to add a comment here that this is an invalid version? And should remain an invalid version if prometheus ever gets to version 3.0?

sorry - that shouldn't have been committed - I'll omit it.

tmshort · 2023-04-13T20:23:15Z

internal/resolution/variable_sources/required_package/required_package_test.go

+	It("should filter by version range", func() {
+		// recreate source with version range option
+		var err error
+		rpvs, err = required_package.NewRequiredPackage(packageName, required_package.InVersionRange(">=1.0.0 !1.0.0 <3.0.0"))


Is this valid range? i.e. include 1.0.0 then exclude it?

That's a fair point. I think it's a valid range, even if it's superfluous (just collapses to >1.0.0 <3.0.0. I'll change it to !2.0.0 to avoid this confusion.

Signed-off-by: perdasilva <perdasilva@redhat.com>

perdasilva · 2023-04-14T10:35:27Z

m1kola

From my very limited experience with v1 - looks good.

tmshort · 2023-04-14T13:42:29Z

The demo image...

Although it might have been easier to use a script with set -x

perdasilva · 2023-04-14T15:40:53Z

The demo image...

Although it might have been easier to use a script with set -x

Yeh, I did that originally - but it didn't look as nice =(

joelanford · 2023-04-18T15:34:12Z

I thing I didn't see in the demo (or maybe missed?) is what happens when you have an existing version already installed and set spec.version to a non-existent version (Items 10 and 11 from #162)?

Another thing we could show (and yes I realize this isn't technically in the scope of #162): what happens when trying to create or update the Operator with an spec.version that is not parseable semver?

perdasilva changed the title ~~Operator crd version~~ Add .spec.version Mar 28, 2023

joelanford reviewed Mar 29, 2023

View reviewed changes

api/v1alpha1/operator_types.go Show resolved Hide resolved

joelanford reviewed Mar 29, 2023

View reviewed changes

perdasilva added 4 commits April 11, 2023 16:13

change log stacktrace level to panic

bfd4b89

Signed-off-by: perdasilva <perdasilva@redhat.com>

add .spec.version field to Operator CRD

372b93f

Signed-off-by: perdasilva <perdasilva@redhat.com>

refactor variable sources and resolver for version range constraint

e9aa0c8

Signed-off-by: perdasilva <perdasilva@redhat.com>

update controller unit tests for .spec.version

0c044fa

Signed-off-by: perdasilva <perdasilva@redhat.com>

perdasilva force-pushed the operator_crd_version branch from a46c141 to a4ab6f7 Compare April 11, 2023 14:13

joelanford added this to the v0.1.0 (OLMv1 Milestone 3) milestone Apr 11, 2023

joelanford added the olm-v1/m3 label Apr 11, 2023

joelanford mentioned this pull request Apr 11, 2023

Honoring an optional spec.version field in the Operator API #162

Closed

1 task

joelanford reviewed Apr 12, 2023

View reviewed changes

perdasilva force-pushed the operator_crd_version branch from a4ab6f7 to c384984 Compare April 12, 2023 12:49

perdasilva added 2 commits April 12, 2023 16:45

add controller admission test to validate OpenAPI property constraint…

5eba278

… annotations Signed-off-by: perdasilva <perdasilva@redhat.com>

update go.mod - zap no longer indirect

d25be66

Signed-off-by: perdasilva <perdasilva@redhat.com>

perdasilva force-pushed the operator_crd_version branch 2 times, most recently from f160ce9 to d1b5345 Compare April 13, 2023 08:01

update resource definition and tests based on review

bc32937

Signed-off-by: perdasilva <perdasilva@redhat.com>

perdasilva force-pushed the operator_crd_version branch from d1b5345 to bc32937 Compare April 13, 2023 08:03

perdasilva force-pushed the operator_crd_version branch from 5d6bd33 to c11ddec Compare April 13, 2023 10:21

tmshort reviewed Apr 13, 2023

View reviewed changes

perdasilva added 4 commits April 13, 2023 16:12

add more robust semver validation and tests

a05c983

Signed-off-by: perdasilva <perdasilva@redhat.com>

fix admission unit test

1d4b959

Signed-off-by: perdasilva <perdasilva@redhat.com>

fix format, imports, and lint

d63fb62

Signed-off-by: perdasilva <perdasilva@redhat.com>

fix typo and example rendering in .spec.version godoc

d124bc8

Signed-off-by: perdasilva <perdasilva@redhat.com>

perdasilva force-pushed the operator_crd_version branch from fc6d90c to d124bc8 Compare April 13, 2023 14:13

m1kola reviewed Apr 13, 2023

View reviewed changes

tmshort reviewed Apr 13, 2023

View reviewed changes

perdasilva mentioned this pull request Apr 14, 2023

Handling multiple controller-side validations #167

Closed

address reviewer comments

ba1f07e

Signed-off-by: perdasilva <perdasilva@redhat.com>

m1kola approved these changes Apr 14, 2023

View reviewed changes

tmshort approved these changes Apr 14, 2023

View reviewed changes

perdasilva merged commit 2f839d0 into operator-framework:main Apr 14, 2023

perdasilva deleted the operator_crd_version branch April 14, 2023 15:40

awgreene mentioned this pull request Apr 17, 2023

Operator-Controller resolution considers spec.channel #170

Closed

perdasilva mentioned this pull request May 9, 2023

Document testing conventions #188

Open

5 tasks

Add .spec.version #142

Add .spec.version #142

Conversation

perdasilva commented Mar 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joelanford Apr 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joelanford commented Mar 29, 2023

joelanford commented Apr 11, 2023

joelanford Apr 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perdasilva Apr 12, 2023 • edited Loading

Choose a reason for hiding this comment

perdasilva commented Apr 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

m1kola left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perdasilva Apr 14, 2023 • edited Loading

Choose a reason for hiding this comment

m1kola Apr 14, 2023 • edited Loading

Choose a reason for hiding this comment

tmshort Apr 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perdasilva commented Apr 14, 2023

m1kola left a comment

Choose a reason for hiding this comment

tmshort commented Apr 14, 2023 • edited Loading

perdasilva commented Apr 14, 2023

joelanford commented Apr 18, 2023

perdasilva commented Mar 28, 2023 •

edited

Loading

joelanford Apr 11, 2023 •

edited

Loading

joelanford Apr 12, 2023 •

edited

Loading

perdasilva Apr 12, 2023 •

edited

Loading

perdasilva Apr 14, 2023 •

edited

Loading

m1kola Apr 14, 2023 •

edited

Loading

tmshort Apr 14, 2023 •

edited

Loading

tmshort commented Apr 14, 2023 •

edited

Loading