Behavior of spec.version when not specified

[tmshort]

What is the behavior of spec.version when not specified? There are multiple possibilities, some of which are listed here:

1. Not valid; spec.version must be specified.
2. spec.version is optional, when not specified, nothing happens.
3. A non-specified spec.version means "latest", such that it can be used to upgrade to the latest version of an operator when it becomes available.
4. A non-specified spec.version means "any", such that when initially created, it will refer to any available version (for example, "latest"). On subsequent updates, a non-specified spec.version will match the currently installed version, so no updates occur.

EDIT: If you think there might be other behaviors, please list them, so we can discuss them!

[joelanford]

The current answer (where spec.version is not yet defined) could be summarized as "there is no cluster-admin-defined version constraint for this operator", or stated in plain English, "I don't care which version this operator is installed"

In #162 and for what's in scope in v0.1.0, the answer is the same (though in v0.1.0, we're introducing spec.version so that a cluster admin can specify a desired version)

In the future, I imagine we'll iterate quite a bit on this answer. But regardless, my belief is that version specification (or lack thereof) should be modeled as a constraint on resolution.

The initial set of possible versions is "any", and then various things contribute to that set being reduced:

• A spec.version, if specified
• A currently installed version (which adds restrictions based on the upgrade graph)
• Conflicts declared by other required operators
• Dependencies declared by other required operators

The Operator API is an on-cluster declarative API, which means that point in time style mechanics like "current best version" is generally difficult to implement and will likely come as a surprise. "Current" from the perspective of a reconciler is time.Now(), not metadata.creationTimestamp. Not saying its impossible to handle this kind of thing, but IMO, saying "let's just do what package managers do" is a bit disingenuous in the context of on-cluster always-on, always-reacting resolution.

If we want an imperative upgrades and version selection UX that aligns with the APIs users are using, perhaps we should seriously consider a helm-like model where a client does off-cluster resolution based on off-cluster catalog configurations and then applies the result to the cluster.

Addressing the specific options laid out above, my two cents are:

1. Not valid; spec.version must be specified.

This is certainly an option that is fairly implementable in a declarative API. Though I have a couple of arguments against it:

1. We already have an API that lets cluster admins specify a version they want: a rukpak BundleDeployment. So the value proposition of something at this level would be reduced to:
   a. dereferencing from a version string to a bundle image reference via a catalog lookup
   b. a resolver telling me whether my full set of Operators (all of which I had to specify a version for) are valid or not. In non-trivial scenarios, this means that I need an off-cluster resolver to build a valid set to satisfy my on-cluster resolver. Pretty head-scratching UX IMO.
2. It plays pretty poorly with dependency scenarios where you have opinions about the versions of operators you care about and you are satisfied to inherit the versions of operators that show up in the dependency tree. Its the same problem mentioned in (1b): you end up needing a resolver to build the set that will satisfy your resolver.

In this scenario, the on-cluster resolver is essentially reduced to "yeah I double checked this for you, looks good on my end". It wouldn't actually do any version selection for you.

2. spec.version is optional, when not specified, nothing happens.

What does "nothing happens" mean here? At a minimum, something would need to happen to tell the user what's (not) happening. Usually "nothing happens" in the context of a declarative API is an anti-pattern because it sounds lot like "ignore user intent", unless we're saying that there's an In-N-Out Burger style secret menu in the Operator API where setting spec.version = "" means "pause the reconciler" 😉

3. A non-specified spec.version means "latest", such that it can be used to upgrade to the latest version of an operator when it becomes available.

By "latest", do you actually mean "best available", where "best" is "highest semver that meets all constraints"? If so, this is my favorite option. It seems the most intuitive and logical to me in the context of a declarative Operator API.

4. A non-specified spec.version means "any", such that when initially created, it will refer to any available version (for example, "latest"). On subsequent updates, a non-specified spec.version will match the currently installed version, so no updates occur.

This sounds a lot like the idea that @bparees and I were tossing around in this slack thread: https://kubernetes.slack.com/archives/C0181L6JYQ2/p1680723779244959

A potential future avenue, one we likely need to explore, but one that needs some careful thought, some prototyping and demoing. And then maybe asking some unsuspecting users to use it in front of us to see if it makes sense to them.

[tmshort]

For (2): When I write "nothing happens", I mean, "nothing happens".
In the case of a create operation, nothing will be installed.
In the case of an update/patch operation, the existing operator version will remain installed, and not be upgraded. This basically fixes the version in place.

EDIT: Not very useful, allows creation of the Operator CR without a version; could be a wrapper on top of an operator installed via the Rukpak?

[tmshort]

For (3), since there are no other constraints defined, we're talking about "latest". This is the type of ambiguity we ought to avoid; as once we start adding other constraints, the behavior of the non-specified value changes.

[tmshort]

For (4) Based on what @bparees wrote (paraphrased):

spec.version="" - only resolves once
spec.version="auto/any" - auto upgrade experience (this is just a special keyword to indicate "unbounded range")
spec.version=$pinned - obvious
spec.version=$range - auto upgrade experience within the specified range

Looking at this; the 3rd option is just a case of the 4th option, and the 2nd option is just a special case of the 4th option and could be expressed as >=0.0.0.

I would say this is the closest to what I had in mind for this (4) option's behavior. Hopefully it's easy to describe and understand.

[joelanford]

For (2): When I write "nothing happens", I mean, "nothing happens".
In the case of an update/patch operation, the existing operator version will remain installed, and not be upgraded. This basically fixes the version in place.

But what if another constraint says that the existing installed version is no longer valid? A trivial example would be: what if there's a spec.channel and I change that to a different channel where the current version doesn't exist?

[tmshort]

I added to that response:

EDIT: Not very useful, allows creation of the Operator CR without a version; could be a wrapper on top of an operator installed via the Rukpak?

So, perhaps (2) isn't a valid behavior.

[joelanford]

At the risk of broadening this discussion, my belief is that we should treat this spec.version field (and these other, channel, catalog, etc. fields) as a pure constraint specification and attempt to decouple it from the upgrade lifecycle entirely. I think we're getting wrapped around the axle conflating them together.

Right now the operator-controller upgrade policy is "anytime a reconciliation happens", but it doesn't have to be that way. In a future iteration, we can zero in on upgrade policy with the understanding that there are two separate concepts:

1. When to upgrade
2. When I do upgrade, what are the constraints?

This is sort of the reason that I'm still somewhat adamant that we allow specifying a version range. On one hand, we all seem to be comfortable with the concept of specifying a channel, but a channel is literally just a subset of versions in a package. Mechanically, its the exact same as a version range except that it is defined by the operator author, not the cluster admin. So IMO arguments against version range are somewhat cluster-admin-hostile because they deny cluster admins a tunable knob that we're giving operator authors. Why should operator authors have more control over a cluster's version constraint configuration than a cluster admin?

[tmshort]

Yes, version can be considered a constraint, and I would call it a "first-order" constraint, in that it is the one most likely to be used. And in some cases may be the only constraint used. We should consider the behavior of the version field with future possible (I'm presuming we know some) constraints to try to avoid churning the behavior. I know this isn't always possible, but we ought to have a target.

[tmshort]

I actually prefer (1), in that the version must be specified; even if it is >=0.0.0; that explicitly tells a reader (and operator-controller) that they'll accept anything. An empty value is just too ambiguous, but (4) might work (see my comment above).

[joelanford]

I don't know. Does that mean that every single possible constraint needs to have an explicit "I don't care" value, and all of those constraints have to be explicitly defined. That seems onerous. It seems perfectly intuitive to me that just not stating an opinion means you don't have an opinion.

[tmshort]

I'm wavering between (1) and (4). That being said, I think version is a "first-order" constraint, most likely to be used, possibly the only one used, and should have very-well defined behavior, possibly overriding behavior.

[joelanford]

Not sure about overriding behavior. If two fields in conflict with each other, we shouldn't ignore one of the fields (a user declared their desired state, we can't achieve that declaration). We should just fail resolution.

[tmshort]

Overriding behavior could be failing behavior. If everything else is "AOK", but the version field isn't, then yes, a failure should result. It does lead to the question, are there fields (constraints) that can be ignored regardless of their values, due to other fields? There probably shouldn't be any like that, as that might imply overlapping functionality. I recall @joelanford mentioning something about fields that interfere with each other, and that's something to be avoided.

I guess where I'm really going with this, is that there shouldn't be multiple fields that deal with versions, or with time, or channel, or any other constraint. Orthogonality! (I realize that a channel may only provide a limited selection of versions, but that's not the issue here.)

[joelanford]

@bparees and I have had further discussions in this realm, and I think we have both formed a consensus (between the two of us, at least) that:

1. Leaving version unspecified means "I don't have an opinion about the version".
2. Separately, not specifying an upgrade policy results in a default of "don't perform upgrades".

So the simplest workflow with minimum field setting would end up looking something like:

1. Create an Operator without specifying a version or (yet undefined) upgrade policy.
2. CRD defaulting sees that upgrade policy field is not set, sets it to semantically "never"
3. Operator controller resolves and chooses an initial version to install (likely the highest possible semver in the entire package that is satisfiable given other specified constraints in a cluster-global resolution)
4. Operator controller creates a BD.
5. After the BD exists, future resolutions include a deppy variable that mandates that the currently installed bundle is in the satisfied result set. This essentially means the operator controller will implicitly pin to the initially installed version.

After this workflow, if a user wants to upgrade, they would need to change the upgrade policy field to something other than "never", at which point step (5) changes to handle the new upgrade policy semantics.

With this sort of implementation, another way to trigger an "upgrade" would be to manually delete the underlying BD. At that point, the operator controller's view of the world is "this operator is not installed", so it would end up going down the path of an initial install resolution and another BD creation.

IMO, it needs to work this way because it is typically an anti-pattern for controllers use the status of their primary CR (that they wrote in a previous reconciliation) as input. Status on primary CRs is reconciliation output.

[stevekuznetsov]

What's the downside of having an explicit field for version? If unset upgrade policy gets CRD defaulting to keep the field as stored in etcd always explicit, why is that approach not valid for version?

[bparees]

What's the downside of having an explicit field for version?

where do you see a suggestion there wouldn't be an explicit field for version? or do you mean an explicit value?

[stevekuznetsov]

Yeah, I mean an explicit value as opposed to implicit semantics for ""

[stevekuznetsov]

Your point about interleaving version & upgrade is a good one though. "I have no opinion on the version" + "you are free to upgrade me when you feel like it" would imply that if some other user had strong opinions on the version of some other package with dependency links to the first, your un-opinionated version would be downgraded as necessary to fit the needs of the other.

[bparees]

i don't think we'd ever be able to downgrade an operator w/o the admin explicitly forcing it. We have no information on whether the operator author supports doing the downgrade (there's no downgrade graph, and just because you can upgrade from v1 to v2 doesn't mean you can downgrade from v2 to v1, given potential data migration needs).

but that is also implicit in the fact that we'll never automatically move you from one version to another unless there's a declared edge between them (you can force the move, but olm won't do it automatically).