MapController Design

[enisoc]

This is a proposal for a new Metacontroller API that attempts to fill a gap in the set of supported controller patterns that has been identified based on user feedback. Namely, the proposed API offers a way to observe objects you don't own and tie them into your custom behavior without trying to adopt (take ownership of) those objects.

[crimsonfaith91]

To clarify, does the parent object = X = non-owned object, and Y = child object? If so, perhaps ... I want to create a list of Ys according ... is clearer (I think it should be one-to-many mapping).

[enisoc]

Good point. I've updated the wording to clarify there can be any number of Y objects for each X. Thanks!

[janetkuo]

nit: this flag is specific to kubectl, but not everyone is using kubectl

[enisoc]

Reworded to be more general.

[janetkuo]

link doesn't work

[enisoc]

Oops! Removed the link for now. I was originally going to write a section about how to actually do the tagging (label vs annotation? name vs uid? etc.), but then I decided that's more of an implementation detail that isn't needed to decide whether MapController is the right abstraction for the set of problems we're targeting.

[janetkuo]

I'm not sure if I understand the MxN input/output case. IIUC, we tag each output project on which input object they'll be generated from?

[enisoc]

On the input side, the fact that there can be multiple input resources is only used for selecting the pool of input objects. In pseudocode, something like this:

inputObjects := []Object{}
for _, inputResource := range inputResources {
  inputObjects = append(inputObjects, getMatchingObjects(inputResource, parentSelector)...)
}

Then we go through each of the input objects and call the user's hook to generate the output objects for that one input. We tag each output with the input that it came from, which also allows us to select the observed outputs belonging to this input. Continuing in pseudocode:

outputObjects := []Object{}
for _, inputObject := range inputObjects {
  // Compute some opaque string identifying this input object.
  mapKey := makeMapKey(inputObject)

  // Gather observed objects of the output resources that are tagged with this key.
  observedOutputs := []Object{}
  for _, outputResource := range outputResources {
    // Gather all outputs owned by this parent.
    allOutputs := getOwnedObjects(outputResource, parent)
    // Filter to only those tagged for this input.
    observedOutputs = append(observedOutputs, filterByMapKey(allOutputs, mapKey)...)
  }

  // Call user's map hook, passing observed state.
  mapResult := mapHook(parent, inputObject, observedOutputs)
  for _, obj := range mapResult.Outputs {
    // Tag outputs to identify which input they came from.
    setMapKey(obj, mapKey)
    outputObjects = append(outputObjects, obj)
  }
}
reconcileObjects(outputObjects)

Does that help? If so, I can add this to the doc somewhere.

[janetkuo]

Yes, this helps.

[janetkuo]

Not sure if I understand this part. Who orphaned the output objects?

[enisoc]

I added some clarification that this doesn't refer to orphaning in the ownerRef/controllerRef sense. I wish there was another word I could use that doesn't have a collision with another common API machinery term, but I can't think of one yet. Any ideas?

[crimsonfaith91]

Will disconnected or disassociated be a good word? Perhaps can set up a new notion of connection or association (I found the later wording is used frequently in the doc).

[crimsonfaith91]

Another suggestion is the notion of decay, which may have a more vivid tone. If an input object is removed, the output objects that are not deleted are said to be decaying (tends to be deleted in future).

[janetkuo]

I see. I did confuse "orphaning" with ownerRef orphaning. Because parent/child metaphor is used for parent & outputs, the use of the term "orphan" could cause confusion. The clarification helps.

Proposing some more alternative terms: "lost", "untraceable".

[janetkuo]

IIUC, parent owns outputs (label/selector + ownerRef), and selects inputs (label/selector only)?

Can a user make map controller enter crazy creation loop by having the same resource in input and output?

[enisoc]

Yes, good point!

Since we put ownerRef on the outputs like you mentioned, we could ignore our own outputs when looking at inputs. Do you think that's enough to prevent the most likely class of mistakes?

Can you think of any other invariants we can enforce -- things that we're pretty sure the user would never actually want?

If possible, I'd rather not go as far as saying you can't have an output resource that's the same type as your input. I can imagine some legitimate uses for that, as long as it's not recursive (which we can help prevent with ownerRefs). We'll never prevent all possible hot loops (you can set up multiple controllers, for example) so I just see our responsibility as providing reasonable guardrails against common mistakes.

[crimsonfaith91]

Is there any example of use cases involving same type of input and output resources? I think it will be a very infrequent use case for MapController. IIUC, both input and output resources will be different even when they are of same type (because the output resources are new resources created by the MapController). Hence, I think this use case may be better served by other meta-controllers.

[crimsonfaith91]

As synced by person, there are some use cases involving same type of input and output resources like:

1. pod as input, another pod as output
2. service as input, companion service as output
3. ConfigMap with data A as input, ConfigMap with data B as output

[janetkuo]

Since we put ownerRef on the outputs like you mentioned, we could ignore our own outputs when looking at inputs.

This stops resources from using their own children as inputs, which seems fine.

We'll never prevent all possible hot loops (you can set up multiple controllers, for example) so I just see our responsibility as providing reasonable guardrails against common mistakes.

Agree. Adding a note/caveat is probably enough.

[crimsonfaith91]

I think this has the potential to be integrated to stateful operators that we are working on. :)

[crimsonfaith91]

nit: will be owned

[crimsonfaith91]

I think tombstone hook is a more flexible design (e.g., user wants to retain type A output object, but not other types of output objects), but wish to verify whether its role can be technically achieved by existing propagation policy and finalizer feature of garbage collection.

[enisoc]

Do you mean, could we use finalizers as an alternative to the tombstone hook? If so, I think the problem is that the objects would go into a "pending deletion" state, which cannot be undone if the controller later decides to keep the object (e.g. a matching input PVC gets recreated, so the SnapshotSchedule controller decides to keep the "detached" VolumeSnapshot output objects after all).

[crimsonfaith91]

Noted. Tombstone hook is necessary because "pending deletion" state cannot be undone.

[crimsonfaith91]

/lgtm
The design is very nicely done. :)

[barney-s]

/lgtm

[janetkuo]

I replied to some of the comments above. Looks good to me in general.

[enisoc]

I've incorporated the results of our discussions into the doc.

[enisoc]

Do you mean, could we use finalizers as an alternative to the tombstone hook? If so, I think the problem is that the objects would go into a "pending deletion" state, which cannot be undone if the controller later decides to keep the object (e.g. a matching input PVC gets recreated, so the SnapshotSchedule controller decides to keep the "detached" VolumeSnapshot output objects after all).