[NODES] Channel Separation Options

[itsmattkc]

Issue by sobotka
Saturday May 09, 2020 at 22:42 GMT
Originally opened as sobotka#122

---

Summary

Nodes should permit selection of application.

For example, the math node should have some sort of UI selection available that permits the creator to choose which channels are applied to. Given that Olive uses a table to pass values and arrays down the chain, it is likely that this channel selection needs to be dynamic.

[itsmattkc]

I feel a little uncertain of the exact behavior of this.

Let's say we had a blur node and then selected only the red channel. Which of these would be the expected output?

(i.e. red channel blurred, rest of the image intact)

(i.e. red channel blurred, no other channels)

[itsmattkc]

Comment by sobotka
Tuesday Jul 14, 2020 at 15:33 GMT

---

It would only apply to the selected data.

A blur with an RGBA+ancillary appends, it would be feasible to apply to only the RGB, or any of the data.

Nodes by nature should be "dumb" and not interpret or shim in any operation that isn't requested. So for example, with an associated alpha image, if a node routes an RGBA to a colour manipulation, it's going to produce wrong output, and it likely should produce wrong output. Otherwise it heads down a neverending rabbit hole of "smart software" checks, which as you can guess, becomes untenable with all possible combinations.

A checkbox table allows for discrete application of the node, in a relatively sane manner. Only want the blur applied to blue, green, and alpha? No problem.

In your example, I'd expect that if we dumped the image raw to the viewer, the result would be the totality of the buffer, RGBA, with the R channel blurred as chosen. As in if the node is passed A B C D X1 Y1 Z1 F Foobar, and Foobar is selected, the node should blur the Foobar and pass down the identical table with the Foobar blurred. In the Viewer node, if only the R of the R G B A Foobar was selected, we'd see only R as with your second image. Of course, it would also be entirely legitimate to select Foobar and dump the results to the viewer.

Ultimately it is up to the nodes to determine what their default action is. That is, in your example, passing R G B A Foobar to the Viewer, the viewer will look for the known defaults of R G B A and display the result, which in this specific example would be the blurred R with unblurred G B A.

TL;DR Nodes should not delete tables without (perhaps?) a delete node. Otherwise it simply passes the table data along, nodes look for specific default labels, and crunch the numbers. Answer to your specific question: Depends on the node that is displaying the data.

[itsmattkc]

Sure I get that, so what happens in a scenario like this:

Say we're adding two textures together and the channels are selected like this:

Input 1:        [x] R        [ ] G         [ ] B
Input 2:        [x] R        [x] G         [ ] B

How are these "added" like this? Obviously we could say add the two reds, keep the green from 2, and discard green 1 and the blues, but that's not the behavior of the second blur image, not the first.

[itsmattkc]

Comment by sobotka
Saturday Jul 18, 2020 at 02:34 GMT

---

My sensibility suggests that the values added are dictated by the check, as in the table parameter.

Consider two tables:

R G B F X Y Z
R G B F A O Z

The selection would be the union of parameters:

R G B F X Y Z A O Z

And the operation would consider all values of that parameter, and apply. In the case where only one buffer has a parameter, consider NULL and pass through.

Blur with R G B A and all four channels selected would blur all four. With only R selected, blur applies to R and the others pass through.

Thoughts?

[itsmattkc]

Comment by sobotka
Saturday Jul 18, 2020 at 03:07 GMT

---

Thinking about it more, I think your approach is more useful, to permit channels to cross talk.

[itsmattkc]

I'm not really proposing or advocating for any approach, this is just the puzzle I ran into while trying to figure out how to actually implement this.

To be honest, I like Blur Example #1 where you could simply select channels to operate on and the rest would pass through, but it becomes complicated whenever more than one input texture is involved (As in the "add" example, what does it mean for two blue channels to get passed through? How does one green channel get added to nothing and the other green channel passes through?)

I feel like the dilemma is kind of like this:

• Blur Example #1 is more akin to selecting "output channels" on the node as a whole, i.e. which channels of the output buffer have been processed by the node, and which have just been passed through from the input.
• Blur Example #2 is more akin to selecting "input channels", i.e. which channels are the input buffer(s) are sent to the node and which are "disabled".

Perhaps both options need to be available to the user.

[itsmattkc]

Comment by sobotka
Saturday Jul 18, 2020 at 15:08 GMT

---

I believe what might be able to work is:

1. A list box in the Parameters with a + / - selection.
2. Add a line to the list box. Two columns.
3. Left column enumerates the input buffer parameters, in a group box.
4. Right side column enumerated the output buffer parameters.

The input then is applied to output. If the goal were to add the R to F, G, and H, three rows, with R selected for each on the left, and F G H respectively on the right.

The others pass through.

I’m guessing that an Append node would add on selected parameters. For example, if an input table contains D E F and the output only has A, the append node could work with the above UI and permit the audience to append any to all of the three from the input.

Seems like a relatively powerful matrix. Nodes that do typical things, could perhaps have a set default series of enumerated matrix elements already predefined, subject to addition or removal as required.

The first example with the image:

1. Add Filter node.
2. Select input table R from the enumeration.
3. Select output table parameter from node on right. Lanczos, Cubic, Box, Prefilter, etc.

Done!

This matrix approach also strikes me as powerful because it would allow a node to perform any and all of its operations to specific channels, and encode them as such.

Perhaps it’s three columns. Input on left, operation next, and output table encoding on right?

[itsmattkc]

Something like this where you could swap channels around or bypass entirely? This could probably be quite doable.

[sobotka]

I think so? Being able to append channels for repeats is useful here too, such as adding another B and using it to drive some other value or append it down the chain?

Perhaps removing them on the node results in removal, while leaving them results in pass through?

[sobotka]

Riffed a bit more on your mock. Let's see where this goes...

• Parameters here would be the node parameters, allowing a single node to apply things multiple different ways potentially. Note how the tabs would contain an instance of the node parameters, subject to a custom name set by the audience. This name and parameter settings would appear in the processing box. Examples might include different sampling approaches per record for image resizing, different blurring settings for a blur, etc. Each parameters instance would be a complete set of the node default parameters.
• Clicking the + / - allows for appending or removal of table records.
• Connectors define the relationships.

Outstanding question is whether the processing table in the matrix is required. Perhaps each processing tab gets the metadata from the incoming records that allows the audience to select with fine granularity all of the incoming processing to drive to node processing? In theory the processing Parameters tabs can hook up the appropriate record. Perhaps only the output is required to order / rename the records?

Open canvas!

[sobotka]

Final one for the night. Perhaps the Matrix tab isn't required at all, and it is a part of the Parameters tab instances?

The following shows an example of the parameters that are impacting the chosen inputs, which show a demonstration of two different parameters being applied within the same node. The second example shows how the table records can drive the parameters as well.

[Shrinks99]

I Would move these routing graphs to their own dedicated tab separate from all the other node parameters. Most of the time users will be operating a node's controls and should not need to switch around channels so these controls don't need to be visible at the same level as all the other main node controls. This separates the routing of data (which will be the same controls for all nodes presumably) from a node's operations (which will vary per-node). This means that when users expect to access these controls they will be in a predictable place — that separate tab.

Simplifying to two is good, perhaps they should be labelled "input" and "operating" or something? All channels are being output through the node but only some are being operated upon, this also needs to be displayed to the user.

Additionally when speaking to Troy he mentioned that this data could possibly be used to drive a node's actual parameters. If this is to be done I would advice that it be controlled separately from this matrix, have one UI element for controlling which channels will be operated upon, and a separate system for controlling the parameters by a certain component.

[sobotka]

I Would move these routing graphs to their own dedicated tab separate from all the other node parameters.

The issue there is the processing is applied to the selected records in the second example.

This means that when users expect to access these controls they will be in a predictable place — that separate tab.

Subject to discussion. It is useful to have the effected records spatially close to the affecting parameters.

Much to ponder, but it feels like there is some decent clay here to sculpt from!

[Shrinks99]

The issue there is the processing is applied to the selected records in the second example.

This is why I initially proposed a separate system for routing data from outside of the node to its properties however looking at this again I think what you have is solid, there's no reason that data can't just connect to multiple places and this seems like it would work well for both routing what types of data the node operates on and routing incoming data to control a property value. The latter option could potentially be very powerful!

I do still think this routing system should probably be its own tab within the node properties panel, just need a good properties selection system to add those properties to the outputs list? Would make a mockup before coming to any conclusions there though (and that I definitely don't have time to do ;P ).