Allow for display of pipeline | canvas widget properties in the Visual Pipeline Editor

[ptitzler]

As the creator of a notebook pipeline I would like to embed human-readable information in my [notebook] flow that enables other users to quickly determine what the purpose of (1) the pipeline and (2) the included notebooks is. For illustrative purposes let's look at this pipeline, courtesy of https://github.com/CODAIT/covid-notebooks

Assuming user A has created this pipeline and shared it with user B (through whatever means), there's currently a potential information loss risk unless additional information is shared through other means (email, slack message, verbal communication, etc).
This information could entail:

• what's the origin of the pipeline
• what's the purpose of the pipeline
• what's the purpose of a pipeline component (this information is likely embedded in the notebook but requires the user to open it to find out)

Even if a creator currently chooses descriptive notebook names(like it was done here, e.g. etl_us_data), user B wouldn't know what us_data is referring to unless he/she opens the notebook or, in case of generic notebooks, inspects input parameter values or is aware of the pipeline origins. As of today, a pipeline creator would have to embed information about the origin and purpose of the pipeline in one of the notebooks (to make the pipeline "self-explaining"), which doesn't seem to be the appropriate place to convey the information, since a notebook might be part of many pipelines.

[vabarbosa]

@ptitzler couldn't the comment box be used for this purpose?

[ptitzler]

Yes, they could be used but this approach unfortunately does not define context information. For example, is a comment describing the pipeline, a notebook or something else? While a human can infer this if comments are properly positioned (or by interpreting the content), it cannot be done automatically. As a result the information could not be easily externalized outside the editor.

For example, let's say sometime in the future we want to enable a pipeline "preview" in a navigator that would display the pipeline comment, if one was defined and the user hovers over the pipeline name

[ptitzler]

With the latest changes to the Pipeline Editor the UI now already has a suitable mechanism that can be used to expose pipeline metadata.

Current behavior (properties are specific to selected node):

Proposed behavior:

• if no node is selected, the properties view exposes pipeline metadata (user provided and system generated)
• if one node is selected, the properties view exposes the node settings (no change)
• if more than one node is selected, the properties view displays a help message (slight change)

[bourdakos1]

I would opt for a dedicated PIPELINE PROPERTIES tab instead maybe?

This would also be a good place to potentially add a global defaults system

[ptitzler]

The following is a list of pipeline related "properties" that we had discussed before that we don't surface yet in the UI:

• Properties associated with a pipeline
  • Description (optional user provided free form text)
  • Pipeline runtime type (user friendly name, e.g. Kubeflow Pipelines or Apache Airflow for pipelines that are not generic)
  • Global pipeline properties that can optionally be overridden by node properties (e.g. runtime image)
• Properties associated with a comment
  • comment text
  • font size

[bourdakos1]

@ptitzler is there a use case where you would want different comment nodes to have different font-sizes? And for this setting to be shared with other people using this pipeline file?

[ptitzler]

is there a use case where you would want different comment nodes to have different font-sizes?

I can't really think of any, assuming that comment rendering takes the font size into account.

And for this setting to be shared with other people using this pipeline file?

Sorry, I didn't realize that more recent versions of JupyterLab now support this, so maybe there is no need to include font size as a pipeline property.

Note the font size in the tabs (larger because I increased it using ^^ setting)

I ran a quick test and it appears that the visual pipeline editor doesn't honor changes to these settings (decreased the font size in the setting - again note the difference in the tab text):

Side note (it's off-topic), the comment text color is rather faint for a light theme and gives more of an impression of a default input text than a user provided text.

[ptitzler]

Based on my last comment

I ran a quick test and it appears that the visual pipeline editor doesn't honor changes to these settings

wdyt, should I create a new issue for that since this is different from what this issue is all about?

[bourdakos1]

The vscode extension respects the font size of the editor, but for JupyterLab I think we will need to check if it exposes the font information somewhere. I would open a new issue for the font size bug

[ptitzler]

Done. Font size -> #1613

[lresende]

@ptitzler This issue has become a little confusing with all its history, and some partial fixes already in via more specific issues.
Should we clarify what is still missing here, or open specific issues to more targetted items to be worked on.

[marthacryan]

#1708 fixes this