v3.12.0

This release requires pipeline migration.

Quick links

• Complete changelog
• Release documentation
• Installation documentation
• Getting help

New feature highlights

Pipeline editor: attach Kubernetes labels to nodes

Kubernetes labels can now be defined as pipeline defaults or assigned to individual nodes.

Use labels to assign identifying metadata to the Kubernetes pods that execute the node.

Pipeline editor: disable node caching

Some runtime environments support node output caching, reducing the need to re-execute nodes, which can improve performance and reduce resource usage. For the Kubeflow Pipelines runtime you can now disable node caching for custom nodes that produce output in a non-deterministic way. Node caching can be disabled for all nodes by specifying a pipeline default or for individual nodes.

Generic nodes are currently never cached, due to an Elyra limitation.

Pipeline editor: user-friendly property inputs

Pipeline default properties and node properties can now be configured using custom widgets:

Refer to the documentation for widget descriptions:

• Disable node caching
• Environment variables
• Data volume mounts
• Kubernetes pod labels
• Kubernetes pod annotations
• Kubernetes secrets
• Kubernetes tolerations

Pipeline editor: use files to provide text inputs

In previous releases string-based input properties for custom components could only be provided by entering text or selecting the output of an upstream node. The text input widget in the pipeline editor was extended to also allow for text input from a local file:

In the example above local file url.txt contains a valid URL. The content of this file is not stored in the .pipeline file, only its name and location. The file is only processed by Elyra when you submit or export a pipeline using the pipeline editor or the elyra-pipeline CLI.

The elyra-pipeline describe CLI command was extended to list input files like these as local file dependencies that must be distributed with the .pipeline file:

$ elyra-pipeline describe download.pipeline
...
Pipeline name: download
Description: None specified
Pipeline type: KUBEFLOW_PIPELINES
Pipeline format version: 7.5
Pipeline runtime: Kubeflow Pipelines
Number of generic nodes: 0
Number of custom nodes: 1
Script dependencies: None specified
Notebook dependencies: None specified
Local file dependencies:
    - url.txt
...

Improved component catalog connectors

The Apache Airflow package component catalog connector, the Apache Airflow provider package component catalog connector, and the URL component catalog connector now support secured environments where SSL server authenticity can only be validated using certificates based on private public key infrastructure with root and optionally intermediate certificate authorities that are not publicly trusted. Refer to the connector documentation links for details.

What's Changed

New Features

• Support disabling node output caching for custom KFP components by @ptitzler in #2905
• Support not publicly trusted certificates in built-in component catalog connectors by @ptitzler in #2912
• Make multi-valued input fields more user-friendly by @kiersten-stokes in #2927
• Pipeline editor: Allow for configuration of labels by @ptitzler in #2943

Bug Fixes

• Move CLI component cache build to occur before pre-processing by @kiersten-stokes in #2913
• Account for how canvas handles icon size by @ajbozarth in #2914
• Fix rendering of toleration effect in generic components by @kiersten-stokes in #2946
• Fix annotations and label processing by @ptitzler in #2947

Other

• simplify handlers test by @ptitzler in #2900
• Organize doc images by source doc by @ptitzler in #2910
• Improve content of the pipelines documentation topic by @ptitzler in #2909
• Update generic nodes to use static icon api by @ajbozarth in #2908
• Enable debugger on kernel change by @karlaspuldaro in #2903
• Error Dialog Integration Tests by @vhada7 in #2870
• Update Python icon to match JupyterLab by @ajbozarth in #2917
• Add initial support for rjsf in pipeline properties by @marthacryan in #2780
• Add migration section to the pipeline documentation topic by @ptitzler in #2928
• Document how to identify installed Elyra version by @ptitzler in #2930
• Update parse-url dependency due to CVE-2022-2900 by @akchinSTC in #2935
• Fix typo in filename by @ptitzler in #2940
• Explicitly specify units of gigabytes for RAM in Airflow DAG template by @kiersten-stokes in #2949
• Update pipeline editor node package to 1.10 by @akchinSTC in #2950

New Contributors

• @vhada7 made their first contribution in #2870

Full Changelog: v3.11.0...v3.12.0