You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
kedro-boot introduces AppPipeline which is a kedro Pipeline that have some additional metadata used by kedro-boot as a specs for the compilation process.
Kedro's pipeline terminorlogy can already be confusing. Adding another Kind of Pipeline will make kedro-boot hard to grasp for users, and make it harder for us to maintain and document. What if we can infer the kedro-boot compilation specs only from the Pipeline object, leveraging namespaces. Here is a potential mapping between AppPipeline attributes and the Pipeline infered attributes:
AppPipeline name --> Pipeline namespace or 'None' if no namespace
AppPipeline inputs --> namespaced free inputs datasets
AppPipeline outputs --> namespaced outputs datasets
AppPipeline parameters --> namespaced parameters datasets
AppPipeline artifacts --> Automatically infered with this spec : free inputs - namespaced free inputs (artifacts inference can be disabled)
This will allow us to leverage also kedro-viz, as we could visualize theses attributes :
Here we see clearly which datasets that will be "touched" by the kedro boot apps (the namespaced part, the blue one). The "regressor" in this case is not a namespaced free inputs datasets, it will be infered as an artifacts.
Note that kedro boot apps could explicitely give their own compilation specs, which take precedence over these automatically infered specs.
The text was updated successfully, but these errors were encountered:
kedro-boot
introducesAppPipeline
which is a kedroPipeline
that have some additional metadata used by kedro-boot as a specs for the compilation process.Kedro's pipeline terminorlogy can already be confusing. Adding another Kind of
Pipeline
will make kedro-boot hard to grasp for users, and make it harder for us to maintain and document. What if we can infer the kedro-boot compilation specs only from thePipeline
object, leveraging namespaces. Here is a potential mapping betweenAppPipeline
attributes and thePipeline
infered attributes:This will allow us to leverage also
kedro-viz
, as we could visualize theses attributes :Here we see clearly which datasets that will be "touched" by the kedro boot apps (the namespaced part, the blue one). The "regressor" in this case is not a namespaced free inputs datasets, it will be infered as an artifacts.
Note that kedro boot apps could explicitely give their own compilation specs, which take precedence over these automatically infered specs.
The text was updated successfully, but these errors were encountered: