Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow users to use version suffix pipeline runs #3183

Closed

Conversation

rkpattnaik780
Copy link
Contributor

What changes were proposed in this pull request?

Currently, when pipelines are created using elyra from the Notebooks, each additional iteration of the pipeline run creates a new pipeline version identified by timestamp.

The PR will allow user to switch from timestamp identifiers to versions numbers (v1, etc) using an environment variable KFP_SUFFIX_USE_VERSION. This will make the pipeline runs easily distinguishable.

Fixes #3182

How was this pull request tested?

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

@lresende lresende added component:pipeline-runtime issues related to pipeline runtimes e.g. kubeflow pipelines platform: pipeline-Kubeflow Related to usage of Kubeflow Pipelines as pipeline runtime labels Sep 23, 2023
@lresende
Copy link
Member

@rkpattnaik780 overall LGTM, but could you mention the new env var in the docs

@lresende
Copy link
Member

@rkpattnaik780 could you also rebase the pr to make sure all the tests are passing

Signed-off-by: rkpattnaik780 <rkpattnaik780@gmail.com>
@rkpattnaik780
Copy link
Contributor Author

In an off-thread discussion, we noticed the current implementation will fail if:

  • once we have few pipeline. ex: 10 , latest version will be -v10
  • a user delete the 6th pipeline, the total_size=9
  • the next call of pipeline, would try to create the version '-v10`
    again

We are trying to find a workaround for this issue. Cc @harshad16

@harshad16
Copy link
Contributor

Thanks for the pull request @rkpattnaik780
and for the review @lresende
due this reason: #3182 (comment)
we don't require the feature, we have closed the issue.
Please feel free to close the PR as well.

@lresende lresende closed this Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:pipeline-runtime issues related to pipeline runtimes e.g. kubeflow pipelines platform: pipeline-Kubeflow Related to usage of Kubeflow Pipelines as pipeline runtime
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Introduce pipeline version with numbers(v1,etc) instead of datastamp
3 participants