ability to kedro run --version 'label'

[pascalwhoop]

Description

We currently have no control over the naming of the kedro version according to this doc
https://docs.kedro.org/en/stable/data/data_catalog.html

However it would be nice to control the name of the folder that the kedro run creates. E.g.

kedro run --env prod --version v1.0.1

Can we get this?
--load-versions seems to be meant to read a previous version but you still have no control over what the current version name is going to be.

[datajoely]

So this falls under a wider topic we have about custom session_ids. It was also identified in our #3094 MLOps research.

We have a weird constraint that session_id/versioning needs to be sortable but I still think a composite key of label_{timestamp} would be helpful here.

[pascalwhoop]

Giving some human label would be great. That way we can do things like "look at the pubmedbert-v3 run" instead of having to juggle date strings.

I'd prefix the date to make the sort easier though

[noklam]

As stated, we need to ensure a date_prefix so the order is maintained, adding a suffix should be fine. I am pro for this as I have need for this many years ago, can we link this to the versioning research at least?

#3997 @iamelijahko

[pascalwhoop]

just for your UX research transparency, we now completely moved away from versioning and instead have a `RUN_ID` env variable that we pick up in `globals.yaml` and prefix all pipeline paths with that. we found this approach (all data of a version bundled under one path) to be preferable. Pascal

…

On Tue, Aug 27, 2024 at 3:53 PM Nok Lam Chan ***@***.***> wrote: As stated, we need to ensure a date_prefix so the order is maintained, adding a suffix should be fine. I am pro for this as I have need for this many years ago, can we link this to the versioning research at least? #3997 <#3997> @iamelijahko <https://github.com/iamelijahko> — Reply to this email directly, view it on GitHub <#4028 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARN3F3DRA6TFFHCGHWM6Y3ZTSAG5AVCNFSM6AAAAABLMKSRO6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJSGYZDSOJYGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- The contents of this email are confidential and intended only for the intended recipient. If you are not the intended recipient or responsible for delivery of the message to such person, you may not use, copy, distribute or deliver this message or any part of its contents to anyone, or take any action in reliance on it. If you have received this email in error, please immediately notify the sender and then permanently delete this message. The views, opinions, conclusions and other information provided herein are not intended to be and should not be interpreted as medical advice. Please consult your physician for all medical advice.

[astrojuanlu]

@pascalwhoop It's unlikely that we add this feature for now (especially since there's a reasonable workaround) so @ankatiyar will check to what extent something like this is possible with DVC and we'll turn this into a Discussion.

[ankatiyar]

I looked into it with DVC, and you can indeed tag versions (but with git). The docs for "Kedro + DVC" integration are under construction 🚧 but and should be out soon but you can refer to this comment or this coffee chat

The versioning in DVC is tied to git commits but you can tag the commits with a label eg.

git add data.dvc model.weights.h5.dvc metrics.csv .gitignore
git commit -m "First model, trained with 1000 images"
git tag -a "v1.0" -m "model v1.0, 1000 images"

And then train a second version and commit this too

git add data.dvc model.weights.h5.dvc metrics.csv
git commit -m "Second model, trained with 2000 images"
git tag -a "v2.0" -m "model v2.0, 2000 images"

And go back to a previous version like this

git checkout v1.0
dvc checkout

ETA: Example above from this DVC tutorial: http://dvc.org/doc/use-cases/versioning-data-and-models/tutorial#tutorial-data-and-model-versioning