task: optionally skip downloading artefacts on destroy

[DavidGOrtega]

Right now destroy syncs data having unnecessary cloud traffic (to be paid). If i do not care about the data back we can have a flag pullOnDestroy to be able to set this like #301

[0x2b3bfa0]

having unnecessary cloud traffic (to be paid)

Cents, if you use directory for code synchronization. If you use it as an intermediate cache for data, it might cost a lot more.

We can either allow this choice or create a separate cache directory that never gets synchronized back; it would be used to store the intermediate data (not artifacts) using during the training process.

[DavidGOrtega]

We can either allow this choice

I prefer also this

Cents, if you use directory for code synchronization.

also the time to sync, can be hours were your machine is asking to die. For some reason it takes some CPU in my local machine

[0x2b3bfa0]

#307 could reduce considerably the pull times, especially when there is nothing to pull. 😜 Still having an option to disable pulling the data doesn't seem like a bad idea either.

[DavidGOrtega]

#307 could reduce considerably the pull times,

Why #307 would reduce the time?

[0x2b3bfa0]

Why #307 would reduce the time?

Because users are the ones who choose what gets downloaded and what doesn't. Still, if artifacts are huge it will be the same.

[MaxHuerlimann]

Would also be interested in this feature. In our use-case with DVC we can handle updating remote cache and creating pull request on the remote cloud machine after model training, so the data syncing back to the CI/CD runner machine is not needed and only increases pipeline run times unnecessarily.