Issue 843 | Files: download files saved during Program execution #896
Conversation
IceKhan13
changed the title
IceKhan13
changed the title
| progress_bar = tqdm( | ||
| total=total_size_in_bytes, unit="iB", unit_scale=True | ||
| ) | ||
| file_name = f"downloaded_{str(uuid.uuid4())[:8]}_{file}" |
There was a problem hiding this comment.
Why does this download into an unique file name? Avoiding override or appending?
There was a problem hiding this comment.
yes, to avoid overriding and we also return file name too, if user want to do something with it programmatically
Co-authored-by: Paul Schweigert <paul@paulschweigert.com>
| with open("./my_file.txt", "w") as f: | ||
| f.write("Hello!") | ||
|
|
||
| with tarfile.open("/data/my_file.tar", "w:gz") as tar: |
There was a problem hiding this comment.
The file in the data directory is not deleted after this execution. The next execution of the same program (or generating the same name file) override the file. It it fails to generate the file, the old file may be down loaded. It is user responsibility to generate an unique file name so that it would not be override or confused?
There was a problem hiding this comment.
yes, it is up to user to handle their files and names of them inside the program.
There was a problem hiding this comment.
LGTM
(since @akihikokuroda also had comments, will let him give final approval)
| """ | ||
| return self._selected_provider.files() | ||
|
|
||
| def download(self, file: str, download_location: str = "./"): |
There was a problem hiding this comment.
Is adding the delete after download option a bod idea?
There was a problem hiding this comment.
I think if user downloads the file, he probably need it 😄
Probably we do not need this flag
There was a problem hiding this comment.
How he can delete it if he is done with it?
There was a problem hiding this comment.
ah, you mean delete from server. For now bucket policy will delete it or user can remove it in a job
There was a problem hiding this comment.
if we want to delete it it's probably worth createing follow up issue request to add /delete endpoint for files
There was a problem hiding this comment.
and probably /upload would be great to add too
There was a problem hiding this comment.
but so far we only had download request :)
Summary
Add feature to download save files to
/datafolder within programs.Example:
Todos:
Details and comments
Closes #843