Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

api: support streaming from Google Drive remotes #3408

Closed
1 task
jorgeorpinel opened this issue Feb 25, 2020 · 5 comments · Fixed by #3916
Closed
1 task

api: support streaming from Google Drive remotes #3408

jorgeorpinel opened this issue Feb 25, 2020 · 5 comments · Fixed by #3916
Assignees
@casperdcl
Labels
enhancement Enhances DVC performance improvement over resource / time consuming tasks refactoring Factoring and re-factoring research

Comments

@jorgeorpinel
Copy link
Contributor

Extracted from iterative/dvc.org#908 (comment)

GDrive is the only remote type that doesn't support file streaming. Should we work on adding it?

  • and update docs (e.g. api ref)
@jorgeorpinel jorgeorpinel added the enhancement Enhances DVC label Feb 25, 2020
@jorgeorpinel
Copy link
Contributor Author

p.s. maybe could be included in #2865 and closed here.

@shcheklein
Copy link
Member

@jorgeorpinel yes, let's add a check box to that list and keep/remove this ticket - I don't have a strong preference

@maxhora maxhora mentioned this issue Feb 26, 2020
Closed
33 tasks
@jorgeorpinel jorgeorpinel mentioned this issue Feb 27, 2020
Merged
1 task
@weekly-digest weekly-digest bot mentioned this issue Mar 1, 2020
Closed
@casperdcl casperdcl mentioned this issue May 5, 2020
Merged
7 tasks
@casperdcl casperdcl self-assigned this May 5, 2020
@casperdcl casperdcl added c8-full-day performance improvement over resource / time consuming tasks refactoring Factoring and re-factoring research labels May 5, 2020
@casperdcl casperdcl reopened this May 5, 2020
@casperdcl casperdcl removed their assignment May 5, 2020
@casperdcl casperdcl self-assigned this May 30, 2020
@casperdcl casperdcl mentioned this issue May 30, 2020
Closed
@casperdcl
Copy link
Contributor

casperdcl commented May 30, 2020 •
edited
Loading

depends on iterative/PyDrive2#39 and iterative/PyDrive2#40

casperdcl added a commit to casperdcl/dvc that referenced this issue May 30, 2020
@casperdcl
gdrive: add open
4ccbc95 
Fixes iterative#3408
Related iterative#2865
Fixes iterative#3897
@casperdcl casperdcl mentioned this issue May 30, 2020
Merged
8 tasks
@jorgeorpinel
Copy link
Contributor Author

Both PRs are merged now, does it mean maybe this is now supported? If so, to QA it. Thanks

@casperdcl
Copy link
Contributor

casperdcl commented May 31, 2020 •
edited
Loading

well it means #3916 is (maybe) no longer blocked so it's getting close to being supported :)

casperdcl added a commit to casperdcl/dvc that referenced this issue Jun 12, 2020
@casperdcl
gdrive: add open
0c4e859 
Fixes iterative#3408
Related iterative#2865
Fixes iterative#3897
efiop pushed a commit that referenced this issue Jun 13, 2020
@casperdcl
gdrive: add open (#3916)
4d140a9 
* gdrive: add open

Fixes #3408
Related #2865
Fixes #3897

* dependency: add gdrive

* test: api: open: gdrive

* rollback pydrive dependency

* Revert "dependency: add gdrive"

This reverts commit fd33326.

* tests: remotes: GDrive: fix get_url

* tests: api: fully test GDrive

* minor typo fix

* attempt gdrive tests fix

* fix gdrive credentials config

* replace GDrivePathNotFound => FileMissingError

* fix test_open_external[GDrive]

* add ensure_dir_scm

* minor exception fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@casperdcl casperdcl

Labels
enhancement Enhances DVC performance improvement over resource / time consuming tasks refactoring Factoring and re-factoring research
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

gdrive: add open casperdcl/dvc
3 participants
@jorgeorpinel @shcheklein @casperdcl