-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
api: fallback to external_repo if not a DVC repository #3222
Conversation
@@ -128,10 +128,10 @@ def pull_to(self, path, to_info): | |||
raise PathMissingError(path, self.url) | |||
|
|||
@contextmanager | |||
def open_by_relpath(self, path, mode="r", encoding=None): | |||
def open_by_relpath(self, path, mode="r", encoding=None, **kwargs): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
open_by_relpath on the Repo
also takes more arguments such as remote
, etc which does not make sense here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it works it should stay as is. Let's not add ignored arguments, this will lead to silent errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does not as it also gets unexpected arguments from Repo.open_by_relpath
:
Line 430 in 4903996
def open_by_relpath(self, path, remote=None, mode="r", encoding=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could though add remote
, or not even send one, but, I prefer this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand. This line is never called from Repo.open_by_relpath().
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's called by dvc.api.open()
here with following args:
Lines 70 to 80 in 4903996
def open(path, repo=None, rev=None, remote=None, mode="r", encoding=None): | |
"""Context manager to open a file artifact as a file object.""" | |
args = (path,) | |
kwargs = { | |
"repo": repo, | |
"remote": remote, | |
"rev": rev, | |
"mode": mode, | |
"encoding": encoding, | |
} | |
return _OpenContextManager(_open, args, kwargs) |
And, successively, _open
calls this line:
Lines 94 to 100 in 4903996
def _open(path, repo=None, rev=None, remote=None, mode="r", encoding=None): | |
with _make_repo(repo, rev=rev) as _repo: | |
with _repo.open_by_relpath( | |
path, remote=remote, mode=mode, encoding=encoding | |
) as fd: | |
yield fd | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, then we should add only remote
param not **kwargs
and assert remote is None
. This way things won't be silently ignored.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Suor assert is not needed, as the point of that method is to provide the same interface for git and dvc repos, so it should be silently ignored there. Or am I missing something?
try: | ||
abs_path = os.path.join(self.root_dir, path) | ||
with open(abs_path, mode, encoding) as fd: | ||
with open(abs_path, mode, encoding=encoding) as fd: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Third argument is buffering
.
e825f64
to
fcd0bc8
Compare
@skshetry Please check the tests, they've failed. |
Codecov Report
@@ Coverage Diff @@
## master #3222 +/- ##
==========================================
- Coverage 91.67% 91.33% -0.34%
==========================================
Files 139 139
Lines 8681 8681
==========================================
- Hits 7958 7929 -29
- Misses 723 752 +29
Continue to review full report at Codecov.
|
908f5c6
to
566aa57
Compare
@@ -128,10 +128,10 @@ def pull_to(self, path, to_info): | |||
raise PathMissingError(path, self.url) | |||
|
|||
@contextmanager | |||
def open_by_relpath(self, path, mode="r", encoding=None): | |||
def open_by_relpath(self, path, mode="r", encoding=None, **kwargs): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it works it should stay as is. Let's not add ignored arguments, this will lead to silent errors.
566aa57
to
003b5c3
Compare
cdd9d0d
to
2e0b143
Compare
Fixes #3221. Now, it will just fallback to the
external_repo
ifRepo()
could not be instantiated.Another idea would have been to check for the existence of
.dvc
directory, but, I went with this, as it feels correct and robust.❗ Have you followed the guidelines in the Contributing to DVC list?
📖 Check this box if this PR does not require documentation updates, or if it does and you have created a separate PR in dvc.org with such updates (or at least opened an issue about it in that repo). Please link below to your PR (or issue) in the dvc.org repo.
❌ Have you checked DeepSource, CodeClimate, and other sanity checks below? We consider their findings recommendatory and don't expect everything to be addressed. Please review them carefully and fix those that actually improve code or fix bugs.
Thank you for the contribution - we'll try to review it as soon as possible. 🙏