Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

analytics: refactor into a module #2826

Merged
merged 46 commits into from Dec 9, 2019
Merged

analytics: refactor into a module #2826

merged 46 commits into from Dec 9, 2019

Conversation

ghost
Copy link

@ghost ghost commented Nov 20, 2019

Follow up from discussion on #2819

TODO:

  • Write tests

dvc/analytics/__main__.py Outdated Show resolved Hide resolved
dvc/analytics/system_info.py Outdated Show resolved Hide resolved
dvc/analytics/system_info.py Outdated Show resolved Hide resolved
@ghost ghost changed the title analytics: refactor into a module [WIP] analytics: refactor into a module Nov 21, 2019
Copy link
Contributor

@Suor Suor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing a class, but splitting it into several files kind of defeat the purpose of trying to make it more readable.

Copy link
Contributor

@Suor Suor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I was too harsh on files, but I think we shouldn't separate them too much. I've read the old code now and see how is it confusing, you can't even understand the structure of the report at a glance.

dvc/analytics/system_info.py Outdated Show resolved Hide resolved
dvc/analytics/user_id.py Outdated Show resolved Hide resolved
dvc/analytics/user_id.py Outdated Show resolved Hide resolved
@ghost
Copy link
Author

ghost commented Nov 26, 2019

Removing a class, but splitting it into several files kind of defeat the purpose of trying to make it more readable.

Agree, @Suor 🙂

dvc/analytics.py Outdated Show resolved Hide resolved
@ghost ghost changed the title [WIP] analytics: refactor into a module analytics: refactor into a module Nov 27, 2019
@ghost ghost requested review from efiop and Suor November 27, 2019 01:28
@ghost
Copy link
Author

ghost commented Nov 27, 2019

Tests were sort of exhaustive

@efiop
Copy link
Contributor

efiop commented Dec 6, 2019

@MrOutis Tests are still failing, please take a look.

@ghost ghost added the c21-full-week label Dec 6, 2019
dvc/analytics.py Outdated Show resolved Hide resolved
dvc/analytics.py Outdated Show resolved Hide resolved
dvc/analytics.py Show resolved Hide resolved
efiop
efiop previously requested changes Dec 6, 2019
Copy link
Contributor

@efiop efiop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good! I'm still worried about breaking the backend analytics logic by this, so please see some requests for tests up above. Really don't fancy working on analytics backend these days, so we need to do our best to make sure that the reports are exactly the same as they were before and they won't break anything.

@ghost ghost requested a review from efiop December 8, 2019 03:19
@ghost ghost dismissed efiop’s stale review December 8, 2019 03:20

excellent observations, @efiop, thanks again for the review 👍 mind checking if my changes address your comments?

@efiop efiop requested a review from pared December 9, 2019 13:40
Copy link
Contributor

@efiop efiop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ghost ghost requested a review from Suor December 9, 2019 14:25
Fixture to prevent modifying the actual global config
"""
with mock.patch(
"dvc.config.Config.get_global_config_dir", return_value=str(tmp_path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not fspath()?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the patch for fspath and now it doesn't work with pathlib.Path in 3.5: #2903

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does for me, patch is only needed for Python 3.5 builtin pathlib, but pytests tmp_path uses pathlib2.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Suor we've been using str in all other places, so it is no biggie. We need to revisit that more carefully later and fix everywhere.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Suor , didn't new that pytest uses pathlib2!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my bad, then, @Suor , I just assumed it was using builtin pathlib

def test_send(mock_post, tmp_path):
url = "https://analytics.dvc.org"
report = {"name": "dummy report"}
fname = str(tmp_path / "report")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use fspath().

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as this one #2826 (comment)

😞

Comment on lines +72 to +73
with open(fname, "w") as fobj:
json.dump(report, fobj)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(tmp_path / "report").write_text(json.dumps(report))

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Suor , would need to use fspath on analytics.send:

@mock.patch("requests.post")
def test_send(mock_post, tmp_path):
    url = "https://analytics.dvc.org"
    report = {"name": "dummy report"}
    file = (tmp_path / "report")
    data = convert_to_unicode(json.dumps(report))

    file.write_text(data)
    analytics.send(fspath(file))

    assert mock_post.called
    assert mock_post.call_args.args[0] == url

I'd prefer to leave it like this for now, since we lack support for python 3.5 🙁

Copy link
Contributor

@Suor Suor Dec 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only we don't, what exactly doesn't work in Python 3.5 for you?

@ghost ghost requested a review from Suor December 9, 2019 15:32
@efiop efiop merged commit 8890daf into iterative:master Dec 9, 2019
@ghost ghost deleted the refactor-analytics branch December 9, 2019 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhances DVC refactoring Factoring and re-factoring
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants