Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add telemetry for language server installs #2461

Closed
qubitron opened this issue Aug 29, 2018 · 8 comments · Fixed by #2597
Closed

Add telemetry for language server installs #2461

qubitron opened this issue Aug 29, 2018 · 8 comments · Fixed by #2597
Assignees
Labels
area-intellisense LSP-related functionality: auto-complete, docstrings, navigation, refactoring, etc. debt Covers everything internal: CI, testing, refactoring of the codebase, etc.

Comments

@qubitron
Copy link

We should collect telemetry on language server download times and failures, so that we can become aware of issues.

I propose we add events that allow us to track the following:

  • How many users attempt to download the language server
  • How many succeed
  • How many fail, and reasons for failure
  • Amount of time taken for each of: download, unpack, and install
  • Time taken to analyze workspace first time after install
@qubitron qubitron added area-intellisense LSP-related functionality: auto-complete, docstrings, navigation, refactoring, etc. needs decision feature-request Request for new features or functionality labels Aug 29, 2018
@brettcannon brettcannon added this to the Aug 2018 milestone Aug 29, 2018
@brettcannon brettcannon added needs PR debt Covers everything internal: CI, testing, refactoring of the codebase, etc. and removed needs decision feature-request Request for new features or functionality labels Aug 29, 2018
@MikhailArkhipov
Copy link

MikhailArkhipov commented Aug 30, 2018

We already report

    PYTHON_LANGUAGE_SERVER_DOWNLOADED,
    PYTHON_LANGUAGE_SERVER_ENABLED,
    PYTHON_LANGUAGE_SERVER_ERROR

The latter has error text attached

@WangElysion

This comment has been minimized.

@brettcannon

This comment has been minimized.

@d3r3kk d3r3kk self-assigned this Sep 11, 2018
@d3r3kk
Copy link

d3r3kk commented Sep 13, 2018

@qubitron , just FYI, there isn't really an install phase per se. There is download, then extract - that is pretty much it.

As for the time to analyze the workspace for the first time, what is the question you are trying to answer here?
(I think perhaps that won't tell you too much useful information as it truly does depend on the size of that workspace? The users processor speed/IO capacity/etc... also factor in)

@qubitron
Copy link
Author

@d3r3kk I'm interested to know on what the first time user experience looks like, for the language server, are people waiting seconds, minutes, or hours before they can start using it. Being able to break that down into download, extract, first time analysis analyzing will help us break that down further to know where any issues are.

I'm also interested to know in general how much time people wait for workspace analysis to complete on an ongoing basis, and how many people are getting stuck on the "analyzing workspace" issue #2297 (analysis starts but never completes).

@d3r3kk
Copy link

d3r3kk commented Sep 13, 2018

Ok, that's cool and makes sense. Generally speaking, how long do our users wait for analysis to complete, and how many folks are affected by any never-complete scenarios.

Would it be better/more preferrable then, to reword the last item in your list as:

  • Time taken to analyze workspace every time.

...and then add:

  • If analysis takes > N seconds, report a never-complete scenario.

@d3r3kk
Copy link

d3r3kk commented Sep 13, 2018

Alternatively, we could say:

  • Time taken to analyze workspace for the first N times.

...to keep it a bit more reasonable perhaps.

@DonJayamanne
Copy link

Time taken to analyze workspace every time.
...and then add:
If analysis takes > N seconds, report a never-complete scenario.

I think this is better. From this we can capture the success rate for completion of analysis.

d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Sep 15, 2018
Fixes microsoft#2461 (alternate fix to PR microsoft#2593)

- Capture telemetry surrounds methods, minimizing change
- Telemetry type can be altered with less code later.
- Add success/fail props modifyer func to `captureTelemetry`
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Sep 17, 2018
Fixes microsoft#2461 (alternate fix to PR microsoft#2593)

- Capture telemetry surrounds methods, minimizing change
- Telemetry type can be altered with less code later.
- Add success/fail props modifyer func to `captureTelemetry`
d3r3kk added a commit that referenced this issue Sep 18, 2018
@lock lock bot locked as resolved and limited conversation to collaborators Oct 16, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-intellisense LSP-related functionality: auto-complete, docstrings, navigation, refactoring, etc. debt Covers everything internal: CI, testing, refactoring of the codebase, etc.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants