Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/telemetry/config: collect gopls/telemetryprompt/accepted #68770

Closed
findleyr opened this issue Aug 7, 2024 · 6 comments
Closed

x/telemetry/config: collect gopls/telemetryprompt/accepted #68770

findleyr opened this issue Aug 7, 2024 · 6 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. telemetry x/telemetry issues Telemetry-Accepted Telemetry-Proposal
Milestone
gopls/v0.16.2

Comments

@findleyr
Copy link
Member

findleyr commented Aug 7, 2024

Summary

From gopls, we ask a select fraction of users to enable telemetry via an LSP prompt (a showMessageRequest dialog). Starting in gopls@v0.16.1, we record a gopls/telemetryprompt/accepted counter when users accept the prompt.

By collecting this counter, we can estimate the fraction of new telemetry participants that are coming from the prompt, which will help us evaluate the efficacy of the prompt itself.

Of course, we cannot know how many users decline the prompt.

Proposed Config Change

https://go.dev/cl/603936
@gopherbot gopherbot added the telemetry x/telemetry issues label Aug 7, 2024
@gopherbot gopherbot added this to the Unreleased milestone Aug 7, 2024
@gabyhelp
Copy link

gabyhelp commented Aug 7, 2024

Related Issues and Documentation

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

@findleyr findleyr modified the milestones: Unreleased, gopls/v0.16.2 Aug 9, 2024
@findleyr
Copy link
Member Author

findleyr commented Aug 9, 2024 •
edited
Loading

It occurred to me that because we have a conservative cutoff, we won't ever upload the counter file for the day that telemetry is enabled: we only upload counter files that do not overlap with any period of time when telemetry uploading was not enabled.

Therefore, I think gopls should record at every startup whether the prompt was accepted. Theoretically, we could also record other prompt outcomes: some users may decline the prompt but enable telemetry via other means.

Putting this in the gopls@v0.16.2 milestone to address the collection issue.

@cagedmantis cagedmantis added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 12, 2024
@cagedmantis
Copy link
Contributor

cc @golang/telemetry

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/603936 mentions this issue: internal/chartconfig: add collection for gopls/telemetryprompt/accepted

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/607758 mentions this issue: gopls: increment the telemetryprompt acceptance counter for each session

gopherbot pushed a commit to golang/tools that referenced this issue Aug 28, 2024
@findleyr
gopls: increment the telemetryprompt acceptance counter for each session
594cdab 
Increment the gopls/telemetryprompt/accepted counter every time we read
the prompt file, so that we can actually use the resulting data. We
don't upload any counter files that overlap with periods of time when
telemetry uploading was not enabled, so with the previous model of only
incrementing the counter at the moment the prompt was accepted, we will
never upload the counter file into which the acceptance is recorded.

With the new model, the counter will give us a sense of what fraction of
telemetry participants opted-in as a result of the prompt.

For golang/go#68770

Change-Id: I8890c73b5bfa19023bb24fd156bcaa9eb46295ad
Reviewed-on: https://go-review.googlesource.com/c/tools/+/607758
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/609135 mentions this issue: [gopls-release-branch.0.16] gopls: increment the telemetryprompt acceptance counter for each session

gopherbot pushed a commit to golang/tools that referenced this issue Aug 29, 2024
@findleyr
[gopls-release-branch.0.16] gopls: increment the telemetryprompt acce…
dcc114e 
…ptance counter for each session

Increment the gopls/telemetryprompt/accepted counter every time we read
the prompt file, so that we can actually use the resulting data. We
don't upload any counter files that overlap with periods of time when
telemetry uploading was not enabled, so with the previous model of only
incrementing the counter at the moment the prompt was accepted, we will
never upload the counter file into which the acceptance is recorded.

With the new model, the counter will give us a sense of what fraction of
telemetry participants opted-in as a result of the prompt.

For golang/go#68770

Change-Id: I8890c73b5bfa19023bb24fd156bcaa9eb46295ad
Reviewed-on: https://go-review.googlesource.com/c/tools/+/607758
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 594cdab)
Reviewed-on: https://go-review.googlesource.com/c/tools/+/609135
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Sep 13, 2024
@0-wiz-0
gopls: update to 0.16.2.
0d7379e 
Changes in this release

This release updates the golang.org/x/telemetry dependency to pick
up fixes for the following bugs:

    golang/go#68946: unnecessary downloading of the golang.org/x/telemetry/config module
    golang/go#68358: a potential crash on Windows when the telemetry counter file is extended
    golang/go#68311: a potential hanging process if the telemetry file is truncated

Additionally, this release changes the gopls/telemetryprompt/accepted
counter to be recorded each time the prompt state is checked
(golang/go#68770).

None of these issues were particularly urgent, but we also wanted
to make a gopls release to exercise our recently added release
automation (golang/go#57643).
Pinkle-pash added a commit to Pinkle-pash/telemetry that referenced this issue Dec 4, 2024
@Pinkle-pash
internal/chartconfig: add collection for gopls/telemetryprompt/accepted
d7241f6 
To help evaluate the efficacy of prompting users to enable telemetry
from gopls, collect a counter that is incremented at gopls startup if
the user has accepted the telemetry prompt. This will help us estimate
what fraction of new telemetry participants are due to prompting. Of
course, we cannot know how many users to not accept the prompt.

Additional counters were added in CL 607758, but are not yet collected
as they were not covered in the proposal.

Fixes golang/go#68770

Change-Id: I8cb75a2399ffee8a15bfdb634b8200cb097e4ace
Reviewed-on: https://go-review.googlesource.com/c/telemetry/+/603936
Auto-Submit: Robert Findley <rfindley@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. telemetry x/telemetry issues Telemetry-Accepted Telemetry-Proposal
Projects
Telemetry Proposals
Status: No status
Milestone
gopls/v0.16.2
Development

No branches or pull requests
5 participants
@cagedmantis @hyangah @gopherbot @findleyr @gabyhelp