Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

For #15367 - DownloadsFragment telemetry #16728

Merged
merged 1 commit into from
Dec 15, 2020
Merged

For #15367 - DownloadsFragment telemetry #16728

merged 1 commit into from
Dec 15, 2020

Conversation

Mugurell
Copy link
Contributor

Adds a counter for how many times the user does the following action:

  • opens the Downloads section inside the app
  • tap to open an item from inside Downloads
  • tap to delete one or more downloads at once

Pull Request checklist

  • Tests: No tests. Simple telemetry added.
  • Screenshots: No screenshots. No visual indicative of the change.
  • Accessibility: The code in this PR does not include any a11y user facing features.

To download an APK when reviewing a PR:

  1. click on Show All Checks,
  2. click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
  3. click on the "Fenix - assemble" task, then click "Run Artifacts".
  4. the APK links should be on the left side of the screen, named for each CPU architecture

@Mugurell Mugurell requested review from a team as code owners November 24, 2020 10:53
@Mugurell
Copy link
Contributor Author

Mugurell commented Nov 24, 2020

@kglazko I have some questions regarding this:

  • what should be the expiration date? I just put one year from now.
  • what should be the lifetime of this counter?: ping (default, currently used) / user / lifetime
  • I understood the ticket as only referring to DownloadFragment.
    Opening a downloaded item is also possible from the "Download completed" dialog. Should this ticket also track that event?
  • Regarding downloads deletion this patch increments the counter for every "delete download(s)" action without further granularity. Is this ok or should we also have different counters for "delete all" / "delete multiple"?
  • Deleting downloads also allows for undo. In which case the user selected an option to delete (which we track) but if he undo the action the file is not actually removed. This boils down to whether the intent for this was to track the user actions or their effect.

@codecov-io
Copy link

codecov-io commented Nov 24, 2020

Codecov Report

Merging #16728 (79c96cf) into master (d5b33ae) will decrease coverage by 0.02%.
The diff coverage is 0.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #16728      +/-   ##
============================================
- Coverage     30.82%   30.79%   -0.03%     
  Complexity     1240     1240              
============================================
  Files           455      455              
  Lines         18582    18599      +17     
  Branches       2597     2601       +4     
============================================
  Hits           5728     5728              
- Misses        12395    12412      +17     
  Partials        459      459              
Impacted Files Coverage Δ Complexity Δ
...a/org/mozilla/fenix/browser/BaseBrowserFragment.kt 9.81% <0.00%> (-0.04%) 14.00 <0.00> (ø)
...java/org/mozilla/fenix/components/metrics/Event.kt 33.87% <0.00%> (-0.34%) 0.00 <0.00> (ø)
...la/fenix/components/metrics/GleanMetricsService.kt 12.57% <0.00%> (-0.15%) 5.00 <0.00> (ø)
...g/mozilla/fenix/downloads/DynamicDownloadDialog.kt 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
...ozilla/fenix/library/downloads/DownloadFragment.kt 9.52% <0.00%> (-0.40%) 1.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5b33ae...79c96cf. Read the comment docs.

@gabrielluong gabrielluong added the needs:review PRs that need to be reviewed label Nov 26, 2020
@Amejia481 Amejia481 self-assigned this Nov 30, 2020
@Amejia481
Copy link
Contributor

  • what should be the expiration date? I just put one year from now.
  • what should be the lifetime of this counter?: ping (default, currently used) / user / lifetime

I think we could use that same default as other views like history and bookmarks.

I understood the ticket as only referring to DownloadFragment.
Opening a downloaded item is also possible from the "Download completed" dialog. Should this ticket also track that event?

As #15367 indicates "How often users (and what % of users) tap on a downloaded file in the browser to open it?" I think we should add it on the dialog if it's not already added.

  • Regarding downloads deletion this patch increments the counter for every "delete download(s)" action without further granularity. Is this ok or should we also have different counters for "download all" / "download multiple"?

Having granularity could help for product taking more accurate decisions, if it's not much harder to do, I think it could be good

@Mugurell
Copy link
Contributor Author

Mugurell commented Dec 4, 2020

Another question for @kglazko or @boek
Looking into what is being logged for this new counters if the user hasn't made an action to increment them
I see that nothing is being logged for that specific probe - not "0" as a counter value.
Is this ok? Or should we add other boolean probes that would track something like "download_has_been_opened" { true / false } like it was suggested here - #9556 (review)
(And I assume this would apply to all recent counter metrics)

@Mugurell
Copy link
Contributor Author

Updated the patch following Kate's and Arturo's review.
Will use Events instead of counters, also track the items opened from the download dialog and use the same expiration date as for other similar features.

@Amejia481 can you please review the code?
I'll also ask @boek to review the data collection form.

@Mugurell
Copy link
Contributor Author

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1) What questions will you answer with this data?
How often do users go to the new Downloads screen and what actions do they perform there.

2) Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?
This is important for appropriately prioritizing enhancing the download experience in relation to other features.

3) What alternative methods did you consider to answer these questions? Why were they not sufficient?
There is already telemetry for other download related events but nothing for this new screen inside the app.
This is the only way to also gather insights about the new feature.

4) Can current instrumentation answer these questions?
No. See the above.

5) List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Description Data Collection Category Tracking Bug #
Count started downloads Category 2 https://github.com//issues/15367

6) Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
This collection is documented in the metrics.md document document included in the project - https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md

7) How long will this data be collected? Choose one of the following:
One year, until 2021-04-01.
Pending renewal.

8) What populations will you measure?
All release channels and locales.

9) If this data collection is default on, what is the opt-out mechanism for users?
Users can opt of of data collection by disabling Usage and technical data from Settings -> Data collection.

10) Please provide a general description of how you will analyze this data.
Glean / Amplitude

11) Where do you intend to share the results of your analysis?
Only on Glean, Amplitude and with mobile teams.

12) Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?
No.

@Mugurell Mugurell added the needs:data-review PR is awaiting a data review label Dec 11, 2020
Copy link
Contributor

@Amejia481 Amejia481 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job 👍🏽 !

Copy link
Contributor

@boek boek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data Review Form (to be filled by Data Stewards)

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
    Yes, metrics.yaml and metrics.md

  2. Is there a control mechanism that allows the user to turn the data collection on and off?
    Yes, Fenix data collection settings.

  3. If the request is for permanent data collection, is there someone who will monitor the data over time?
    No, Fenix team will monitor

  4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
    Type 2

  5. Is the data collection request for default-on or default-off?
    Default on

  6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?
    No

  7. Is the data collection covered by the existing Firefox privacy notice?
    Yes

  8. Does there need to be a check-in in the future to determine whether to renew the data?
    Has expiry

  9. Does the data collection use a third-party collection tool?
    No

@mergify
Copy link
Contributor

mergify bot commented Dec 15, 2020

This pull request has conflicts when rebasing. Could you fix it @Mugurell? 🙏

Adds a counter for how many times the user does the following action:
- opens the Downloads section inside the app
- tap to open an item from inside Downloads / from the download dialog
- tap to delete one or more downloads at once
@Mugurell Mugurell merged commit 574eac4 into mozilla-mobile:master Dec 15, 2020
@Mugurell Mugurell deleted the 15367DownloadsTelemetry branch December 15, 2020 07:14
pkirakosyan pushed a commit to gexsi/user-agent-android that referenced this pull request Aug 4, 2021
…e#16728)

Adds a counter for how many times the user does the following action:
- opens the Downloads section inside the app
- tap to open an item from inside Downloads / from the download dialog
- tap to delete one or more downloads at once
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
needs:data-review PR is awaiting a data review needs:review PRs that need to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants