Closes #18948: Update tabstray telemetry #19004

rocketsroger · 2021-04-13T20:18:12Z

Pull Request checklist

Tests: This PR includes thorough tests or an explanation of why it does not
Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

To download an APK when reviewing a PR:

click on Show All Checks,
click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
click on the "Fenix - assemble" task, then click "Run Artifacts".
the APK links should be on the left side of the screen, named for each CPU architecture

rocketsroger · 2021-04-13T20:31:54Z

Request for data collection review

What questions will you answer with this data?
Understanding of how frequently user uses Synced Tabs
Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? 
This allows us to understand Synced Tabs feature priority
What alternative methods did you consider to answer these questions? Why were they not sufficient? 
No alternative is available to answer this question.
Can current instrumentation answer these questions? 
No current instrumentation can answer this question.
List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurement Description	Data Collection Category	Tracking Bug #
User switched to Synced Tab mode	category 2 - interaction data	#18948

Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md
How long will this data be collected? Choose one of the following: 
This is scoped to a time-limited experiment/project until date 08-01-2022.
What populations will you measure? 
No filters - all locales, channels, etc.
If this data collection is default on, what is the opt-out mechanism for users? 
Default Glean SDK opt-out mechanism.
Please provide a general description of how you will analyze this data. 
Glean / Amplitude
Where do you intend to share the results of your analysis? 
Only on Glean, Amplitude and with mobile teams.
Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?
 No third-party tool will use this data.

jonalmeida

One question, but LGTM for a code review.

jonalmeida · 2021-04-13T20:49:00Z

app/src/main/java/org/mozilla/fenix/tabstray/TabsTrayFragment.kt

@@ -225,6 +227,11 @@ class TabsTrayFragment : AppCompatDialogFragment(), TabsTrayInteractor {
        )
    }

+    override fun onDestroyView() {
+        requireComponents.analytics.metrics.track(Event.TabsTrayClosed)


I'm not sure about adding this to onDestroyView. Would this be called even on non-user generated events? System wants to regain memory, for example.

Thinking about where else this could go, we could put it in the NavigationInteractor.onTabTrayDismissed

We can de-dupe the two callers dismissTabTray and dismissTabTrayAndNavigateHome, and make sure they both invoke the event.

Just a thought; haven't tested it out.

Agreed. I'll update. Thanks,

jonalmeida · 2021-04-13T21:36:07Z

app/src/main/java/org/mozilla/fenix/tabstray/TraySheetBehaviorCallback.kt

 ) : BottomSheetBehavior.BottomSheetCallback() {

    override fun onStateChanged(bottomSheet: View, newState: Int) {
        if (newState == STATE_HIDDEN) {
-            metrics.track(Event.TabsTrayClosed)


Nice catch!

rocketsroger · 2021-04-14T19:06:14Z

One test is stuck. Closing and reopening to rerun them.

travis79 · 2021-04-14T19:21:16Z

Request for data collection review

1. What questions will you answer with this data?
   Understanding of how frequently user uses Synced Tabs

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?    
   This allows us to understand Synced Tabs feature priority

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?    
   No alternative is available to answer this question.

4. Can current instrumentation answer these questions?    
   No current instrumentation can answer this question.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurement Description Data Collection Category Tracking Bug #
User switched to Synced Tab mode category 2 - interaction data #18948

1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
   [https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md](https://github.com/mozilla-mobile/fenix/blob/master/docs/metrics.md?rgh-link-date=2021-04-13T20%3A31%3A54Z)

2. How long will this data be collected? Choose one of the following:    
   This is scoped to a time-limited experiment/project until date 08-01-2022.

3. What populations will you measure?    
   No filters - all locales, channels, etc.

4. If this data collection is default on, what is the opt-out mechanism for users?    
   Default Glean SDK opt-out mechanism.

5. Please provide a general description of how you will analyze this data.    
   Glean / Amplitude

6. Where do you intend to share the results of your analysis?    
   Only on Glean, Amplitude and with mobile teams.

7. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?
       No third-party tool will use this data.

Data Review

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
Yes, through the metrics.md file and the Glean Dictionary.
Is there a control mechanism that allows the user to turn the data collection on and off?
Yes, via the "Send Usage Data" toggle in preferences.
If the request is for permanent data collection, is there someone who will monitor the data over time?
N/A. Collection set to end 2022-08-01.
Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
Category 2 - User Interaction.
Is the data collection request for default-on or default-off?
Default on.
Does the instrumentation include the addition of any new identifiers?
No.
Is the data collection covered by the existing Firefox privacy notice?
Yes.
Does there need to be a check-in in the future to determine whether to renew the data?
Collection scheduled to end 2022-08-01, product may review to extend collection before then.
Does the data collection use a third-party collection tool?
No.

Result

data-review+

travis79

Approval for data-review+ only, I didn't code-review this.

codecov-io · 2021-04-14T19:27:56Z

Codecov Report

Merging #19004 (5abe06d) into master (445091d) will increase coverage by 0.19%.
The diff coverage is 32.00%.

@@             Coverage Diff              @@
##             master   #19004      +/-   ##
============================================
+ Coverage     34.14%   34.34%   +0.19%     
- Complexity     1575     1576       +1     
============================================
  Files           535      535              
  Lines         21636    21653      +17     
  Branches       3222     3227       +5     
============================================
+ Hits           7387     7436      +49     
+ Misses        13352    13334      -18     
+ Partials        897      883      -14

Impacted Files	Coverage Δ	Complexity Δ
...a/org/mozilla/fenix/tabstray/TabsTrayController.kt	`0.00% <0.00%> (ø)`	`0.00 <0.00> (ø)`
...ava/org/mozilla/fenix/tabstray/TabsTrayFragment.kt	`0.00% <0.00%> (ø)`	`0.00 <0.00> (ø)`
...ozilla/fenix/tabstray/TabsTrayInfoBannerBinding.kt	`82.60% <0.00%> (-3.76%)`	`10.00 <0.00> (ø)`
...a/org/mozilla/fenix/tabstray/TabsTrayViewHolder.kt	`50.00% <0.00%> (-0.69%)`	`11.00 <0.00> (ø)`
...ozilla/fenix/tabstray/TraySheetBehaviorCallback.kt	`85.71% <ø> (-1.79%)`	`4.00 <0.00> (ø)`
...java/org/mozilla/fenix/components/metrics/Event.kt	`41.97% <100.00%> (+2.42%)`	`0.00 <0.00> (ø)`
...la/fenix/components/metrics/GleanMetricsService.kt	`26.41% <100.00%> (+4.79%)`	`5.00 <0.00> (ø)`
...va/org/mozilla/fenix/tabstray/TabLayoutMediator.kt	`85.71% <100.00%> (+14.88%)`	`5.00 <0.00> (ø)`
...components/toolbar/BrowserToolbarMenuController.kt	`58.29% <0.00%> (+0.50%)`	`0.00% <0.00%> (ø%)`
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 445091d...5abe06d. Read the comment docs.

rocketsroger added the needs:review PRs that need to be reviewed label Apr 13, 2021

rocketsroger requested a review from jonalmeida April 13, 2021 20:18

rocketsroger requested review from a team as code owners April 13, 2021 20:18

rocketsroger added the needs:data-review PR is awaiting a data review label Apr 13, 2021

rocketsroger force-pushed the FB_18948 branch from 3fe06c3 to 5f94116 Compare April 13, 2021 20:39

jonalmeida approved these changes Apr 13, 2021

View reviewed changes

jonalmeida removed the needs:review PRs that need to be reviewed label Apr 13, 2021

rocketsroger force-pushed the FB_18948 branch from 5f94116 to 2bf13a0 Compare April 14, 2021 14:50

Closes mozilla-mobile#18948: Update tabstray telemetry

5abe06d

rocketsroger force-pushed the FB_18948 branch from 2bf13a0 to 5abe06d Compare April 14, 2021 15:45

rocketsroger closed this Apr 14, 2021

rocketsroger reopened this Apr 14, 2021

travis79 removed the needs:data-review PR is awaiting a data review label Apr 14, 2021

travis79 approved these changes Apr 14, 2021

View reviewed changes

rocketsroger merged commit 8246f81 into mozilla-mobile:master Apr 14, 2021

rocketsroger deleted the FB_18948 branch April 14, 2021 19:32

eliserichards mentioned this pull request Jul 27, 2021

For #20516: Renew product telemetry probes expiring in August #20517

Merged

3 tasks

mdboom mentioned this pull request Aug 26, 2021

For #21025: Set long term business-critical metrics as non-expiring #21038

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Closes #18948: Update tabstray telemetry #19004

Closes #18948: Update tabstray telemetry #19004

rocketsroger commented Apr 13, 2021

rocketsroger commented Apr 13, 2021

jonalmeida left a comment

jonalmeida Apr 13, 2021

rocketsroger Apr 13, 2021

jonalmeida Apr 13, 2021

rocketsroger commented Apr 14, 2021

travis79 commented Apr 14, 2021

Request for data collection review

travis79 left a comment

codecov-io commented Apr 14, 2021

Closes #18948: Update tabstray telemetry #19004

Closes #18948: Update tabstray telemetry #19004

Conversation

rocketsroger commented Apr 13, 2021

Pull Request checklist

To download an APK when reviewing a PR:

rocketsroger commented Apr 13, 2021

Request for data collection review

jonalmeida left a comment

Choose a reason for hiding this comment

jonalmeida Apr 13, 2021

Choose a reason for hiding this comment

rocketsroger Apr 13, 2021

Choose a reason for hiding this comment

jonalmeida Apr 13, 2021

Choose a reason for hiding this comment

rocketsroger commented Apr 14, 2021

travis79 commented Apr 14, 2021

Request for data collection review

Data Review

Result

travis79 left a comment

Choose a reason for hiding this comment

codecov-io commented Apr 14, 2021

Codecov Report