For #22147 - New search term groups telemetry #22368

Mugurell · 2021-11-08T15:43:01Z

3 new events added for when the user opens a tab, closes one or closes all from a search group.
Kimmy to confirm if the current https://dictionary.telemetry.mozilla.org/apps/fenix/metrics/history_search_term_group_tapped is okay to be considered the 4th we needed.

Pull Request checklist

Tests: This PR includes thorough tests or an explanation of why it does not
Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

To download an APK when reviewing a PR:

click on Show All Checks,
click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
click on the "Fenix - assemble" task, then click "Run Artifacts".
the APK links should be on the left side of the screen, named for each CPU architecture

Mugurell · 2021-11-08T15:47:15Z

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

What questions will you answer with this data?

This data will help us understand how users interact with new search groups from history.

Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

Better asses feature importance.

What alternative methods did you consider to answer these questions? Why were they not sufficient?

There are no other alternatives.

Can current instrumentation answer these questions?

No.

List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki. 
Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Description	Data Collection Category	Tracking Bug #
When a tab from the search term group is opened	Category 2 - interaction data	#22147
When a tab from the search term group is closed	Category 2 - interaction data	#22147
When all tabs from the search term group are closed	Category 2 - interaction data	#22147

Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

https://dictionary.telemetry.mozilla.org/apps/fenix

How long will this data be collected? Choose one of the following:

One year (ending on November 1st, 2022), with the option to renew at that point.

What populations will you measure?

All channels, all locales, all countries

If this data collection is default on, what is the opt-out mechanism for users?

Default Glean SDK opt-out mechanism.

Please provide a general description of how you will analyze this data.

Glean and Amplitude.

Where do you intend to share the results of your analysis?

 Only on Glean, Amplitude, and with mobile teams.

Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?

No third-party tools.

jonalmeida

eng r+

jonalmeida · 2021-11-08T19:38:36Z

app/metrics.yaml

+    bugs:
+      - https://github.com/mozilla-mobile/fenix/issues/22147
+    data_reviews:
+      - ????????????


jonalmeida · 2021-11-08T19:43:34Z

app/metrics.yaml

+  search_term_group_open_tab:
+    type: event
+    description: |
+      A user opens a tab from the search term group.


From reading these descriptions, I thought these were probes for the tabs tray itself. Might be good to clarify the location where these probes are being added. For example..

Suggested change

A user opens a tab from the search term group.

A user opens a tab from the search term group from the history list.

(same for the other two probes)

Agreed. Looks like these are for History. FYI, I recently add one telemetry for this requirement #22300

rocketsroger

Please move these to the history category. Also double check the PR I linked. Thanks,

rocketsroger · 2021-11-08T19:49:24Z

app/metrics.yaml

+  search_term_group_open_tab:
+    type: event
+    description: |
+      A user opens a tab from the search term group.


Agreed. Looks like these are for History. FYI, I recently add one telemetry for this requirement #22300

Mugurell · 2021-11-09T08:04:08Z

Please move these to the history category. Also double check the PR I linked. Thanks,

These new probes are already in the history category and use the naming scheme from #22300, them being recorded as

        {
          "timestamp": 10862,
          "category": "history",
          "name": "search_term_group_tapped"
        },
        {
          "timestamp": 14257,
          "category": "history",
          "name": "search_term_group_open_tab"
        },
                {
          "timestamp": 20389,
          "category": "history",
          "name": "search_term_group_remove_tab"
        },
        {
          "timestamp": 21755,
          "category": "history",
          "name": "search_term_group_remove_all"
        }

so in Glean similarly to https://dictionary.telemetry.mozilla.org/apps/fenix/metrics/history_search_term_group_tapped the 3 events will actually be:

- history.search_term_group_open_tab
- history.search_term_group_remove_tab
- history.search_term_group_remove_all

Will add the history reference in the probe description and also add some tests.

rocketsroger · 2021-11-09T14:51:10Z

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?


* This data will help us understand how users interact with new search groups from history.


1. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?


* Better asses feature importance.


1. What alternative methods did you consider to answer these questions? Why were they not sufficient?


* There are no other alternatives.


1. Can current instrumentation answer these questions?


* No.


1. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.    
   Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Description Data Collection Category Tracking Bug #
When a tab from the search term group is opened Category 2 - interaction data #22147
When a tab from the search term group is closed Category 2 - interaction data #22147
When all tabs from the search term group are closed Category 2 - interaction data #22147

1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.


* [https://dictionary.telemetry.mozilla.org/apps/fenix   ](https://dictionary.telemetry.mozilla.org/apps/fenix%E2%80%A8)


1. How long will this data be collected? Choose one of the following:


* One year (ending on November 1st, 2022), with the option to renew at that point.


1. What populations will you measure?


* All channels, all locales, all countries


1. If this data collection is default on, what is the opt-out mechanism for users?


* Default Glean SDK opt-out mechanism.


1. Please provide a general description of how you will analyze this data.


* Glean and Amplitude.


1. Where do you intend to share the results of your analysis?


* Only on Glean, Amplitude, and with mobile teams.


1. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?


* No third-party tools.

Data Review

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the "Send Usage Data" preference in the application settings

If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A, collection set to end or be renewed by 2022-11-01

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction data

Is the data collection request for default-on or default-off?

default-on

Does the instrumentation include the addition of any new identifiers?

No

Is the data collection covered by the existing Firefox privacy notice?

Yes

Does the data collection use a third-party collection tool?

No

Result

data-review+

rocketsroger · 2021-11-09T14:52:16Z

Looks good. I've data reviewed. Please update the data-review link in metrics.yml. Thanks

Mugurell · 2021-11-09T15:08:28Z

Thank you Roger!
Since I've also added some tests and updated that documentation I'll wait a bit for @jonalmeida to maybe have a second quick look, if everything seems okay.

Mugurell · 2021-11-09T16:30:31Z

Kimmy said will also leave a comment on whether these events look right or not.
Adding the do-not-land label in the interim.

Mugurell · 2021-11-16T06:28:48Z

Kimmy confirmed these events look good.

Mugurell · 2021-11-16T10:24:31Z

Ohh I see @rocketsroger requested changes and the merge is blocked.
Asking for a re-review.

Mugurell requested review from a team as code owners November 8, 2021 15:43

Mugurell added needs:data-review PR is awaiting a data review needs:review PRs that need to be reviewed labels Nov 8, 2021

jonalmeida approved these changes Nov 8, 2021

View reviewed changes

rocketsroger suggested changes Nov 8, 2021

View reviewed changes

rocketsroger removed the needs:data-review PR is awaiting a data review label Nov 9, 2021

For #22147 - New search term groups telemetry

22ea021

jonalmeida approved these changes Nov 9, 2021

View reviewed changes

Mugurell added pr:do-not-land and removed needs:review PRs that need to be reviewed labels Nov 9, 2021

rocketsroger self-requested a review November 10, 2021 17:15

Mugurell added pr:needs-landing PRs that are ready to land [Will be merged by Mergify] and removed pr:do-not-land labels Nov 16, 2021

Mugurell added needs:review PRs that need to be reviewed and removed pr:needs-landing PRs that are ready to land [Will be merged by Mergify] labels Nov 16, 2021

rocketsroger approved these changes Nov 16, 2021

View reviewed changes

rocketsroger removed the needs:review PRs that need to be reviewed label Nov 16, 2021

Mugurell added the pr:needs-landing PRs that are ready to land [Will be merged by Mergify] label Nov 16, 2021

Merge branch 'main' into searchTermGroupsTelemetry

1f2a7ec

mergify bot merged commit 17865bd into mozilla-mobile:main Nov 16, 2021

Mugurell deleted the searchTermGroupsTelemetry branch November 16, 2021 16:40

gabrielluong mentioned this pull request Aug 22, 2022

For #26582 - Renew telemetry expiring in 107 #26583

Closed

4 tasks

MatthewTighe mentioned this pull request Sep 19, 2022

For #26582 - Renew telemetry expiring in 107 #27068

Merged

4 tasks

rocketsroger mentioned this pull request Aug 17, 2023

Bug 1849184 - Renew Fenix telemetry probes expiring in v120 mozilla-mobile/firefox-android#3319

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

For #22147 - New search term groups telemetry #22368

For #22147 - New search term groups telemetry #22368

Mugurell commented Nov 8, 2021

Mugurell commented Nov 8, 2021

jonalmeida left a comment

jonalmeida Nov 8, 2021

jonalmeida Nov 8, 2021 •

edited

Loading

rocketsroger Nov 8, 2021

rocketsroger left a comment

rocketsroger Nov 8, 2021

Mugurell commented Nov 9, 2021

rocketsroger commented Nov 9, 2021

Request for data collection review form

rocketsroger commented Nov 9, 2021

Mugurell commented Nov 9, 2021

Mugurell commented Nov 9, 2021

Mugurell commented Nov 16, 2021

Mugurell commented Nov 16, 2021

	A user opens a tab from the search term group.
	A user opens a tab from the search term group from the history list.

For #22147 - New search term groups telemetry #22368

For #22147 - New search term groups telemetry #22368

Conversation

Mugurell commented Nov 8, 2021

Pull Request checklist

To download an APK when reviewing a PR:

Mugurell commented Nov 8, 2021

Request for data collection review form

jonalmeida left a comment

Choose a reason for hiding this comment

jonalmeida Nov 8, 2021

Choose a reason for hiding this comment

jonalmeida Nov 8, 2021 • edited Loading

Choose a reason for hiding this comment

rocketsroger Nov 8, 2021

Choose a reason for hiding this comment

rocketsroger left a comment

Choose a reason for hiding this comment

rocketsroger Nov 8, 2021

Choose a reason for hiding this comment

Mugurell commented Nov 9, 2021

rocketsroger commented Nov 9, 2021

Request for data collection review form

Data Review

Result

rocketsroger commented Nov 9, 2021

Mugurell commented Nov 9, 2021

Mugurell commented Nov 9, 2021

Mugurell commented Nov 16, 2021

Mugurell commented Nov 16, 2021

jonalmeida Nov 8, 2021 •

edited

Loading