For #26551 - Add telemetry for MR Onboarding welcome and sync dialogs. #26596

Alexandru2909 · 2022-08-23T10:14:39Z

Added telemetry for MR onboarding welcome and sync dialogs.

Pull Request checklist

Tests: This PR includes thorough tests or an explanation of why it does not
Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

QA

QA Needed

To download an APK when reviewing a PR (after all CI tasks finished running):

Click on Checks at the top of the PR page.
Click on the firefoxci-taskcluster group on the left to expand all tasks.
Click on the build-debug task.
Click on View task in Taskcluster in the new DETAILS section.
The APK links should be on the right side of the screen, named for each CPU architecture.

GitHub Automation

Fixes #26551

Alexandru2909 · 2022-08-23T10:20:53Z

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

What questions will you answer with this data?

How many users saw the MR welcome onboarding card?
How many users dismissed the MR welcome onboarding card?
How many users interacted with the `Get started` button from the welcome onboarding card?
How many users saw the MR sync onboarding card?
How many users dismissed the MR sync onboarding card?
How many users interacted with the `Sign in` button from the MR sync onboarding card?
How many users interacted with the `Skip` button from the MR sync onboarding card?

Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

We need to count the MR onboarding impressions and interactions with the MR onboarding screens to understand how many users are reminded of the value Firefox is providing and how they decide to interact with the onboarding dialogs.

What alternative methods did you consider to answer these questions? Why were they not sufficient?

No other alternatives.

Can current instrumentation answer these questions?

No.

List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki. 
Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Description	Data Collection Category	Tracking Bug #
Welcome card impression: The welcome onboarding card was displayed to the user.	Category 2 - interaction data	#26551
Welcome close clicked: The user clicked the `X` button from the welcome onboarding card.	Category 2 - interaction data	#26551
Welcome get started clicked: The user clicked the `Get started` button from the welcome onboarding card.	Category 2 - interaction data	#26551
Sync card impression: The sync sign in onboarding card was displayed to the user.	Category 2 - interaction data	#26551
Sync close clicked: The user clicked the `X` button from the sync sign in onboarding card.	Category 2 - interaction data	#26551
Sync sign in clicked: The user clicked the `Sign in` button from the sync sign in onboarding card.	Category 2 - interaction data	#26551
Sync skip clicked: The user clicked on the `Skip` button from the sign in onboarding card.	Category 2 - interaction data	#26551

Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

https://dictionary.telemetry.mozilla.org/apps/fenix

How long will this data be collected? Choose one of the following:

Up until version 119, with the option to renew at that point.

What populations will you measure?

All channels, all locales, all countries.

If this data collection is default on, what is the opt-out mechanism for users?

Default Glean SDK opt-out mechanism.

Please provide a general description of how you will analyze this data.

Glean.

Where do you intend to share the results of your analysis?

 Only on Glean and with mobile teams.

Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?

No third-party tools.

mergify · 2022-08-25T18:24:24Z

This pull request has conflicts when rebasing. Could you fix it @Alexandru2909? 🙏

…d sync dialogs.

MozillaNoah · 2022-08-29T18:32:11Z

app/src/main/java/org/mozilla/fenix/onboarding/view/Onboarding.kt

+            IconButton(
+                onClick = {
+                    if (onboardingState == OnboardingState.Welcome) {
+                        OnboardingMetrics.welcomeCloseClicked.record(NoExtras())


I'm shocked that the Glean metrics don't break the Compose previews lol. I feel the metrics could live within HomeOnboardingDialogFragment though. @gabrielluong, do you have a strong opinion on this?

I think this is fine for now. I would want to look at refactoring some of this and make use of an Interactor so that we can test this, but that can be a followup after this lands.

gabrielluong

Code looks good to me, but we'll need a data review

travis79 · 2022-08-29T21:11:02Z

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.
1. What questions will you answer with this data?


* How many users saw the MR welcome onboarding card?

* How many users dismissed the MR welcome onboarding card?

* How many users interacted with the `Get started` button from the welcome onboarding card?

* How many users saw the MR sync onboarding card?

* How many users dismissed the MR sync onboarding card?

* How many users interacted with the `Sign in` button from the MR sync onboarding card?

* How many users interacted with the `Skip` button from the MR sync onboarding card?


2. Why does Mozilla need to answer these questions? Are there benefits 
   for users? Do we need this information to address product or business 
   requirements?


* We need to count the MR onboarding impressions and interactions with the MR onboarding screens to understand how many users are reminded of the value Firefox is providing and how they decide to interact with the onboarding dialogs.


3. What alternative methods did you consider to answer these questions? Why were they not sufficient?


* No other alternatives.


4. Can current instrumentation answer these questions?


* No.


5. List all proposed measurements and indicate the category of data 
   collection for each measurement, using the Firefox data collection 
   categories found on the Mozilla wiki.    
   Note that the data steward reviewing your request will characterize your
    data collection based on the highest (and most sensitive) category.
Measurement Description Data Collection Category Tracking Bug #
Welcome card impression: The welcome onboarding card was displayed to the user. Category 2 - interaction data #26551
Welcome close clicked: The user clicked the X button from the welcome onboarding card. Category 2 - interaction data #26551
Welcome get started clicked: The user clicked the Get started button from the welcome onboarding card. Category 2 - interaction data #26551
Sync card impression: The sync sign in onboarding card was displayed to the user. Category 2 - interaction data #26551
Sync close clicked: The user clicked the X button from the sync sign in onboarding card. Category 2 - interaction data #26551
Sync sign in clicked: The user clicked the Sign in button from the sync sign in onboarding card. Category 2 - interaction data #26551
Sync skip clicked: The user clicked on the Skip button from the sign in onboarding card. Category 2 - interaction data #26551
6. Please provide a link to the documentation for this data collection 
   which describes the ultimate data set in a public, complete, and 
   accurate way.


* [dictionary.telemetry.mozilla.org/apps/fenix](https://dictionary.telemetry.mozilla.org/apps/fenix)


7. How long will this data be collected? Choose one of the following:


* Up until version 119, with the option to renew at that point.


8. What populations will you measure?


* All channels, all locales, all countries.


9. If this data collection is default on, what is the opt-out mechanism for users?


* Default Glean SDK opt-out mechanism.


10. Please provide a general description of how you will analyze this data.


* Glean.


11. Where do you intend to share the results of your analysis?


* Only on Glean and with mobile teams.


12. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?


* No third-party tools.

Data Review

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the "Send Usage Data" preference in the application settings.

If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A, collection to end by version 119

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction data

Is the data collection request for default-on or default-off?

Default-on

Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

Is the data collection covered by the existing Firefox privacy notice?

Yes

Does the data collection use a third-party collection tool?

No

Result

data-review+

travis79

data-review+

gabrielluong added the pr:work-in-progress PRs that are not ready to be reviewed yet and are actively being worked on label Aug 23, 2022

gabrielluong self-requested a review August 23, 2022 17:00

gabrielluong self-assigned this Aug 23, 2022

Alexandru2909 force-pushed the 26551 branch from 8bef183 to 03a458b Compare August 26, 2022 06:16

For mozilla-mobile#26551 - Add telemetry for MR Onboarding welcome an…

e62ef2b

…d sync dialogs.

Alexandru2909 force-pushed the 26551 branch from 03a458b to e62ef2b Compare August 26, 2022 06:26

Alexandru2909 added needs:data-review PR is awaiting a data review and removed pr:work-in-progress PRs that are not ready to be reviewed yet and are actively being worked on labels Aug 26, 2022

Alexandru2909 marked this pull request as ready for review August 26, 2022 06:27

Alexandru2909 requested review from a team as code owners August 26, 2022 06:27

gabrielluong requested a review from rocketsroger August 29, 2022 16:31

gabrielluong added the needs:review PRs that need to be reviewed label Aug 29, 2022

MozillaNoah reviewed Aug 29, 2022

View reviewed changes

gabrielluong approved these changes Aug 29, 2022

View reviewed changes

gabrielluong added pr:approved PR that has been approved and removed needs:review PRs that need to be reviewed labels Aug 29, 2022

gabrielluong requested a review from travis79 August 29, 2022 20:36

travis79 approved these changes Aug 29, 2022

View reviewed changes

travis79 removed the needs:data-review PR is awaiting a data review label Aug 29, 2022

Alexandru2909 added the pr:needs-landing PRs that are ready to land [Will be merged by Mergify] label Aug 30, 2022

Merge branch 'main' into 26551

a77fcbd

mergify bot merged commit 22b8b87 into mozilla-mobile:main Aug 30, 2022

rocketsroger mentioned this pull request Aug 17, 2023

Bug 1849184 - Renew Fenix telemetry probes expiring in v120 mozilla-mobile/firefox-android#3319

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

For #26551 - Add telemetry for MR Onboarding welcome and sync dialogs. #26596

For #26551 - Add telemetry for MR Onboarding welcome and sync dialogs. #26596

Alexandru2909 commented Aug 23, 2022 •

edited by github-actions bot

Loading

Alexandru2909 commented Aug 23, 2022 •

edited

Loading

mergify bot commented Aug 25, 2022

MozillaNoah Aug 29, 2022

gabrielluong Aug 29, 2022 •

edited

Loading

gabrielluong left a comment

travis79 commented Aug 29, 2022

Request for data collection review form

travis79 left a comment

For #26551 - Add telemetry for MR Onboarding welcome and sync dialogs. #26596

For #26551 - Add telemetry for MR Onboarding welcome and sync dialogs. #26596

Conversation

Alexandru2909 commented Aug 23, 2022 • edited by github-actions bot Loading

Pull Request checklist

QA

To download an APK when reviewing a PR (after all CI tasks finished running):

GitHub Automation

Alexandru2909 commented Aug 23, 2022 • edited Loading

Request for data collection review form

mergify bot commented Aug 25, 2022

MozillaNoah Aug 29, 2022

Choose a reason for hiding this comment

gabrielluong Aug 29, 2022 • edited Loading

Choose a reason for hiding this comment

gabrielluong left a comment

Choose a reason for hiding this comment

travis79 commented Aug 29, 2022

Request for data collection review form

Data Review

Result

travis79 left a comment

Choose a reason for hiding this comment

Alexandru2909 commented Aug 23, 2022 •

edited by github-actions bot

Loading

Alexandru2909 commented Aug 23, 2022 •

edited

Loading

gabrielluong Aug 29, 2022 •

edited

Loading