Add Enroll/Unenroll Failed events to Nimbus

[travis79]

No description provided.

[codecov-commenter]

Codecov Report

Base: 39.00% // Head: 39.06% // Increases project coverage by +0.06% 🎉

Coverage data is based on head (4c04f2d) compared to base (37a38e4).
Patch coverage: 75.00% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5137      +/-   ##
==========================================
+ Coverage   39.00%   39.06%   +0.06%     
==========================================
  Files         169      169              
  Lines       12461    12486      +25     
==========================================
+ Hits         4860     4878      +18     
- Misses       7601     7608       +7     

Impacted Files	Coverage Δ
components/nimbus/src/enrollment.rs	93.80% <69.69%> (-1.07%)	⬇️
components/nimbus/src/tests/test_enrollment.rs	97.42% <100.00%> (+<0.01%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[travis79]

Request for data collection review form

1. What questions will you answer with this data?

The purpose of these events are for Nimbus monitoring to answer questions about failure to enroll or unenroll.

1. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

These metrics are important for monitoring of Nimbus experiments, to ensure proper experiment execution, and to be equivalent to the metrics collected in Firefox Desktop.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

No other telemetry was available to answer these questions.

4. Can current instrumentation answer these questions?

No

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Enroll Failed	Category 1 - Technical Data	[Jira EXP-2354](https://mozilla-hub.atlassian.net/browse/EXP-2534)
Unenroll Failed	Category 1 - Technical Data	[Jira EXP-2354](https://mozilla-hub.atlassian.net/browse/EXP-2534)

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

This collection is documented in the Glean Dictionary and in the metrics.yaml

7. How long will this data be collected? Choose one of the following:

I want to permanently monitor this data - Travis Long (tlong@mozilla.com)

8. What populations will you measure?

All channels, countries, and locales

9. If this data collection is default on, what is the opt-out mechanism for users?

Through the standard telemetry preference in the application settings

10. Please provide a general description of how you will analyze this data.

Through Jetstream analysis of Nimbus experiments and through Looker.

11. Where do you intend to share the results of your analysis?

With experiment team members and experiment owners

12. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?

No

cc @eliserichards for data-review?

[jhugman]

A drive by. All looking good so far; @jeddai has said everything that I would've.

[travis79]

This is still awaiting data-review, and could conflict with my plan to land #5009 this week. Landing the other PR takes priority over this one, since it would be more difficult to modify #5009 after this lands than vice versa. So, while this awaits data-review, I'll land the other PR and then update this one to record these events in Rust for iOS.

[eliserichards]

Request for data collection review form

1. What questions will you answer with this data?

The purpose of these events are for Nimbus monitoring to answer questions about failure to enroll or unenroll.

1. Why does Mozilla need to answer these questions?  Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

These metrics are important for monitoring of Nimbus experiments, to ensure proper experiment execution, and to be equivalent to the metrics collected in Firefox Desktop.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

No other telemetry was available to answer these questions.

4. Can current instrumentation answer these questions?

No

5. List all proposed measurements and indicate the category of data collection for each measurement, using the [Firefox data collection categories](https://wiki.mozilla.org/Data_Collection) found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.
Enroll Failed Category 1 - Technical Data Jira EXP-2354
Unenroll Failed Category 1 - Technical Data Jira EXP-2354

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

This collection is documented in the Glean Dictionary and in the metrics.yaml

7. How long will this data be collected?  Choose one of the following:

I want to permanently monitor this data - Travis Long (tlong@mozilla.com)

8. What populations will you measure?

All channels, countries, and locales

9. If this data collection is default on, what is the opt-out mechanism for users?

Through the standard telemetry preference in the application settings

10. Please provide a general description of how you will analyze this data.

Through Jetstream analysis of Nimbus experiments and through Looker.

11. Where do you intend to share the results of your analysis?

With experiment team members and experiment owners

12. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?

No

Data Review Form (to be filled by Data Stewards)

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

   • Yes, through the metrics.yaml file and the Glean Dictionary.

2. Is there a control mechanism that allows the user to turn the data collection on and off? (Note, for data collection not needed for security purposes, Mozilla provides such a control mechanism) Provide details as to the control mechanism available.

   • Yes, through the standard telemetry preference in the application settings.

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

   • Yes, Travis Long (tlong@mozilla.com) will monitor this data.

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

   • Category 1 - technical data

5. Is the data collection request for default-on or default-off?

   • Default-on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

   • No

7. Is the data collection covered by the existing Firefox privacy notice?

   • Yes

8. Does the data collection use a third-party collection tool? If yes, escalate to legal.

   • No

Result

data-review+

[eliserichards]

data-review+