Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

Feature/#6557 Organic search telemetry #10167

Conversation

BranescuMihai
Copy link
Contributor

@BranescuMihai BranescuMihai commented Apr 23, 2020

Pull Request checklist

  • Tests: WIP - Edit: DONE!
  • Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
  • Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

After merge

  • Milestone: Make sure issues finished by this pull request are added to the milestone of the version currently in development.

To download an APK when reviewing a PR:

  1. click on Show All Checks,
  2. click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
  3. click on the "Fenix - assemble" task, then click "Run Artifacts".
  4. the APK links should be on the left side of the screen, named for each CPU architecture

@BranescuMihai BranescuMihai force-pushed the feature/#6557-organic_search_telemetry branch from 8978595 to c0f9f6b Compare April 24, 2020 16:52
@BranescuMihai
Copy link
Contributor Author

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
  • These probes will be used to understand what percentage of our searches come from organic searches versus standard browser access points.
  1. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?
  • To determine what volume of searches are not monetising via Firefox.
  1. What alternative methods did you consider to answer these questions? Why were they not sufficient?
  • No current alternative methods.
  1. Can current instrumentation answer these questions?
  • Currently there are no events to track this.
  1. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.
  • All data is Category 2
  1. How long will this data be collected?
  • Until 2020-09-01
  1. What populations will you measure?
  • All release, beta, and nightly users with telemetry enabled.
  1. Please provide a general description of how you will analyse this data.
  • We will analyse the volume differences between organic and SAPs through Glean / Amplitude
  1. Where do you intend to share the results of your analysis?
  • Only on Glean, Amplitude and with mobile teams.

@BranescuMihai BranescuMihai added the needs:data-review PR is awaiting a data review label Apr 24, 2020
@BranescuMihai BranescuMihai force-pushed the feature/#6557-organic_search_telemetry branch from c0f9f6b to 2fcb2c8 Compare April 24, 2020 17:51
Copy link
Contributor

@liuche liuche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would like a little more detail in the documentation before finishing data review.

Still needs code review from someone.

in_content:
type: labeled_counter
description: >
Records the type of interaction a user has on SERP pages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you include the types of data that this could be, or provide an example? That way, if someone from data science is looking at our documentation, they have a very clear idea of what kind of data this looks like.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this PR was merged already, I'll add a new task with the docs change

@@ -0,0 +1,32 @@
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: License?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSONs don't allow standard comments. Looked over in AC and none of the extension manifest files have license added, so I think we have to make exceptions for these files

@boek boek self-requested a review April 24, 2020 22:46
Copy link
Contributor

@boek boek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data Review Form (to be filled by Data Stewards)

Instructions: Data Stewards will review a request for data collection and endorse responses to each question.

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?
    Yes, with metrics.yaml and metrics.md

  2. Is there a control mechanism that allows the user to turn the data collection on and off?
    Yes, in the Fenix settings

  3. If the request is for permanent data collection, is there someone who will monitor the data over time?
    Has expiry, Fenix team will monitor

  4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?
    All data is category 2.

  5. Is the data collection request for default-on or default-off?
    Default on

  6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?
    No

  7. Is the data collection covered by the existing Firefox privacy notice?
    yes

  8. Does there need to be a check-in in the future to determine whether to renew the data?
    Has expiry. Fenix team will monitor

  9. Does the data collection use a third-party collection tool?
    No

@boek boek merged commit a16bfb2 into mozilla-mobile:master Apr 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
needs:data-review PR is awaiting a data review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants