Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StepperTracking: Investigate Experiment Setup #96725

Open
escapemanuele opened this issue Nov 25, 2024 · 2 comments
Open

StepperTracking: Investigate Experiment Setup #96725

escapemanuele opened this issue Nov 25, 2024 · 2 comments
Labels
Customer Report Issues or PRs that were reported via Happiness. Previously known as "Happiness Request". [Experiment] AI labels added [Feature] API [Feature Group] Other Infrastructure Miscellaneous infrastructure features and improvements on WordPress.com. [Feature] Tracking Metrics & Monitoring

Comments

@escapemanuele
Copy link
Contributor

escapemanuele commented Nov 25, 2024 •
edited
Loading

Experiment Setup (Assignment Mechanism)

  • We checked whether the participants in the ExPlat’s exposure strategy actually had exposure events after they were assigned. The control had a lot more participants lacking an exposure event post-assignment (43,635 vs. 2,678). This suggests that many unintended participants were included in the control group in the ExPlat’s exposure strategy, indicating a potential issue with the experiment setup. More details are available pbmxuV-3Lt-p2#comment-4882.
  • I am not sure how we can investigate this as the experiment has stopped, but it would be helpful if we could validate the implemented experiment assignment mechanism.
@github-actions GitHub Actions
Copy link

OpenAI suggested the following labels for this issue:

  • [Feature Group] Other Infrastructure: The issue highlights a potential problem with the experiment setup, suggesting infrastructure-related concerns.
  • [Feature] Tracking Metrics & Monitoring: The issue discusses the monitoring of participants in the exposure strategy, which involves tracking metrics related to user engagement.
  • [Feature] API: Validating the implemented experiment assignment mechanism may require API interactions to investigate how assignments were handled.

@github-actions GitHub Actions
Copy link

Support References

This comment is automatically generated. Please do not edit it.

  • pbmxuV-3Lt-p2#comment-4882

@github-actions github-actions bot added the Customer Report Issues or PRs that were reported via Happiness. Previously known as "Happiness Request". label Nov 25, 2024
@escapemanuele escapemanuele changed the title StepperTracking: StepperTracking: Investigate Experiment Setup Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Customer Report Issues or PRs that were reported via Happiness. Previously known as "Happiness Request". [Experiment] AI labels added [Feature] API [Feature Group] Other Infrastructure Miscellaneous infrastructure features and improvements on WordPress.com. [Feature] Tracking Metrics & Monitoring
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@escapemanuele