forked from DonJayamanne/pythonVSCode
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add A/A testing #6793
Labels
feature-request
Request for new features or functionality
Milestone
Comments
luabud
added
feature-request
Request for new features or functionality
triage-needed
Needs assignment to the proper sub-team
labels
Jul 31, 2019
Just to be clear, we want to separate users who are not in any A/B experiments between experiment and control groups so we can see if our sampling method is being fair. We want to make sure that there's no significant change in the behavior in each group /cc @qubitron |
Size - 10% |
brettcannon
modified the milestones:
September sprint 2019.1,
2019 - September Sprint 2
Sep 16, 2019
brettcannon
modified the milestones:
September sprint 2019.1,
2019 - September Sprint 2
Sep 16, 2019
karthiknadig
modified the milestones:
2019 - September Sprint 2,
2019 - October Sprint 1
Sep 25, 2019
Validated. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
A/A testing is used to validate metrics for A/B testing and/or the sampling logic itself.
Considering we have X experiments with Y% users in each group, we should take the remaining Z=100%-(X*Y)% (i.e. people who are not in any of the experiment groups) and sample them into control and treatment groups, with Y% users in each one.
The text was updated successfully, but these errors were encountered: