Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sample size calculation #40

Merged
merged 3 commits into from
Sep 19, 2022
Merged

sample size calculation #40

merged 3 commits into from
Sep 19, 2022

Conversation

jancervenka
Copy link
Collaborator

No description provided.

@jancervenka jancervenka force-pushed the sample-size-calculation branch from 4e242a0 to 7aeda09 Compare September 14, 2022 00:23
Copy link
Collaborator

@marekbenes marekbenes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

two_vars = 2 * (std ** 2) if std_2 is None else (std ** 2 + std_2 ** 2)
delta = mean * minimum_effect

alpha = 1 - confidence_level
m = n_variants - 1
alpha = alpha / m # Bonferroni correction
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it is relevant for pre-test calculation that we use Holm-Bonferroni instead of Bonferroni.

Copy link
Collaborator Author

@jancervenka jancervenka Sep 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not really applicable here. Holm-Bonferroni sorts the variants by their $p$-values and assigns the biggest correction to the variant with the lowest $p$-value. Since in the pre-test calculation we don't know the $p$-values, we assign the same correction $\frac{\alpha}{m}$ to each variant (vanilla Bonferroni).

Copy link
Collaborator

@marekbenes marekbenes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, LGTM 👍

@jancervenka jancervenka merged commit 5cf6139 into master Sep 19, 2022
@jancervenka jancervenka deleted the sample-size-calculation branch September 19, 2022 14:42
@jancervenka jancervenka mentioned this pull request Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants