-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Show all combinations prompt selection algorithm #43
base: master
Are you sure you want to change the base?
Conversation
Cool. Thanks @lukebaker. This sounds great, and I'd be interested in having it merged in, but I have a few questions first.
Thanks. |
I was hesitant to go down the route of generating all pairs due to the way that the number of pairs grows exponentially as the number of choices increase. I recalled the catchup algorithm having troubles with surveys with lots of choices. IIRC, our largest question has somewhere in the range of 18,000 choices. I think this algorithm scales linearly with the number of choices.
You bring up a good point. This algorithm doesn't fare as well when new ideas are added during a survey. The use-case is more targeted at the survey having all the ideas seeded initially and then folks are told to vote on them. This was the particular use-case of this client and similar to others that we've heard where creators want people to vote on all the options.
I haven't tested this on questions with lots of ideas, but did keep that performance in mind when creating the algorithm.
There is a test to verify that when the There's also a test that votes on all the combinations, confirms that it saw no duplicates and votes one more time and confirms that it did see a duplicate.
Sure. I'll see if the client is interested, otherwise I can. |
Sorry for the slow reply. This all sounds good. Once we have the blog post up, let's merge and deploy. |
This is exactly what my requirement was modulo duplicates. Thanks for your work on this.
Can you point me to the part of code where I can modify this to show the user that you have voted for all the combinations? Is it a simple change? |
@akshitkrnagpal, the algorithm here will not notify the user of this. This sort of change would happen in the pairwise client (e.g., the allourideas.org code). If I remember correctly, there is a total vote count for a visitor that gets incremented and displayed to the user. You could trigger the message when their total vote count indicates that they've seen all the combinations using the formula the following formula to determine how many votes they need to see all of them, where
|
@lukebaker |
@msalganik we had a client ask for this customization to All Our Ideas and also asked that we see if it makes sense to merge into the main repository.
This adds a new prompt selection algorithm called "all-combos". The goal of this selection algorithm is to show a particular visitor all possible combinations before showing any duplicates. This algorithm is concerned with combinations not permutations, so it considers a comparison between choice A and B equivalent to a comparison between choice B and A.
After a visitor has seen all combinations, the algorithm essentially starts over with a new "round". The visitor will see duplicates from the previous round, but the new round will complete with no duplicates within this round. There's randomness built-in to the algorithm so that each round is not identical and each visitor will see the prompts in different orders.
The mechanics of the algorithm is as follows:
This algorithm is only used when the API caller (All Our Ideas) explicitly requests this algorithm by name:
all-combos
. Related All Our Ideas pull request: allourideas/allourideas.org#60Does this look like the sort of thing that you'd like merged into the main repository?