Fix special character problem by making FindValues prefer exact matches #4671
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #4619
Description
Because the default tokenizer ignores special characters, choices that only differ in their special characters are seen as identical and our choice recognizers had no way of taking special characters into account when resolving a tie between two choices. Our choice recognition system was designed for natural language processing, but choice prompts usually allow users to click on buttons rather than typing their choices and so it makes sense to expect exact matches much of the time. This PR resolves the special character problem by checking for exact matches before any tokenization occurs.
Note that since special characters still aren't tokenized, they still aren't taken into account if an exact match isn't found. So if the choices are ":) face" and ":-( face" and the user types "I like :) face" the recognizer will still see the two choices as identical and won't be able to use the special characters to identify the correct choice. To resolve situations like that, I recommend that the developer creates their own tokenizer function as I mentioned in the issue.
Because the intent behind this fix is to accommodate the buttons created by choice prompts, I was considering a "high level" fix by putting the exact match check in
ChoicePrompt
instead ofFind
. However, I ended up going with the "low level" fix because the choice prompt code is largely duplicated in adaptive dialog choice inputs so I wanted to put this in a central place so it gets applied in all cases.Specific Changes
Find.FindValues
now checks for exact matches and bypasses the rest of its logic if an exact match is foundTesting
Included