Simplify query-time analysis #370
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…zer_peliasQueryPartialToken, analyzer_peliasQueryFullToken and analyzer_peliasPhrase
missinglink
force-pushed
the
peliasQueryAnalyzer
branch
from
1146ba7
to
985dd01
Compare
|
It might also be nice to set the search-analyzer for each field, this would remove the need to specify one at query-time. |
|
Edit: actually no changes are required, I already removed that token filter. |
orangejulius
added a commit
to pelias/acceptance-tests
that referenced
this pull request
Aug 5, 2019
Could be related to some of the following changes: pelias/schema#370 pelias/schema#359
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have two 'query-time' analyzers right now:
The difference between these analyzers is due to how the synonyms substitutions are handled.
Some time ago we made a change to how synonyms are applied in schema, so instead of 'expanding' or 'contracting' like we used to do, we now allow pure synonym substitution where either/or version of the token will match.
So this means that query-time synonym generation is no longer required as all versions are available in the index.
I started doing the work to remove the synonyms filter and realised quickly that
peliasQueryPartialTokenandpeliasQueryFullTokenwould now do exactly the same thing, albeit under a different name.I would like to remove both of those analyzers as they are just confusing and don't serve any functional purpose, plus in their current form they will have a negative performance impact.
Removing or renaming the analyzers would be a breaking change for users who are using the old analyzer names in their defaults configs.
So this PR adds a new analyzer (ironic I know!), which we can migrate the API code to use, and then later, after a courtesy window we can remove the old analyzers.
note: I wouldn't merge this until after the ampersand PR is merged, because that is the last synonyms file which does expansion/contraction