Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support to discard compound tokens in the Kuromoji tokenizer #61545

Closed
jimczi opened this issue Aug 25, 2020 · 2 comments
Closed

Add support to discard compound tokens in the Kuromoji tokenizer #61545

jimczi opened this issue Aug 25, 2020 · 2 comments
Labels
>enhancement :Search Relevance/Analysis How text is split into tokens Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch

Comments

@jimczi
Copy link
Contributor

jimczi commented Aug 25, 2020 •
edited
Loading

In LUCENE-9123 a new option called discardCompoundToken was added to drop compound tokens at indexing time. We should expose this option for the SEARCH and EXTENDED mode of the Kuromoji tokenizer.
@jimczi jimczi added >enhancement help wanted adoptme needs:triage Requires assignment of a team area label :Search Relevance/Analysis How text is split into tokens and removed needs:triage Requires assignment of a team area label labels Aug 25, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Analysis)

@elasticmachine elasticmachine added the Team:Search Meta label for search team label Aug 25, 2020
@jimczi jimczi removed the help wanted adoptme label Aug 26, 2020
@jimczi
Copy link
Contributor Author

jimczi commented Aug 26, 2020

That was already exposed in #57421, hence closing

@jimczi jimczi closed this as completed Aug 26, 2020
@javanna javanna added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch and removed Team:Search Meta label for search team labels Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>enhancement :Search Relevance/Analysis How text is split into tokens Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@javanna @elasticmachine @jimczi