Leverage the Lucene's Matches API in a new highlighter type

[jimczi]

We should leverage the Matches API capabilities to build a new Highlighter type. This new highlighter would:

• Highlight the text part that matches any boolean query accurately.
• Not split phrase query into individual terms.
• Accept matches_fields natively.

[elasticmachine]

Pinging @elastic/es-search-aggs

[mgrishaber-merlin]

The current highlighters are far too slow and tend to bloat the index.
I have a request for a new highlighter that returns a simple word list as part of the json, rather than returning marked up html text for each hit. So if I get 1000 hits for a query, I would expect to get back a single word list that contains all of the words that matched rather than 1000 html snippets.
Something like the following,
"hits": {
"total": 1000,
"max_score": null,
"hitlist": [
{
"ski",
"skiier",
"skiing",
"skiied",
"skiis"
]
}
"hits": [
{
"_index": "myindex",
"_type": "mytype",
"_id": "1",
"_score": 2.7186205,
"sort": [
1438971741000,
2.7186205,
.
.
.
"_index": "myindex",
"_type": "mytype",
"_id": "1000",
"_score": 2.7186205,
"sort": [
1438971741000,
2.7186205
]
}
}

Thanks
Mike

[jimczi]

that's a different thing @mgrishaber-merlin and probably more related to #34214 which a proposal to extend named queries to return the terms that match.

[oersted]

How is progress on this? More accurate highlights and not splitting phrase matches are important properties for our use case.

[StijnArnauts]

Has there been any progress on this?

[Daantie]

I see a new label is added on May 5th. Can you give us an update of the progress on this?

[stefanobranco]

Any progress? This is a pretty big concern for us, so if there's any way for us to support this feel free to let us know.

Also, is it correct that this would also allow intervals queries to be highlighted correctly?

[OFeshchenko]

Is there any progress on this?

[mayya-sharipova]

@OFeshchenko and others. Reporting the progress, although we did not implement a new highlighter, from Elasticsearch 8.10 by default unified highlighter uses Lucene Matches API that allows to highlight queries accurately.

Closing this, as we have decided not to implement a new highlighter, but rather focus on enhancing unified highlighter.