Explore model inference options in OpenSearch

[wrigleyDan]

OpenSearch supports a couple of models directly (e.g. linear regression models, see https://opensearch.org/docs/latest/ml-commons-plugin/algorithms/#linear-regression).

Together with the ML inference processor we want to find out if there is a way run the dynamic model-driven approach as part of the main query.

That would mean a low code integration for OpenSearch users.

The idea would be to

1. index the training data (index features in an index)
2. train a model based on the training data
3. create a pipeline with the ML inference request processor
   a. the request processor takes the query together with its features as its input
   b. it generates a prediction as its output
   c. the prediction is the neural search weight, we can derive the keyword search weight from that value and use these in the hybrid search part (basically a result processor)

What would still live outside of OpenSearch is feature generation. Maybe that's alright as not everyone might use an identical set of features.

[wrigleyDan]

Reached out to the OpenSearch community to see how to overcome the current blocker of only getting NaNs when predicting via OpenSearch models:
https://opensearch.slack.com/archives/C05BGJ1N264/p1731077205560749

Created a Github issue after reaching out: opensearch-project/ml-commons#3210

[wrigleyDan]

With the changed approach (predict NDCG instead of neuralness) we are thinking about a custom pipeline. To be discussed in ml-commons community meeting.

The ml_inference pipeline is probably one to use as a starting point.
https://opensearch.org/docs/latest/search-plugins/search-pipelines/ml-inference-search-request/

[wrigleyDan]

Feature request opened and discussed in Search Community Triage meeting on 12/05:
opensearch-project/OpenSearch#16775