Fix user defined preprocess function missing prediction issue

[zane-neo]

Description

When users use self defined preprocess scripts to process inputs, there'll be missing predictions for all inputs in the list except the first one.

Issues Resolved

#2417

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[ylwu-amzn]

Is this understanding correct ?

1. textDocsLength means how many chunks
2. 1 means step or chunk size ?

If correct, why hard code the chunk size as 1 ? This issue #2417 is an example, it's not the only case. For example some user may process two documents in one chunk.

[ylwu-amzn]

https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/bedrock_connector_titan_multimodal_embedding_blueprint.md

[zane-neo]

textDocsLength is chunks, 1 means step. When user process two documents in one chunk, user has to specify the input_docs_processed_step_size, in this case, this won't enter this branch. Multi-modal is a case that user needs to handle two documents in one chunk, I've tested this case and it works well.

[ylwu-amzn]

Thanks for testing multi-modal case.
It's possible that step size > 1 and no input_docs_processed_step_size. For example, before async http client, user can confiture pre_process function to read three documents and construct "inputs": [doc1, doc2, doc3], when response comes back, will move to next 3 docs

[zane-neo]

This is a rare case, and we have workaround which is to add input_docs_processed_step_size configuration.

[ylwu-amzn]

Thanks, based on our discussion, this sounds good