Bedrock mistral

[aashsach]

Title:

• Support for Mistral models in bedorck

Description: (optional)

• Adding support for completion mistral bedrock

Motivation: (optional)

• Getting mistral response from Portkey via bedrock

Related Issues: (optional)

• #issue-number

[VisargD]

Hey @aashsach - Thanks for the PR. We have released some changes today related to code formatting as well as some changes related to provider config structure. You will have to update your branch with those changes.

[VisargD]

Please let me know if you need any help with this.

[VisargD]

@aashsach - For all the current bedrock models, we support chat as well as complete requests by making transformations accordingly. In case of mistral, we can transform messages into prompt and allow chat completions calls as well. You can check out the examples in src/providers/bedrock/chatComplete.ts. It has transformers written for many bedrock models. Will it be possible for you to add this change as well?

[aashsach]

@aashsach - For all the current bedrock models, we support chat as well as complete requests by making transformations accordingly. In case of mistral, we can transform messages into prompt and allow chat completions calls as well. You can check out the examples in src/providers/bedrock/chatComplete.ts. It has transformers written for many bedrock models. Will it be possible for you to add this change as well?

@VisargD mistral models are only text models, do not support chat modality as per
https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html

confirmed from their playground, chat playground doesn't support mistral

[VisargD]

Correct. They are text models. But gateway internally makes transformations to convert chat completions input to text completions input. This is done to just make gateway more flexible in terms of request signature. You can check the BedrockAI21ChatCompleteConfig for an example

[aashsach]

Correct. They are text models. But gateway internally makes transformations to convert chat completions input to text completions input. This is done to just make gateway more flexible in terms of request signature. You can check the BedrockAI21ChatCompleteConfig for an example

done

[VisargD]

Apart from the one open comment related to complete non-stream token count, everything else looks good. I will run final tests and merge the PR once the comment is resolved.

[VisargD]

LGTM