Skip to content

Comments

feature: support custom headers for LLM call when using custom embedding#62

Merged
planetf1 merged 3 commits intoAI4quantum:mainfrom
planetf1:customheader
Sep 9, 2025
Merged

feature: support custom headers for LLM call when using custom embedding#62
planetf1 merged 3 commits intoAI4quantum:mainfrom
planetf1:customheader

Conversation

@planetf1
Copy link
Collaborator

@planetf1 planetf1 commented Sep 5, 2025

Adds support when specifying custom headers when using CUSTOM_EMBEDDING_MODEL & associated parameters
milvus only (only milvus supports custom embeddings)

Example:

CUSTOM_EMBEDDING_HEADERS="MY_LLM_SECRET=0123456789abcdef"

or

CUSTOM_EMBEDDING_HEADERS='{"MY_LLM_SECRET": "0123456789abcdef"}'

Alternatively multiple values can be comma separated, or the variable can be specified in JSON

Tested on a platform which requires custom headers for authentication

Fixes #61

@planetf1
Copy link
Collaborator Author

planetf1 commented Sep 5, 2025

I had to add 4 type corrections in the milvus test that were added in a previous PR (unrelated to this change) - lint now runs cleanly

@planetf1
Copy link
Collaborator Author

planetf1 commented Sep 5, 2025

Parsing of environment in start.sh was also fixed to be more robust with quoted values

Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@maximilien maximilien changed the title Support custom headers for LLM call when using custom embedding feature: support custom headers for LLM call when using custom embedding Sep 5, 2025
@maximilien maximilien added the feature new feature label Sep 5, 2025
Copy link
Member

@ajbozarth ajbozarth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested local using RITS and this LGTM.

One big caveat: You need to remove the cli changes and open a PR on the new repo with those instead since it got moved out after you opened this

Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@planetf1
Copy link
Collaborator Author

planetf1 commented Sep 8, 2025

Removed cli changes. Will open issue on new cli repo.
The change didn't actually enable custom embeddings - it was just a command I noticed was missing from the cli, whilst it was present in the mcp server (and a useful command I was using to debug!)

@planetf1
Copy link
Collaborator Author

planetf1 commented Sep 8, 2025

Raised issue AI4quantum/maestro-cli#3 for cli changes

@planetf1 planetf1 merged commit eca4586 into AI4quantum:main Sep 9, 2025
6 checks passed
@planetf1 planetf1 deleted the customheader branch September 9, 2025 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ability to pass custom headers for custom embeddings

3 participants