Databricks SDK: 500 Server Error due to unrecognized "input" field in embedding request payload

[Rambo-AI97]

Description
The Databricks SDK raises a 500 Server Error when calling embed_documents on the DatabricksEmbeddings class. This error occurs because the SDK uses the field "input" in the JSON payload, which is not recognized by the Databricks API, resulting in a Bad request: json: unknown field "input" response.

Reproduction
from langchain_databricks import DatabricksEmbeddings

Initialize DBRX Embeddings

embedding_model = DatabricksEmbeddings(endpoint="dbrx_instruct")

Attempt to embed a test input

try:
response = embedding_model.embed_documents(["Test input"])
print("Response:", response)
except Exception as e:
print("Error:", e)

Expected behavior
The SDK should successfully pass the embedding request to the Databricks API with a correctly recognized field name (e.g., "data" or "text") and return an embedding response.

Is it a regression?
Unknown. This behavior may be specific to the current SDK version. Previous functionality is unconfirmed.

Debug Logs
Error: 500 Server Error: unable to parse response. This is likely a bug in the Databricks SDK for Python or the underlying API...
Request log:
POST /serving-endpoints/dbrx_instruct/invocations
{
"input": [
"Test input"
]
}
400 Bad Request: json: unknown field "input"

Other Information

• OS: [e.g. macOS]
• Version: SDK Version: 0.32.3

Additional context
The SDK appears to enforce "input" as the field name when calling embed_documents, leading to a compatibility issue with the Databricks API. This prevents the API from accepting valid requests, even for simple string inputs, due to an unrecognized field. A potential fix would involve allowing the field name to be configurable or defaulting to an accepted field like "data" or "text".