Model Support

[jackgerrits]

Tracking issue for overall area of model support in AutoGen. See sub-issues for specific areas as well as following areas:

• We need to ensure that expectations are clear for which models are supported, and in what capacity.
• What model providers need support added?
• [Exploration] Automatic model selection
• [Exploration] Prompt fine tuning

[SongChiYoung]

Hi, I’d like to highlight a compatibility issue we encountered when trying to support the Mistral family.

Currently, some parts of the codebase use startswith() checks on model_family or model_name to determine model type. This worked fine in the past, but Mistral-style models use names like "mistral-large-latest" or "codestral-latest", which don’t match current assumptions.

This caused issues during integration, which we addressed in the following PR: #6158

To ensure robust support for models like Mistral, it would be helpful to audit and replace loose startswith() checks with a more reliable mapping or registry-based method (e.g., MODEL_POINTERS, etc.).

[nicsuzor]

Hi there, thanks so much for the recent updates. I just want to note an inconsistency between merge #6158 and the docs.

Specifically, at 27da37e#r154885312, the change for Mistral etc introduces a new problem for those relying on the default. Previously, if a user provides an unregistered string for model_family, that would be interpreted as ModelFamily.UNKNOWN. Now it errors out on not finding a transformer instead. So the fix for 'codestral-latest' breaks e.g. 'llama-3-3-xyz', which in many cases would otherwise work perfectly fine with the default.

The docs suggest that autogen should provide a default for unknown model_family values:

Other families definitely exist and can be represented by a string, however, AutoGen will treat them as unknown

[SongChiYoung]

@nicsuzor
Thanks for the feedback! I believe the current behavior still supports the default fallback logic.

In particular, ModelFamily.UNKNOWN is explicitly handled via:

for model in __unknown_models:
    register_transformer("openai", model, __BASE_TRANSFORMER_MAP)

And we also register:

register_transformer("openai", "default", __BASE_TRANSFORMER_MAP)

If you encountered a KeyError with "llama-3-3-xyz", could you share a minimal repro? It might be a mismatch with ModelFamily.ANY not including that string, or fallback lookup logic missing in get_transformer.

Happy to review this further if we can pinpoint the path where fallback fails!

And here is my testcase

client2 = OpenAIChatCompletionClient(
    model="llama3.1:latest",
    base_url="http://127.0.0.1:11434/v1",
    api_key="ollama",
    model_info={
        "vision": False,
        "function_calling": True,
        "json_output": True,
        "family": "unknown",
    },
)

messages = [
    UserMessage(content="hello", source="user"),
]

# print("OLLAMA SDK : ", asyncio.run(client1.create(messages=messages)))
print("OPENAI SDK : ", asyncio.run(client2.create(messages=messages)))
print(client2)
print("DONE IT")

Additional...

You're absolutely right that the fallback path in get_transformer() might look like it raises on unknown families:

if model_family == ModelFamily.UNKNOWN:
    model_family = _find_model_family(api, model)

But as you noted, _find_model_family() is itself designed to be fail-safe. Here’s the actual logic:

def _find_model_family(api: str, model: str) -> str:
    family = ModelFamily.UNKNOWN
    for _family in MESSAGE_TRANSFORMERS[api].keys():
        if model.startswith(_family):
            family = _family
    return family

So even if no match is found, it simply returns "unknown" and avoids crashing. That’s intentional – it’s meant to gracefully handle cases where a new model (e.g., "llama-3-3-xyz") isn’t explicitly mapped but might still inherit OpenAI-compatible behavior.

And yes, we use startswith for _find_model_family() because this is about unmapped models, and we want best-effort affinity. If there’s a better heuristic, I’m open to refining it. 😄

Appreciate your thoughtful review as always!