[Issue] Function call not working with Non-OpenAI models + LiteLLM proxy.

[ragesh2000]

I was following exactly as the notebook for function calling
https://github.com/microsoft/autogen/blob/main/notebook/agentchat_function_call_currency_calculator.ipynb
but instead of the output shown there Iam keep on getting an error

The only change i made is instead of open ai model i used open source model using litellm.
Can anybody tell me why it is happening ?

[rickyloynd-microsoft]

Does your local LLM support function calling?

@kevin666aa

[ragesh2000]

Yes it supports function calling

[qingyun-wu]

Yes it supports function calling

Which model specifically are you using? Do you have examples of it successfully incurring function calls without AutoGen?

[davorrunje]

I believe this is the same issue we are trying to solve in #1206

@ragesh2000 can you please check if it is still failing for you with the latest version from git, you can install it with

pip install git+https://github.com/microsoft/autogen.git@main

[ragesh2000]

Now I am getting an error as soon as I start running

@davorrunje

[ekzhu]

@ragesh2000 can you post your code? The currency function notebook you linked works for me on main branch. The error message looks like a misconfiguration of llm_config.

[ragesh2000]

Sure

import autogen
import pandas as pd
import os
# from typing import Literal
from typing_extensions import Annotated

config_list = [
    {
        'base_url': "http://0.0.0.0:8000",
        'api_key': "NULL"
    }
]

llm_config = {
    "config_list": config_list,
    "timeout": 120,
}
chatbot = autogen.AssistantAgent(
    name="chatbot",
    system_message="For currency exchange tasks, only use the functions you have been provided with. Reply TERMINATE when the task is done.",
    llm_config=llm_config,
)

# create a UserProxyAgent instance named "user_proxy"
user_proxy = autogen.UserProxyAgent(
    name="user_proxy",
    is_termination_msg=lambda x: x.get("content", "") and x.get("content", "").rstrip().endswith("TERMINATE"),
    human_input_mode="NEVER",
    max_consecutive_auto_reply=10,
)


CurrencySymbol = Literal["USD", "EUR"]


def exchange_rate(base_currency: CurrencySymbol, quote_currency: CurrencySymbol) -> float:
    if base_currency == quote_currency:
        return 1.0
    elif base_currency == "USD" and quote_currency == "EUR":
        return 1 / 1.1
    elif base_currency == "EUR" and quote_currency == "USD":
        return 1.1
    else:
        raise ValueError(f"Unknown currencies {base_currency}, {quote_currency}")


@user_proxy.register_for_execution()
@chatbot.register_for_llm(description="Currency exchange calculator.")
def currency_calculator(
    base_amount: Annotated[float, "Amount of currency in base_currency"],
    base_currency: Annotated[CurrencySymbol, "Base currency"] = "USD",
    quote_currency: Annotated[CurrencySymbol, "Quote currency"] = "EUR",
) -> str:
    quote_amount = exchange_rate(base_currency, quote_currency) * base_amount
    return f"{quote_amount} {quote_currency}"

assert user_proxy.function_map["currency_calculator"]._origin == currency_calculator

# start the conversation
user_proxy.initiate_chat(
    chatbot,
    message="How much is 123.45 USD in EUR?",
)

Also Iam using llama2 model using litellm
@ekzhu

[ekzhu]

I see. It looks like you may have to specify a model value in the config list entry.

[ragesh2000]

Can i set it as the model I am using?
'model': 'llama2' ?

[ekzhu]

You can try it. We are currently relying on the openai client library. But we are currently working toward customizable client #831

[ragesh2000]

It wasn't working.

[ekzhu]

It wasn't working.

Did you get a new error message?