OpenAI Functions support and demo example

[votkon]

Hi,
I was wondering how we could plug the OpenAI Functions into Lanarky.

It takes the conversation chain as an input and Functions seem to be called by Agents
Do you think it's possible to add that and maybe some examples as well?
Thanks!

[ajndkr]

@votkon sure! do you maybe have a langchain example or a use case?

[ajndkr]

@votkon I think you can use OpenAI functions directly via initialise_agent() (https://python.langchain.com/docs/modules/agents/agent_types/openai_functions_agent). The function creates an AgentExecutor instance which is already supported. See example: https://github.com/ajndkr/lanarky/blob/main/examples/app/zero_shot_agent.py

[votkon]

Thanks for the prompt reply!

I somehow missed this example.

After giving it a try it looks like it doesn't communicate correctly with the frontend app.

In the zero_shot_agent.py example, when I change the

  return initialize_agent(
        tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
    ) 

to

  return initialize_agent(
        tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True
    )

it doesn't stream responses back to the chat, although it shows them in the Python output.

Also, it doesn't look like initialize_agent supports the memory parameter:
https://python.langchain.com/docs/modules/memory/how_to/agent_with_memory

I am doing a simple web chat with conversation memory and the ability to call different tools and/or functions.

Thanks!

[ajndkr]

it doesn't stream responses back to the chat, although it shows them in the Python output.

I see! It's likely due to this:

lanarky/lanarky/callbacks/agents.py

Line 24 in 6127973

answer_prefix_tokens: list[str] = ["Final", " Answer", ":"]

I need to update this to handle all agent types. I think currently only zero shot agents are supported.

[votkon]

Thanks! Have you had a chance to look into it?

[ajndkr]

Thanks! Have you had a chance to look into it?

nope! i have been busy these past few weeks. I will try to find more time to work on this repo.

[GuillaumeBeal]

Hi @votkon,

Using this custom class

    class CustomAsyncAgentsStreamingCallback(AsyncAgentsLanarkyCallback, AsyncLLMChainStreamingCallback):
        """AsyncStreamingResponseCallback handler for AgentExecutor."""
        answer_prefix_tokens: list[str] = [""]

and using it as a callback in your StreamingResponse may be a quickfix for you

StreamingResponse.from_chain( 
        agent_executor, 
        {"input": query}, 
        media_type="text/event-stream",
        callback=CustomAsyncAgentsStreamingCallback
    )

[ajndkr]

I will add native support for OpenAI functions as part of v0.9 roadmap.