ChatInterface callback cannot return ChatMessage

[MarcSkovMadsen]

I'm on the main branch of Panel trying to create a LangChain Pandas DataFrame agent. I'm seeing some issues with the ChatInterface.

• callback with ChatMessage return value is not working.
• callback with ChatMessage yield value raises exception.

[MarcSkovMadsen]

callback with ChatMessage return value is not working.

import panel as pn
import pandas as pd

agent_user="Pandas Agent"
agent_avatar="🐼"

pn.extension("perspective")

def _create_message(message):
    return pn.chat.ChatMessage("I'm your LangChain Pandas Agent!", user="Pandas Agent", avatar="🐼")

async def callback(contents, user, instance):
    if isinstance(contents, pd.DataFrame):
        instance.active=1
        print("I was here")
        return _create_message("You uploaded a dataframe. Ask me anything about it")
        

    return _create_message("That is a great question. I don't know.")

chat_interface = pn.chat.ChatInterface(
    widgets=[pn.widgets.FileInput(name="Upload"), pn.widgets.TextInput(name="Message")],
    renderers=pn.pane.Perspective,
    callback=callback,
    callback_exception="verbose",
)
    
chat_interface.send("I'm your LangChain Pandas Agent!", user="Pandas Agent", avatar="🐼", respond=False)
chat_interface.servable()

After upload of the file I would expect the ChatInterface to add the message "You uploaded a dataframe. Ask me anything about it". But it does not.

I do see "I was here" printed in the console though.

You can use the file penguins (1).csv for testing

If I change to a sync callback it has the same issue. If I return a string it works.

[MarcSkovMadsen]

callback with ChatMessage yield value raises exception.

If I replace return with yield in the callback I get an exception instead.

import panel as pn
import pandas as pd

agent_user="Pandas Agent"
agent_avatar="🐼"

pn.extension("perspective")

def _create_message(message):
    return pn.chat.ChatMessage("I'm your LangChain Pandas Agent!", user="Pandas Agent", avatar="🐼")

async def callback(contents, user, instance):
    if isinstance(contents, pd.DataFrame):
        instance.active=1
        print("I was here")
        yield _create_message("You uploaded a dataframe. Ask me anything about it")
        

    yield _create_message("That is a great question. I don't know.")

chat_interface = pn.chat.ChatInterface(
    widgets=[pn.widgets.FileInput(name="Upload"), pn.widgets.TextInput(name="Message")],
    renderers=pn.pane.Perspective,
    callback=callback,
    callback_exception="verbose",
)
    
chat_interface.send("I'm your LangChain Pandas Agent!", user="Pandas Agent", avatar="🐼", respond=False)
chat_interface.servable()

Traceback (most recent call last):
  File "/home/jovyan/.local/share/hatch/env/virtual/panel-chat-examples/5SvpzOQu/panel-chat-examples/lib/python3.10/site-packages/panel/chat/feed.py", line 430, in _prepare_response
    await task
  File "/home/jovyan/.local/share/hatch/env/virtual/panel-chat-examples/5SvpzOQu/panel-chat-examples/lib/python3.10/site-packages/panel/chat/feed.py", line 386, in _handle_callback
    response_message = await self._serialize_response(response)
  File "/home/jovyan/.local/share/hatch/env/virtual/panel-chat-examples/5SvpzOQu/panel-chat-examples/lib/python3.10/site-packages/panel/chat/feed.py", line 373, in _serialize_response
    response_message = self._upsert_message(token, response_message)
  File "/home/jovyan/.local/share/hatch/env/virtual/panel-chat-examples/5SvpzOQu/panel-chat-examples/lib/python3.10/site-packages/panel/chat/feed.py", line 339, in _upsert_message
    message.update(value, user=user, avatar=avatar)
  File "/home/jovyan/.local/share/hatch/env/virtual/panel-chat-examples/5SvpzOQu/panel-chat-examples/lib/python3.10/site-packages/panel/chat/message.py", line 553, in update
    raise ValueError(
ValueError: Cannot set user or avatar when explicitly sending a ChatMessage. Set them directly on the ChatMessage.

If I change to a sync callback I experience the same issue. If I return a string it works.

[MarcSkovMadsen]

Workaround

As a workaround you can replace

return message

with

instance.send(message)
return

in the callback.