Agent Implementation of MathChat

[yiranwu0]

Implement the MathChat paper with the current agent framework:

• Put the specified math problem-solving prompt in system message of AssistantAgent for the LLM. If the behavior is not desired, the assistant agent need to be modified to accept this prompt in user message.
• Implement a MathUserProxyAgent from UserProxyAgent, need to inherit the autoreply function.
• MathChat proposes a conversational framework to solve math problems, which is equivalent to interactions between the two agents. Building on the agents also allow human inputs.

Tasks

Beta Give feedback

1. Review PR Agent notebook example with human feedback; Support shell command and multiple code blocks; Improve the system message for assistant agent; Improve utility functions for config lists; reuse docker image  #1056
2. Evaluate the performance of AssistantAgent and UserProxyAgent on math problem solving (on several representative problems)
3. Agent implementation of MathChat. PR made and approvals from reviewers pending in An agent implementation of MathChat #1076
4. Update arxiv: Update link to implementation(keep the old one for the purpose of reproducing the exp results)
5. Update FLAML's autogen documentation: mention MathChat
6. Write a blogpost about MathChat

[sonichi]

What does "Put the specified math problem-solving prompt in system message of AssistantAgent for the LLM." mean? A complete replacement or a modification? There are certain designs in the existing prompt of AssistantAgent that makes it collaborate w/ the UserProxyAgent. Would you like to keep those working designs?

[yiranwu0]

I mean that when calling the AssistantAgent, I will not use the default system_message but input one. I will implement a corresponding MathUserProxyAgent, and I will modify the autoreply function. It will follow the working design, however the autoreply will be very different from the base proxyagent.

[qingyun-wu]

@kevin666aa, I revised the task list.