Multi agents with manager

[aymeric-roucher]

What does this PR do?

Add native support for multi-agents in a tree-like hierarchic structure, by allowing an agent to manage other agents and call them as if they were tools.

cc @LysandreJik and @plaggy

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[plaggy]

@aymeric-roucher did you consider wrapping managed agents as tools, the way you did with Gaia? Would passing a separate list of managed agents be beneficial in some way?

[aymeric-roucher]

@plaggy passing the list of agents as tools works as done on GAIA, but it leads to confusion between managed agents and tools, which leads to errors.
For instance if you use a search agent and name it search_agent, when building tool descriptions, search_agent being amongst tools means that your manager agent will understand it should be called the same way as tools. So instead of giving it detailed and complex requests as it would do for an agent (something like "Give me the 3rd player listed on the Denver Nuggets homepage from the 15th of October"), it will give it minimal requests as for a normal google search : "3rd player Denver Nuggets", and thus get suboptimal results. This bad behaviour persists even if you prompt it correctly in the description like" this is not a tool but an agent, give it detailed explanations"

This is why passing managed agents as a different argument than tools is mandatory IMO.

[plaggy]

looks like this part can be moved to self.toolbox() since it's a dedicated function for that

[aymeric-roucher]

I'd prefer to leave managed agents as a separate thing from Tools, since they're supposed to be handled differently here.

[plaggy]

Is this supposed to be a list of ManagedAgent objects? It'd help to have some hints on how/where ManagedAgent is designed to be used

[aymeric-roucher]

Yes it's supposed to be! Since ManagedAgent is not defined yet, I have a chicken-and-egg definition problem, that's why I did not put a type hint. But if you have an idea on how to do this hint i'll include it!

[plaggy]

@plaggy passing the list of agents as tools works as done on GAIA, but it leads to confusion between managed agents and tools, which leads to errors. For instance if you use a search agent and name it search_agent, when building tool descriptions, search_agent being amongst tools means that your manager agent will understand it should be called the same way as tools. So instead of giving it detailed and complex requests as it would do for an agent (something like "Give me the 3rd player listed on the Denver Nuggets homepage from the 15th of October"), it will give it minimal requests as for a normal google search : "3rd player Denver Nuggets", and thus get suboptimal results. This bad behaviour persists even if you prompt it correctly in the description like" this is not a tool but an agent, give it detailed explanations"

This is why passing managed agents as a different argument than tools is mandatory IMO.

@aymeric-roucher is the idea to provide agents-as-tools a verbose description of the task in a prompt so they'd perform better?
The implementation makes sense to me! Left a couple of comments

[LysandreJik]

@aymeric-roucher let me know when you'd like me to review this PR!

[aymeric-roucher]

@LysandreJik this is ready for review! 🤗
File agents.py is becoming quite big with the addition of planning and ManagedAgent : at some point we might have to export both these components to other files.

[LysandreJik]

Nice work! Yes, no problem for me to either split the module in several or to have a specific folder to host different modules/objects