- A major limitation of current LLMs is its difficulty to share/hold context across hundreds of chats (including "long context LLMs")
- Proposed solution: provide the LLM with a "function-call interface" platform, with the ability to store and search for specific/important information.
- Example: When a user reveals their name / date-of-birth, LLM detects that it is important, and internally issues a command to store that information externally (in a text database, for example). Next time, even if the LLM context is reset, the LLM can issue a command to look-up certain information when the context requires it, before sending a response to the user.
- This "function-call interface" can be extended to provide more capabilities to the LLM such as
web_search,calculate,execute_code, all of which would be handled by the platform that the LLM interacts with. - The LLM must be large enough to be able to produce syntactically and semantically valid outputs that can be interpreted by the platform.
- Anthropic has implemented this idea in their Python API and provide tutorials on how we can define custom functions ("tools") that we can then expose to a Claude model so that it can "use" these "tools" as necessary.
- Andrej Karpathy also mentions this idea in his great "Intro to LLMs" talk.
- Another current limitation of LLMs that's in research is for it to be able to rationally think, follow logical steps, reflect and correct itself (System 2 thinking, "Tree of Thoughts").
Sources:
- MemGPT: State Management for LLMs (Charles Packer)
- Claude Tool-Use Tutorials
- Intro to LLMs (Andrej Karpathy): Tool Use chapter -- also see chapters "Thinking, System 1/2"