awesome_llm_agents.md

Awesome llm agents

• Awesome llm agents
  • Survey
  • LLM OS
  • Agents
    • Other
  • AutoGPT
    • Other
  • Augmented LLM
    • Other
  • Web browsing
    • Other
  • Retrieval agumented generation
    • Embedding
    • Other
  • Code Interpreter
  • GPTs
    • Plugins
    • Other
  • Evaluation
  • Other
  • Vector Database
    • Other
  • Extra reference

Survey

• From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future, arXiv, 2408.02479, arxiv, pdf, cication: -1

  Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, Huaming Chen

• Retrieval-Augmented Generation for Natural Language Processing: A Survey, arXiv, 2407.13193, arxiv, pdf, cication: -1

  Shangyu Wu, Ying Xiong, Yufei Cui, Haolun Wu, Can Chen, Ye Yuan, Lianming Huang, Xue Liu, Tei-Wei Kuo, Nan Guan

• Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach, arXiv, 2407.16833, arxiv, pdf, cication: -1

  Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky

• A Survey on RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models, arXiv, 2405.06211, arxiv, pdf, cication: -1

  Yujuan Ding, Wenqi Fan, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, Qing Li

• RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing, arXiv, 2404.19543, arxiv, pdf, cication: -1

  Yucheng Hu, Yuxing Lu · (ralm_survey - 2471023025)

• The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey, arXiv, 2404.11584, arxiv, pdf, cication: -1

  Tula Masterman, Sandi Besen, Mason Sawtell, Alex Chao

• Retrieval-Augmented Generation for AI-Generated Content: A Survey, arXiv, 2402.19473, arxiv, pdf, cication: -1

  Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Bin Cui · (RAG-Survey - hymie122)

  · (mp.weixin.qq)

• Large Multimodal Agents: A Survey, arXiv, 2402.15116, arxiv, pdf, cication: -1

  Junlin Xie, Zhihong Chen, Ruifei Zhang, Xiang Wan, Guanbin Li

  · (awesome-large-multimodal-agents - jun0wanan)

• Large Language Model based Multi-Agents: A Survey of Progress and Challenges, arXiv, 2402.01680, arxiv, pdf, cication: -1

  Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, Xiangliang Zhang

• Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security, arXiv, 2401.05459, arxiv, pdf, cication: -1

  Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun · (Personal_LLM_Agents_Survey - MobileLLM)

• Retrieval-Augmented Generation for Large Language Models: A Survey, arXiv, 2312.10997, arxiv, pdf, cication: -1

  Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Haofen Wang · (rag-survey - tongji-kgllm)

• Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives, arXiv, 2312.11970, arxiv, pdf, cication: -1

  Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, Yong Li · (mp.weixin.qq)

• LLM Powered Autonomous Agents | Lil'Log

  · (mp.weixin.qq)

LLM OS

• phidata - phidatahq

• AIOS: LLM Agent Operating System, arXiv, 2403.16971, arxiv, pdf, cication: -1

  Kai Mei, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, Yongfeng Zhang

  · (AIOS - agiresearch)

• 01 - OpenInterpreter

  The open-source language model computer

  · (qbitai)

• UFO: A UI-Focused Agent for Windows OS Interaction, arXiv, 2402.07939, arxiv, pdf, cication: -1

  Chaoyun Zhang, Liqun Li, Shilin He, Xu Zhang, Bo Qiao, Si Qin, Minghua Ma, Yu Kang, Qingwei Lin, Saravan Rajmohan · (UFO - microsoft)

• OS-Copilot: Towards Generalist Computer Agents with Self-Improvement, arXiv, 2402.07456, arxiv, pdf, cication: -1

  Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, Lingpeng Kong

  · (FRIDAY - OS-Copilot)

• At the Intersection of LLMs and Kernels - Research Roundup

• llama2.c - trholding

  Llama 2 Everywhere (L2E) · (jiqizhixin)

• MemGPT - cpacker

  Teaching LLMs memory management for unbounded context 📚🦙

  · (jiqizhixin)

Agents

• Automated Design of Agentic Systems, arXiv, 2408.08435, arxiv, pdf, cication: -1

  Shengran Hu, Cong Lu, Jeff Clune · (ADAS - ShengranHu)

• AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents, arXiv, 2407.17490, arxiv, pdf, cication: -1

  Yuxiang Chai, Siyuan Huang, Yazhe Niu, Han Xiao, Liang Liu, Dingyu Zhang, Peng Gao, Shuai Ren, Hongsheng Li · (yuxiangchai.github)

• Very Large-Scale Multi-Agent Simulation in AgentScope, arXiv, 2407.17789, arxiv, pdf, cication: -1

  Xuchen Pan, Dawei Gao, Yuexiang Xie, Zhewei Wei, Yaliang Li, Bolin Ding, Ji-Rong Wen, Jingren Zhou · (agentscope - modelscope)

• LAMBDA: A Large Model Based Data Agent, arXiv, 2407.17535, arxiv, pdf, cication: -1

  Maojun Sun, Ruijian Han, Binyan Jiang, Houduo Qi, Defeng Sun, Yancheng Yuan, Jian Huang · (polyu.edu)

• Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks, arXiv, 2408.03615, arxiv, pdf, cication: -1

  Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, Liqiang Nie · (cybertronagent.github) · (Optimus-1 - JiuTian-VL)

• Fetching Title#nl37

  · (odyssey - zju-vipa)

• ioa - openbmb

  An open-source framework for collaborative AI agents, enabling diverse, distributed agents to team up and tackle complex tasks through internet-like connectivity.

• Recursive Introspection: Teaching Foundation Model Agents How to Self-Improve | OpenReview

• AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents, arXiv, 2407.04363, arxiv, pdf, cication: -1

  Petr Anokhin, Nikita Semenov, Artyom Sorokin, Dmitry Evseev, Mikhail Burtsev, Evgeny Burnaev

  · (AriGraph - AIRI-Institute)

• Agentless: Demystifying LLM-based Software Engineering Agents, arXiv, 2407.01489, arxiv, pdf, cication: -1

  Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, Lingming Zhang

  · (Agentless - OpenAutoCoder)

• AI Agents That Matter, arXiv, 2407.01502, arxiv, pdf, cication: -1

  Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, Nitya Nadgir, Arvind Narayanan

• MIRAI: Evaluating LLM Agents for Event Forecasting, arXiv, 2407.01231, arxiv, pdf, cication: -1

  Chenchen Ye, Ziniu Hu, Yihe Deng, Zijie Huang, Mingyu Derek Ma, Yanqiao Zhu, Wei Wang

  · (MIRAI - yecchen)

• GUICourse: From General Vision Language Models to Versatile GUI Agents, arXiv, 2406.11317, arxiv, pdf, cication: -1

  Wentong Chen, Junbo Cui, Jinyi Hu, Yujia Qin, Junjie Fang, Yue Zhao, Chongyi Wang, Jun Liu, Guirong Chen, Yupeng Huo · (GUICourse - yiye3)

• Mixture-of-Agents Enhances Large Language Model Capabilities, arXiv, 2406.04692, arxiv, pdf, cication: -1

  Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, James Zou

• Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration, arXiv, 2406.01014, arxiv, pdf, cication: -1

  Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, Jitao Sang

  · (MobileAgent - X-PLUG)

• AgentGym: Evolving Large Language Model-based Agents across Diverse Environments, arXiv, 2406.04151, arxiv, pdf, cication: -1

  Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Dingwen Yang, Chenyang Liao, Xin Guo, Wei He

  · (AgentGym - WooooDyy) · (AgentGym - WooooDyy)

• Luban: Building Open-Ended Creative Agents via Autonomous Embodied Verification, arXiv, 2405.15414, arxiv, pdf, cication: -1

  Yuxuan Guo, Shaohui Peng, Jiaming Guo, Di Huang, Xishan Zhang, Rui Zhang, Yifan Hao, Ling Li, Zikang Tian, Mingju Gao

• agentscope - modelscope

  Start building LLM-empowered multi-agent applications in an easier way.

• pywinassistant - a-real-ai

  The first open source Large Action Model generalist Artificial Narrow Intelligence that controls completely human user interfaces by only using natural language. PyWinAssistant utilizes Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models.

• agentkit - holmeswww

  An intuitive LLM prompting framework for multifunctional agents, by explicitly constructing a complex "thought process" from simple natural language prompts.

• FlowMind: Automatic Workflow Generation with LLMs, arXiv, 2404.13050, arxiv, pdf, cication: -1

  Zhen Zeng, William Watson, Nicole Cho, Saba Rahimi, Shayleen Reynolds, Tucker Balch, Manuela Veloso

• maestro - Doriandarko

  A framework for Claude Opus to intelligently orchestrate subagents.

• Scaling Instructable Agents Across Many Simulated Worlds, arXiv, 2404.10179, arxiv, pdf, cication: -1

  SIMA Team, Maria Abi Raad, Arun Ahuja, Catarina Barros, Frederic Besse, Andrew Bolt, Adrian Bolton, Bethanie Brownfield, Gavin Buttimore, Max Cant

• Autonomous Evaluation and Refinement of Digital Agents, arXiv, 2404.06474, arxiv, pdf, cication: -1

  Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, Alane Suhr · (Agent-Eval-Refine - Berkeley-NLP)

• More Agents Is All You Need, arXiv, 2402.05120, arxiv, pdf, cication: -1

  Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, Deheng Ye

• AgentStudio: A Toolkit for Building General Virtual Agents, arXiv, 2403.17918, arxiv, pdf, cication: -1

  Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, Shuicheng Yan · (skyworkai.github)

• AllHands: Ask Me Anything on Large-scale Verbatim Feedback via Large Language Models, arXiv, 2403.15157, arxiv, pdf, cication: -1

  Chaoyun Zhang, Zicheng Ma, Yuhao Wu, Shilin He, Si Qin, Minghua Ma, Xiaoting Qin, Yu Kang, Yuyi Liang, Xiaoyu Gou

• Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models, arXiv, 2403.12881, arxiv, pdf, cication: -1

  Zehui Chen, Kuikun Liu, Qiuchen Wang, Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, Feng Zhao · (Agent-FLAN - InternLM)

• SOTOPIA-$π$: Interactive Learning of Socially Intelligent Language Agents, arXiv, 2403.08715, arxiv, pdf, cication: -1

  Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Graham Neubig, Yonatan Bisk, Hao Zhu

• AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System, arXiv, 2402.15538, arxiv, pdf, cication: -1

  Zhiwei Liu, Weiran Yao, Jianguo Zhang, Liangwei Yang, Zuxin Liu, Juntao Tan, Prafulla K. Choubey, Tian Lan, Jason Wu, Huan Wang

  · (agentlite - salesforceairesearch)

• KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents, arXiv, 2403.03101, arxiv, pdf, cication: -1

  Yuqi Zhu, Shuofei Qiao, Yixin Ou, Shumin Deng, Ningyu Zhang, Shiwei Lyu, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen

  · (KnowAgent - zjunlp) · (zjunlp.github)

• Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents, arXiv, 2403.02502, arxiv, pdf, cication: -1

  Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, Bill Yuchen Lin · (ETO - Yifan-Song793)

• Qwen-Agent - QwenLM

  Agent framework and applications built upon Qwen1.5, featuring Function Calling, Code Interpreter, RAG, and Chrome extension.

• Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models, arXiv, 2402.14207, arxiv, pdf, cication: -1

  Yijia Shao, Yucheng Jiang, Theodore A. Kanell, Peter Xu, Omar Khattab, Monica S. Lam · (storm - stanford-oval)

• AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning, arXiv, 2402.15506, arxiv, pdf, cication: -1

  Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Juntao Tan, Thai Hoang, Liangwei Yang, Yihao Feng, Zuxin Liu

• AgentScope: A Flexible yet Robust Multi-Agent Platform, arXiv, 2402.14034, arxiv, pdf, cication: -1

  Dawei Gao, Zitao Li, Weirui Kuang, Xuchen Pan, Daoyuan Chen, Zhijian Ma, Bingchen Qian, Liuyi Yao, Lin Zhu, Chen Cheng · (agentscope - modelscope)

• LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration, arXiv, 2402.11550, arxiv, pdf, cication: -1

  Jun Zhao, Can Zu, Hao Xu, Yi Lu, Wei He, Yiwen Ding, Tao Gui, Qi Zhang, Xuanjing Huang

• Small LLMs Are Weak Tool Learners: A Multi-LLM Agent, arXiv, 2401.07324, arxiv, pdf, cication: 1

  Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, Fei Huang · (Multi-LLM-agent - X-PLUG) · (qbitai)

• An Interactive Agent Foundation Model, arXiv, 2402.05929, arxiv, pdf, cication: -1

  Zane Durante, Bidipta Sarkar, Ran Gong, Rohan Taori, Yusuke Noda, Paul Tang, Ehsan Adeli, Shrinidhi Kowshika Lakshmikanth, Kevin Schulman, Arnold Milstein

• More Agents Is All You Need, arXiv, 2402.05120, arxiv, pdf, cication: -1

  Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, Deheng Ye · (anonymous.4open)

• PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models, arXiv, 2402.01118, arxiv, pdf, cication: -1

  Sihao Hu, Tiansheng Huang, Ling Liu · (PokeLLMon - git-disl) · (poke-llm-on.github)

• V-IRL: Grounding Virtual Intelligence in Real Life, arXiv, 2402.03310, arxiv, pdf, cication: -1

  Jihan Yang, Runyu Ding, Ellis Brown, Xiaojuan Qi, Saining Xie · (virl-platform.github) · (VIRL - VIRL-Platform)

• TravelPlanner: A Benchmark for Real-World Planning with Language Agents, arXiv, 2402.01622, arxiv, pdf, cication: -1

  Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, Yu Su · (osu-nlp-group.github) · (TravelPlanner - OSU-NLP-Group) · (mp.weixin.qq)

• Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolution, arXiv, 2401.13996, arxiv, pdf, cication: -1

  Cheng Qian, Shihao Liang, Yujia Qin, Yining Ye, Xin Cong, Yankai Lin, Yesai Wu, Zhiyuan Liu, Maosong Sun · (jiqizhixin)

• Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception, arXiv, 2401.16158, arxiv, pdf, cication: -1

  Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, Jitao Sang · (MobileAgent - X-PLUG)

  · (huggingface)

• SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents, arXiv, 2401.10935, arxiv, pdf, cication: -1

  Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, Zhiyong Wu · (SeeClick - njucckevin)

• Investigate-Consolidate-Exploit: A General Strategy for Inter-Task Agent Self-Evolution, arXiv, 2401.13996, arxiv, pdf, cication: -1

  Cheng Qian, Shihao Liang, Yujia Qin, Yining Ye, Xin Cong, Yankai Lin, Yesai Wu, Zhiyuan Liu, Maosong Sun

• ChatQA: Building GPT-4 Level Conversational QA Models, arXiv, 2401.10225, arxiv, pdf, cication: -1

  Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Mohammad Shoeybi, Bryan Catanzaro

• Tool-LMM: A Large Multi-Modal Model for Tool Agent Learning, arXiv, 2401.10727, arxiv, pdf, cication: -1

  Chenyu Wang, Weixin Luo, Qianyu Chen, Haonan Mai, Jindi Guo, Sixun Dong, Xiaohua, Xuan, Zhengxin Li, Lin Ma · (Tool-LMM?tab=readme-ov-file - Tool-LMM)

• Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk, arXiv, 2401.05033, arxiv, pdf, cication: -1

  Dennis Ulmer, Elman Mansimov, Kaixiang Lin, Justin Sun, Xibin Gao, Yi Zhang

• GitAgent: Facilitating Autonomous Agent with GitHub by Tool Extension, arXiv, 2312.17294, arxiv, pdf, cication: -1

  Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Yujia Qin, Yining Ye, Yaxi Lu, Zhong Zhang, Yukun Yan, Yankai Lin

• Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning, arXiv, 2312.14878, arxiv, pdf, cication: -1

  Filippos Christianos, Georgios Papoudakis, Matthieu Zimmer, Thomas Coste, Zhihao Wu, Jingxuan Chen, Khyati Khandelwal, James Doran, Xidong Feng, Jiacheng Liu

• AppAgent: Multimodal Agents as Smartphone Users, arXiv, 2312.13771, arxiv, pdf, cication: -1

  Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, Gang Yu

  · (AppAgent - mnotgod96)

• KwaiAgents: Generalized Information-seeking Agent System with Large Language Models, arXiv, 2312.04889, arxiv, pdf, cication: -1

  Haojie Pan, Zepeng Zhai, Hao Yuan, Yaojia Lv, Ruiji Fu, Ming Liu, Zhongyuan Wang, Bing Qin · (kwaiagents - kwaikeg)

• CogAgent: A Visual Language Model for GUI Agents, arXiv, 2312.08914, arxiv, pdf, cication: -1

  Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding

  · (CogVLM - THUDM)

• Creative Agents: Empowering Agents with Imagination for Creative Tasks, arXiv, 2312.02519, arxiv, pdf, cication: -1

  Chi Zhang, Penglin Cai, Yuhui Fu, Haoqi Yuan, Zongqing Lu

  · (Creative-Agents - PKU-RL) · (mp.weixin.qq)

• An LLM Compiler for Parallel Function Calling, arXiv, 2312.04511, arxiv, pdf, cication: -1

  Sehoon Kim, Suhong Moon, Ryan Tabrizi, Nicholas Lee, Michael W. Mahoney, Kurt Keutzer, Amir Gholami · (llmcompiler - squeezeailab)

• Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses, arXiv, 2312.00763, arxiv, pdf, cication: -1

  Xiao Ma, Swaroop Mishra, Ariel Liu, Sophie Su, Jilin Chen, Chinmay Kulkarni, Heng-Tze Cheng, Quoc Le, Ed Chi

• taskweaver - microsoft

  A code-first agent framework for seamlessly planning and executing data analytics tasks.

  · (jiqizhixin)

• Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents, arXiv, 2311.11797, arxiv, pdf, cication: -1

  Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, Zhiwei He, Yiming Wang, Mark Gerstein, Rui Wang, Gongshen Liu · (CoT-Igniting-Agent - Zoeyyao27)

• ToolTalk: Evaluating Tool-Usage in a Conversational Setting, arXiv, 2311.10775, arxiv, pdf, cication: -1

  Nicholas Farn, Richard Shin · (ToolTalk - microsoft)

• TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems, arXiv, 2311.11315, arxiv, pdf, cication: -1

  Yilun Kong, Jingqing Ruan, Yihong Chen, Bin Zhang, Tianpeng Bao, Shiwei Shi, Guoqing Du, Xiaoru Hu, Hangyu Mao, Ziyue Li

• multi-agent-postgres-data-analytics - disler

  The way we interact with our data is changing.

• ProAgent - OpenBMB

  · (ProAgent - OpenBMB)

• JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models, arXiv, 2311.05997, arxiv, pdf, cication: -1

  Zihao Wang, Shaofei Cai, Anji Liu, Yonggang Jin, Jinbing Hou, Bowei Zhang, Haowei Lin, Zhaofeng He, Zilong Zheng, Yaodong Yang · (craftjarvis-jarvis1.github)

• Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs, arXiv, 2311.05657, arxiv, pdf, cication: -1

  Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin · (lumos - allenai) · (allenai.github)

• OpenAI_Agent_Swarm - daveshap

  HAAS = Hierarchical Autonomous Agent Swarm - "Resistance is futile!"

• LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents, arXiv, 2311.05437, arxiv, pdf, cication: -1

  Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu

• Octopus: Embodied Vision-Language Programmer from Environmental Feedback, arXiv, 2310.08588, arxiv, pdf, cication: -1

  Jingkang Yang, Yuhao Dong, Shuai Liu, Bo Li, Ziyue Wang, Chencheng Jiang, Haoran Tan, Jiamu Kang, Yuanhan Zhang, Kaiyang Zhou · (Octopus - dongyh20) · (mp.weixin.qq)

• War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars, arXiv, 2311.17227, arxiv, pdf, cication: -1

  Wenyue Hua, Lizhou Fan, Lingyao Li, Kai Mei, Jianchao Ji, Yingqiang Ge, Libby Hemphill, Yongfeng Zhang · (mp.weixin.qq)

• Neural MMO 2.0: A Massively Multi-task Addition to Massively Multi-agent Learning, arXiv, 2311.03736, arxiv, pdf, cication: -1

  Joseph Suárez, Phillip Isola, Kyoung Whan Choe, David Bloomin, Hao Xiang Li, Nikhil Pinnaparaju, Nishaanth Kanna, Daniel Scott, Ryan Sullivan, Rose S. Shuman

• From Copilot to CoOrchestration

• OpenAgents: An Open Platform for Language Agents in the Wild, arXiv, 2310.10634, arxiv, pdf, cication: -1

  Tianbao Xie, Fan Zhou, Zhoujun Cheng, Peng Shi, Luoxuan Weng, Yitao Liu, Toh Jing Hua, Junning Zhao, Qian Liu, Che Liu

• agenttuning - thudm

  AgentTuning: Enabling Generalized Agent Abilities for LLMs

• Humanoid Agents: Platform for Simulating Human-like Generative Agents, arXiv, 2310.05418, arxiv, pdf, cication: 1

  Zhilin Wang, Yu Ying Chiu, Yu Cheung Chiu · (humanoidagents - humanoidagents)

• XAgent - OpenBMB

  An Autonomous LLM Agent for Complex Task Solving · (jiqizhixin)

• Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency, arXiv, 2309.17382, arxiv, pdf, cication: -1

  Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu, Zhaoran Wang

• Humanoid Agents: Platform for Simulating Human-like Generative Agents, arXiv, 2310.05418, arxiv, pdf, cication: 1

  Zhilin Wang, Yu Ying Chiu, Yu Cheung Chiu · (mp.weixin.qq)

• A Zero-Shot Language Agent for Computer Control with Structured Reflection, arXiv, 2310.08740, arxiv, pdf, cication: -1

  Tao Li, Gang Li, Zhiwei Deng, Bryan Wang, Yang Li

• Lemur: Harmonizing Natural Language and Code for Language Agents, arXiv, 2310.06830, arxiv, pdf, cication: 1

  Yiheng Xu, Hongjin Su, Chen Xing, Boyu Mi, Qian Liu, Weijia Shi, Binyuan Hui, Fan Zhou, Yitao Liu, Tianbao Xie

• EcoAssistant: Using LLM Assistant More Affordably and Accurately, arXiv, 2310.03046, arxiv, pdf, cication: -1

  Jieyu Zhang, Ranjay Krishna, Ahmed H. Awadallah, Chi Wang

• khoj - khoj-ai

  An AI personal assistant for your digital brain

• AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn, arXiv, 2306.08640, arxiv, pdf, cication: -1

  Difei Gao, Lei Ji, Luowei Zhou, Kevin Qinghong Lin, Joya Chen, Zihan Fan, Mike Zheng Shou

• Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4, arXiv, 2309.17277, arxiv, pdf, cication: -1

  Jiaxian Guo, Bo Yang, Paul Yoo, Bill Yuchen Lin, Yusuke Iwasawa, Yutaka Matsuo

• autogen - microsoft

  Enable Next-Gen Large Language Model Applications. Join our Discord: https://discord.gg/pAbnFJrkgZ

• How FaR Are Large Language Models From Agents with Theory-of-Mind?, arXiv, 2310.03051, arxiv, pdf, cication: 2

  Pei Zhou, Aman Madaan, Srividya Pranavi Potharaju, Aditya Gupta, Kevin R. McKee, Ari Holtzman, Jay Pujara, Xiang Ren, Swaroop Mishra, Aida Nematzadeh · (qbitai)

• AutoAgents - LinkSoul-AI

  Generate different roles for GPTs to form a collaborative entity for complex tasks.

• LASER: LLM Agent with State-Space Exploration for Web Navigation, arXiv, 2309.08172, arxiv, pdf, cication: -1

  Kaixin Ma, Hongming Zhang, Hongwei Wang, Xiaoman Pan, Dong Yu

• Agents: An Open-source Framework for Autonomous Language Agents, arXiv, 2309.07870, arxiv, pdf, cication: 4

  Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang · (agents - aiwaves-cn)

• MindAgent: Emergent Gaming Interaction - Microsoft Research

  · (qbitai)

• The Rise and Potential of Large Language Model Based Agents: A Survey, arXiv, 2309.07864, arxiv, pdf, cication: 23

  Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou · (jiqizhixin) · (LLM-Agent-Paper-List - WooooDyy)

• Cognitive Architectures for Language Agents, arXiv, 2309.02427, arxiv, pdf, cication: 11

  Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, Thomas L. Griffiths · (awesome-language-agents - ysymyth)

• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors, arXiv, 2308.10848, arxiv, pdf, cication: 13

  Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian · (agentverse - openbmb)

• AI-town - a16z-infra

  A MIT-licensed, deployable starter kit for building and customizing your own version of AI town - a virtual town where AI characters live, chat and socialize.

• TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage, arXiv, 2308.03427, arxiv, pdf, cication: 11

  Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng

• SHOW-1 and Showrunner Agents in Multi-Agent Simulations

  · (fablestudio.github) · (mp.weixin.qq)

• Building Cooperative Embodied Agents Modularly with Large Language Models, arXiv, 2307.02485, arxiv, pdf, cication: -1

  Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B. Tenenbaum, Tianmin Shu, Chuang Gan

• autotab-starter - Planetary-Computers

  Build browser agents for real world tasks

• openagents - xlang-ai

  OpenAgents: An Open Platform for Language Agents in the Wild

• octopus - dongyh20

  🐙Octopus, an embodied vision-language model trained with RLEF, emerging superior in embodied visual planning and programming.

• gollie - hitz-zentroa

  Guideline following Large Language Model for Information Extraction

• NexusRaven-13B: Surpassing the state-of-the-art in open-source function calling LLMs.

  · (nexusflow)

• ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models, arXiv, 2309.00986, arxiv, pdf, cication: 2

  Chenliang Li, Hehong Chen, Ming Yan, Weizhou Shen, Haiyang Xu, Zhikai Wu, Zhicheng Zhang, Wenmeng Zhou, Yingda Chen, Chen Cheng · (modelscope-agent - modelscope)

• trl-text-environment - trl-lib 🤗

• awesome-ai-devtools - jamesmurdza

  Curated list of AI-powered developer tools.

• TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage, arXiv, 2308.03427, arxiv, pdf, cication: 11

  Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng

• functionary - musabgultekin

  Chat language model that can interpret and execute functions/plugins

• Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models, arXiv, 2308.00675, arxiv, pdf, cication: 4

  Cheng-Yu Hsieh, Si-An Chen, Chun-Liang Li, Yasuhisa Fujii, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, Tomas Pfister

• gorilla - ShishirPatil

  Gorilla: An API store for LLMs

• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs, arXiv, 2307.16789, arxiv, pdf, cication: 33

  Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian · (ToolBench - OpenBMB)

• Android in the Wild: A Large-Scale Dataset for Android Device Control, arXiv, 2307.10088, arxiv, pdf, cication: 4

  Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, Timothy Lillicrap · (google-research - google-research)

• amadeusgpt - adaptivemotorcontrollab

  We turn natural language descriptions of behaviors into machine-executable code

• Towards Language Models That Can See: Computer Vision Through the LENS of Natural Language, arXiv, 2306.16410, arxiv, pdf, cication: -1

  William Berrios, Gautam Mittal, Tristan Thrush, Douwe Kiela, Amanpreet Singh · (lens - contextualai)

• ViperGPT: Visual Inference via Python Execution for Reasoning, arXiv, 2303.08128, arxiv, pdf, cication: 76

  Dídac Surís, Sachit Menon, Carl Vondrick

• HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face, arXiv, 2303.17580, arxiv, pdf, cication: 233

  Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, Yueting Zhuang

• LOVM: Language-Only Vision Model Selection, arXiv, 2306.08893, arxiv, pdf, cication: -1

  Orr Zohar, Shih-Cheng Huang, Kuan-Chieh Wang, Serena Yeung

• CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models, arXiv, 2305.14318, arxiv, pdf, cication: 7

  Cheng Qian, Chi Han, Yi R. Fung, Yujia Qin, Zhiyuan Liu, Heng Ji · (jiqizhixin)

• gorilla - ShishirPatil

  Gorilla: An API store for LLMs · (jiqizhixin) · (mp.weixin.qq)

• ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models, arXiv, 2305.18323, arxiv, pdf, cication: 10

  Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, Dongkuan Xu · (rewoo - billxbf)

• OlaGPT: Empowering LLMs With Human-like Problem-Solving Abilities, arXiv, 2305.16334, arxiv, pdf, cication: 2

  Yuanzhen Xie, Tao Xie, Mingxiong Lin, WenTao Wei, Chenglin Li, Beibei Kong, Lei Chen, Chengxiang Zhuo, Bo Hu, Zang Li · (mp.weixin.qq)

• Natural Language Commanding via Program Synthesis, arXiv, 2306.03460, arxiv, pdf, cication: 1

  Apurva Gandhi, Thong Q. Nguyen, Huitian Jiao, Robert Steen, Ameya Bhatawdekar

• Think Before You Act: Decision Transformers with Internal Working Memory, arXiv, 2305.16338, arxiv, pdf, cication: -1

  Jikun Kang, Romain Laroche, Xindi Yuan, Adam Trischler, Xue Liu, Jie Fu · (qbitai)

• Visual Programming: Compositional visual reasoning without training, arXiv, 2211.11559, arxiv, pdf, cication: -1

  Tanmay Gupta, Aniruddha Kembhavi

---

• Stanford CS25: V3 I Beyond LLMs: Agents, Emergent Abilities, Intermediate-Guided Reasoning, BabyLM - YouTube
• Open-source LLMs as LangChain Agents

---

• 从第一性原理看大模型Agent技术
• AI最大赛道Agent机遇全解析
• Chat 向左，Agent 向右 - 知乎
• 功能超全的AI Agents开源库来了，能写小说，还能当导购、销售 | 机器之心
• AI革新之路：14篇AI Agents论文，探讨人工智能未来
• 数字身份智能体的基本原理及应用前景展望

AutoGPT

• AutoGroq - jgravelle

• plandex - plandex-ai

• aideml - WecoAI

  AIDE: Autonomous AI for Data Science · (weco)

• codel - semanser

  ✨ Fully autonomous AI Agent that can perform complicated tasks and projects using terminal, browser, and editor.

• Data Interpreter: An LLM Agent For Data Science, arXiv, 2402.18679, arxiv, pdf, cication: -1

  Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang · (MetaGPT - geekan)

• AutoDev: Automated AI-Driven Development, arXiv, 2403.08299, arxiv, pdf, cication: -1

  Michele Tufano, Anisha Agarwal, Jinu Jang, Roshanak Zilouchian Moghaddam, Neel Sundaresan

  · (mp.weixin.qq)

• crewAI - joaomdmoura

  Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

• self-operating-computer - OthersideAI

• open-interpreter - KillianLucas

  OpenAI's Code Interpreter in your terminal, running locally.

• ChatDev - OpenBMB

  Create Customized Software using Natural Language Idea (through Multi-Agent Collaboration)

• gpt-researcher - assafelovic

  GPT based autonomous agent that does online comprehensive research on any given topic

• gpt-llm-trainer - mshumer

  · (qbitai)

• MetaGPT - geekan

  The Multi-Agent Meta Programming Framework: Given one line Requirement, return PRD, Design, Tasks, Repo | 多智能体元编程框架：给定老板需求，输出产品文档、架构设计、任务列表、代码

  · (qbitai)

• Toward Actionable Generative AI

• PromptAppGPT - mleoking

  A rapid prompt app development framework based on GPT · (mp.weixin.qq)

• Responsible Task Automation: Empowering Large Language Models as Responsible Task Automators, arXiv, 2306.01242, arxiv, pdf, cication: 2

  Zhizheng Zhang, Xiaoyi Zhang, Wenxuan Xie, Yan Lu · (jiqizhixin)

• Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions, arXiv, 2306.02224, arxiv, pdf, cication: 10

  Hui Yang, Sifu Yue, Yunzhong He · (mp.weixin.qq)

• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society, arXiv, 2303.17760, arxiv, pdf, cication: -1

  Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, Bernard Ghanem

• Language Models can Solve Computer Tasks, arXiv, 2303.17491, arxiv, pdf, cication: 50

  Geunwoo Kim, Pierre Baldi, Stephen McAleer

• SuperAGI - TransformerOptimus

  <⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

• babyagi - yoheinakajima

• Re3: Generating Longer Stories With Recursive Reprompting and Revision, arXiv, 2210.06774, arxiv, pdf, cication: 55

  Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein

• Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents, ICML, 2022, arxiv, pdf, cication: 341

  Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch · (huangwl18.github)

---

• Godmode.space

  · (mp.weixin.qq) · (cognosys) · (doanythingmachine)

• AgentGPT

Augmented LLM

• ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities, arXiv, 2408.04682, arxiv, pdf, cication: -1

  Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Felix Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin · (ToolSandbox - apple)

• TinyAgent: Function Calling at the Edge

• FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions, arXiv, 2403.15246, arxiv, pdf, cication: -1

  Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini

• WhatAreToolsAnyway.pdf

• LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error, arXiv, 2403.04746, arxiv, pdf, cication: -1

  Boshi Wang, Hao Fang, Jason Eisner, Benjamin Van Durme, Yu Su

  · (simulated-trial-and-error - microsoft)

• gorilla - ShishirPatil

  Gorilla: An API store for LLMs · (gorilla.cs.berkeley)

• API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs, arXiv, 2402.15491, arxiv, pdf, cication: -1

  Kinjal Basu, Ibrahim Abdelaziz, Subhajit Chaudhury, Soham Dan, Maxwell Crouse, Asim Munawar, Sadhana Kumaravel, Vinod Muthusamy, Pavan Kapanipathi, Luis A. Lastras

• AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls, arXiv, 2402.04253, arxiv, pdf, cication: -1

  Yu Du, Fangyun Wei, Hongyang Zhang

• Efficient Tool Use with Chain-of-Abstraction Reasoning, arXiv, 2401.17464, arxiv, pdf, cication: -1

  Silin Gao, Jane Dwivedi-Yu, Ping Yu, Xiaoqing Ellen Tan, Ramakanth Pasunuru, Olga Golovneva, Koustuv Sinha, Asli Celikyilmaz, Antoine Bosselut, Tianlu Wang

• LLM Augmented LLMs: Expanding Capabilities through Composition, arXiv, 2401.02412, arxiv, pdf, cication: -1

  Rachit Bansal, Bidisha Samanta, Siddharth Dalmia, Nitish Gupta, Shikhar Vashishth, Sriram Ganapathy, Abhishek Bapna, Prateek Jain, Partha Talukdar

• ProTIP: Progressive Tool Retrieval Improves Planning, arXiv, 2312.10332, arxiv, pdf, cication: -1

  Raviteja Anantha, Bortik Bandyopadhyay, Anirudh Kashi, Sayantan Mahinder, Andrew W Hill, Srinivas Chappidi

• Memory Augmented Language Models through Mixture of Word Experts, arXiv, 2311.10768, arxiv, pdf, cication: -1

  Cicero Nogueira dos Santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David Uthus

• ControlLLM: Augment Language Models with Tools by Searching on Graphs, arXiv, 2310.17796, arxiv, pdf, cication: -1

  Zhaoyang Liu, Zeqiang Lai, Zhangwei Gao, Erfei Cui, Zhiheng Li, Xizhou Zhu, Lewei Lu, Qifeng Chen, Yu Qiao, Jifeng Dai

• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language, arXiv, 2204.00598, arxiv, pdf, cication: 202

  Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani · (socraticmodels.github)

• Understanding Retrieval Augmentation for Long-Form Question Answering, arXiv, 2310.12150, arxiv, pdf, cication: 1

  Hung-Ting Chen, Fangyuan Xu, Shane Arora, Eunsol Choi

• Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model, arXiv, 2310.09520, arxiv, pdf, cication: 1

  Haikang Deng, Colin Raffel

• RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation, arXiv, 2310.04408, arxiv, pdf, cication: -1

  Fangyuan Xu, Weijia Shi, Eunsol Choi

• InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining, arXiv, 2310.07713, arxiv, pdf, cication: -1

  Boxin Wang, Wei Ping, Lawrence McAfee, Peng Xu, Bo Li, Mohammad Shoeybi, Bryan Catanzaro

• RA-DIT: Retrieval-Augmented Dual Instruction Tuning, arXiv, 2310.01352, arxiv, pdf, cication: -1

  Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Rich James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis

• Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization, arXiv, 2308.02151, arxiv, pdf, cication: 6

  Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh Murthy, Zeyuan Chen, Jianguo Zhang, Devansh Arpit

• Meta-training with Demonstration Retrieval for Efficient Few-shot Learning, arXiv, 2307.00119, arxiv, pdf, cication: -1

  Aaron Mueller, Kanika Narang, Lambert Mathias, Qifan Wang, Hamed Firooz

• AVIS: Autonomous Visual Information Seeking with Large Language Model Agent, arXiv, 2306.08129, arxiv, pdf, cication: -1

  Ziniu Hu, Ahmet Iscen, Chen Sun, Kai-Wei Chang, Yizhou Sun, David A Ross, Cordelia Schmid, Alireza Fathi · (mp.weixin.qq)

• Modular Visual Question Answering via Code Generation, arXiv, 2306.05392, arxiv, pdf, cication: 1

  Sanjay Subramanian, Medhini Narasimhan, Kushal Khangaonkar, Kevin Yang, Arsha Nagrani, Cordelia Schmid, Andy Zeng, Trevor Darrell, Dan Klein

• Reimagining Retrieval Augmented Language Models for Answering Queries, arXiv, 2306.01061, arxiv, pdf, cication: -1

  Wang-Chiew Tan, Yuliang Li, Pedro Rodriguez, Richard James, Xi Victoria Lin, Alon Halevy, Scott Yih

• TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs, arXiv, 2303.16434, arxiv, pdf, cication: 113

  Yaobo Liang, Chenfei Wu, Ting Song, Wenshan Wu, Yan Xia, Yu Liu, Yang Ou, Shuai Lu, Lei Ji, Shaoguang Mao · (taskmatrix)

---

• 陈丹琦ACL学术报告来了！详解大模型「外挂」数据库7大方向3大挑战，3小时干货满满 | 量子位

Web browsing

• OmniParser for Pure Vision Based GUI Agent, arXiv, 2408.00203, arxiv, pdf, cication: -1

  Yadong Lu, Jianwei Yang, Yelong Shen, Ahmed Awadallah

• MindSearch - InternLM

  🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)

• AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?, arXiv, 2407.15711, arxiv, pdf, cication: -1

  Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya, Ben Bogin, Ofir Press, Jonathan Berant · (assistantbench.github) · (assistantbench - oriyor)

• DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning, arXiv, 2406.11896, arxiv, pdf, cication: -1

  Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, Aviral Kumar

• fuji-web - normal-computing

  Fuji is an AI agent that lives in your browser's sidepanel. You can now get tasks done online with a single command!

• webllama - McGill-NLP

• AutoCrawler: A Progressive Understanding Web Agent for Web Crawler Generation, arXiv, 2404.12753, arxiv, pdf, cication: -1

  Wenhao Huang, Chenghao Peng, Zhixu Li, Jiaqing Liang, Yanghua Xiao, Liqian Wen, Zulong Chen · (AutoCrawler - EZ-hwh)

• Perplexica - ItzCrazyKns

  Perplexica is an AI-powered search engine. It is an Open source alternative to Perplexity AI

• WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents, arXiv, 2404.05902, arxiv, pdf, cication: -1

  Michael Lutz, Arth Bohra, Manvel Saroyan, Artem Harutyunyan, Giovanni Campagna

• FreeAskInternet - nashsu

  FreeAskInternet is a completely free, private and locally running search aggregator & answer generate using LLM, without GPU needed. The user can ask a question and the system will make a multi engine search and combine the search result to the ChatGPT3.5 LLM and generate the answer based on search results.

• Stream of Search (SoS): Learning to Search in Language, arXiv, 2404.03683, arxiv, pdf, cication: -1

  Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, Noah D. Goodman

  · (stream-of-search - kanishkg)

• AutoWebGLM: Bootstrap And Reinforce A Large Language Model-based Web Navigating Agent, arXiv, 2404.03648, arxiv, pdf, cication: -1

  Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong · (AutoWebGLM - THUDM)

• llm-answer-engine - developersdigest

  Build a Perplexity-Inspired Answer Engine Using Next.js, Groq, Mixtral, Langchain, OpenAI, Brave & Serper

  · (twitter)

• skyvern - Skyvern-AI

  Automate browser-based workflows with LLMs and Computer Vision

• LaVague - lavague-ai

  Automate automation with Large Action Model framework

• api - MULTI-ON

  MultiOn API

• OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web, arXiv, 2402.17553, arxiv, pdf, cication: -1

  Raghav Kapoor, Yash Parag Butala, Melisa Russak, Jing Yu Koh, Kiran Kamble, Waseem Alshikh, Ruslan Salakhutdinov

• VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks, arXiv, 2401.13649, arxiv, pdf, cication: 1

  Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, Daniel Fried · (visualwebarena - web-arena-x) · (mp.weixin.qq)

• WebLINX: Real-World Website Navigation with Multi-Turn Dialogue, arXiv, 2402.05930, arxiv, pdf, cication: -1

  Xing Han Lù, Zdeněk Kasner, Siva Reddy · (mcgill-nlp.github)

• search_with_lepton - leptonai

  Building a quick conversation-based search demo with Lepton AI.

• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models, arXiv, 2401.13919, arxiv, pdf, cication: -1

  Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, Dong Yu

• GPT-4V(ision) is a Generalist Web Agent, if Grounded, arXiv, 2401.01614, arxiv, pdf, cication: -1

  Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, Yu Su · (SeeAct - OSU-NLP-Group) · (osu-nlp-group.github)

• webglm - thudm

  WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023)

• FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation, arXiv, 2310.03214, arxiv, pdf, cication: 2

  Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le · (jiqizhixin)

• GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation, arXiv, 2311.07562, arxiv, pdf, cication: -1

  An Yan, Zhengyuan Yang, Wanrong Zhu, Kevin Lin, Linjie Li, Jianfeng Wang, Jianwei Yang, Yiwu Zhong, Julian McAuley, Jianfeng Gao · (MM-Navigator - zzxslp)

  · (qbitai)

• vimGPT - ishan0102

  Browse the web with GPT-4V and Vimium

• A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis, arXiv, 2307.12856, arxiv, pdf, cication: 13

  Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, Aleksandra Faust

• WebGLM - THUDM

  WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023)

• WebArena: A Realistic Web Environment for Building Autonomous Agents

  · (twitter)

• Query2doc: Query Expansion with Large Language Models, arXiv, 2303.07678, arxiv, pdf, cication: 23

  Liang Wang, Nan Yang, Furu Wei · (mp.weixin.qq)

---

• GPT-4V学会用键鼠上网，人类眼睁睁看着它发帖玩游戏 | 量子位

Retrieval agumented generation

• RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation, arXiv, 2408.02545, arxiv, pdf, cication: -1

  Daniel Fleischer, Moshe Berchansky, Moshe Wasserblat, Peter Izsak · (RAGFoundry) - IntelLabs  - IntelLabs

• Scaling Retrieval-Based Language Models with a Trillion-Token Datastore, arXiv, 2407.12854, arxiv, pdf, cication: -1

  Rulin Shao, Jacqueline He, Akari Asai, Weijia Shi, Tim Dettmers, Sewon Min, Luke Zettlemoyer, Pang Wei Koh · (retrieval-scaling - RulinShao)

• mem0 - mem0ai

  The memory layer for Personalized AI

• Context Embeddings for Efficient Answer Generation in RAG, arXiv, 2407.09252, arxiv, pdf, cication: -1

  David Rau, Shuai Wang, Hervé Déjean, Stéphane Clinchant

• llama-recipes - meta-llama

• cohere-toolkit - cohere-ai

  Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.

• SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation, arXiv, 2406.19215, arxiv, pdf, cication: -1

  Zijun Yao, Weijian Qi, Liangming Pan, Shulin Cao, Linmei Hu, Weichuan Liu, Lei Hou, Juanzi Li

  · (SeaKR - THU-KEG)

• Searching for Best Practices in Retrieval-Augmented Generation, arXiv, 2407.01219, arxiv, pdf, cication: -1

  Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian

• graphrag - microsoft

  A modular graph-based Retrieval-Augmented Generation (RAG) system

• Towards Retrieval Augmented Generation over Large Video Libraries, arXiv, 2406.14938, arxiv, pdf, cication: -1

  Yannis Tevissen, Khalil Guetari, Frédéric Petitpont

• LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs, arXiv, 2406.15319, arxiv, pdf, cication: -1

  Ziyan Jiang, Xueguang Ma, Wenhu Chen · (LongRAG - TIGER-AI-Lab) · (tiger-ai-lab.github)

• PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers, arXiv, 2406.12430, arxiv, pdf, cication: -1

  Myeonghwa Lee, Seonho An, Min-Soo Kim · (PlanRAG - myeon9h)

• Multi-Head RAG: Solving Multi-Aspect Problems with LLMs, arXiv, 2406.05085, arxiv, pdf, cication: -1

  Maciej Besta, Ales Kubicek, Roman Niggli, Robert Gerstenberger, Lucas Weitzendorf, Mingyuan Chi, Patrick Iff, Joanna Gajda, Piotr Nyczyk, Jürgen Müller · (mrag - spcl)

• CRAG -- Comprehensive RAG Benchmark, arXiv, 2406.04744, arxiv, pdf, cication: -1

  Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang · (aicrowd)

• FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research, arXiv, 2405.13576, arxiv, pdf, cication: -1

  Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, Zhicheng Dou · (FlashRAG - RUC-NLPIR)

• HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models, arXiv, 2405.14831, arxiv, pdf, cication: -1

  Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, Yu Su · (hipporag - osu-nlp-group)

• Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts, arXiv, 2405.19893, arxiv, pdf, cication: -1

  Chunjing Gan, Dan Yang, Binbin Hu, Hanxiao Zhang, Siyuan Li, Ziqi Liu, Yue Shen, Lin Ju, Zhiqiang Zhang, Jinjie Gu

• Verba - weaviate

  Retrieval Augmented Generation (RAG) chatbot powered by Weaviate

• context-cite - MadryLab

  Attribute (or cite) statements generated by LLMs back to in-context information. · (gradientscience) · (huggingface)

• When to Retrieve: Teaching LLMs to Utilize Information Retrieval Effectively, arXiv, 2404.19705, arxiv, pdf, cication: -1

  Tiziano Labruna, Jon Ander Campos, Gorka Azkune

• Retrieval Head Mechanistically Explains Long-Context Factuality, arXiv, 2404.15574, arxiv, pdf, cication: -1

  Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, Yao Fu

• phidata - phidatahq

  Add memory, knowledge and tools to LLMs

• Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation, arXiv, 2404.06910, arxiv, pdf, cication: -1

  Thomas Merth, Qichen Fu, Mohammad Rastegari, Mahyar Najibi

• goku - aishwaryaprabhat

  · (linkedin)

• Reducing hallucination in structured outputs via Retrieval-Augmented Generation, arXiv, 2404.08189, arxiv, pdf, cication: -1

  Patrice Béchard, Orlando Marquez Ayala

• A Survey on Retrieval-Augmented Text Generation for Large Language Models, arXiv, 2404.10981, arxiv, pdf, cication: -1

  Yizheng Huang, Jimmy Huang

• How faithful are RAG models? Quantifying the tug-of-war between RAG and LLMs' internal prior, arXiv, 2404.10198, arxiv, pdf, cication: -1

  Kevin Wu, Eric Wu, James Zou

• MaxKB - 1Panel-dev

  💬 基于 LLM 大语言模型的知识库问答系统。开箱即用，支持快速嵌入到第三方业务系统，1Panel 官方出品。

• StreamRAG - video-db

  Video Search and Streaming Agent 🕵️‍♂️

• ARAGOG: Advanced RAG Output Grading, arXiv, 2404.01037, arxiv, pdf, cication: -1

  Matouš Eibich, Shivay Nagpal, Alexander Fred-Ojala

• Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity, arXiv, 2403.14403, arxiv, pdf, cication: -1

  Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, Jong C. Park · (twitter) · (twitter) · (notebooks - cohere-ai) · (youtube)

• AutoRAG - Marker-Inc-Korea

  RAG AutoML Tool - Find optimal RAG pipeline for your own data.

• cookbook - mistralai

• LLocalSearch - nilsherzig

  LLocalSearch is a completely locally running search aggregator using LLM Agents. The user can ask a question and the system will use a chain of LLMs to find the answer. The user can see the progress of the agents and the final answer. No OpenAI or Google API keys are needed.

• ragflow - infiniflow

  RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

• llama_parse - run-llama

  Parse files for optimal RAG

• RAFT: Adapting Language Model to Domain Specific RAG, arXiv, 2403.10131, arxiv, pdf, cication: -1

  Tianjun Zhang, Shishir G. Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, Joseph E. Gonzalez

  · (gorilla - ShishirPatil)

  • (RAFT) for enhancing LLMs for open-book, in-domain question answering by training them to identify and disregard non-helpful "distractor" documents while accurately citing relevant information from the right sources.

• rerankers - AnswerDotAI

• fully-local-pdf-chatbot - jacoblee93

  Yes, it's another chat over documents implementation... but this one is entirely local!

• RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation, arXiv, 2403.05313, arxiv, pdf, cication: -1

  Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma, Yitao Liang

  · (craftjarvis.github)

• Backtracing: Retrieving the Cause of the Query, arXiv, 2403.03956, arxiv, pdf, cication: -1

  Rose E. Wang, Pawan Wirawarn, Omar Khattab, Noah Goodman, Dorottya Demszky · (backtracing - rosewang2008)

  • "backtracing" as a task to help content creators like lecturers identify the text segments that led to user queries, aiming to enhance content delivery in education, news, and conversation domains.

• chat-with-mlx - qnguyen3

  Chat with your data natively on Apple Silicon using MLX Framework.

• In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss, arXiv, 2402.10790, arxiv, pdf, cication: -1

  Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Dmitry Sorokin, Artyom Sorokin, Mikhail Burtsev

• PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers, arXiv, 2402.08327, arxiv, pdf, cication: -1

  Weizhe Lin, Jingbiao Mei, Jinghong Chen, Bill Byrne

  · (preflmr.github) · (jiqizhixin)

• What Evidence Do Language Models Find Convincing?, arXiv, 2402.11782, arxiv, pdf, cication: -1

  Alexander Wan, Eric Wallace, Dan Klein

• ARKS: Active Retrieval in Knowledge Soup for Code Generation, arXiv, 2402.12317, arxiv, pdf, cication: -1

  Hongjin Su, Shuyang Jiang, Yuhang Lai, Haoyuan Wu, Boao Shi, Che Liu, Qian Liu, Tao Yu · (arks - xlang-ai) · (arks-codegen.github)

• Seven Failure Points When Engineering a Retrieval Augmented Generation System, arXiv, 2401.05856, arxiv, pdf, cication: 2

  Scott Barnett, Stefanus Kurniawan, Srikanth Thudumu, Zach Brannelly, Mohamed Abdelrazek · (mp.weixin.qq)

• MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries, arXiv, 2401.15391, arxiv, pdf, cication: -1

  Yixuan Tang, Yi Yang · (MultiHop-RAG - yixuantt)

• GeneGPT - ncbi

  Code and data for GeneGPT.

• trt-llm-rag-windows - NVIDIA

  A developer reference project for creating Retrieval Augmented Generation (RAG) chatbots on Windows using TensorRT-LLM

• RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval, arXiv, 2401.18059, arxiv, pdf, cication: -1

  Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, Christopher D. Manning

  · (RAPTOR - parthsarthi03)

• Corrective Retrieval Augmented Generation, arXiv, 2401.15884, arxiv, pdf, cication: -1

  Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, Zhen-Hua Ling

• flagembedding - flagopen

  Dense Retrieval and Retrieval-augmented LLMs

• autollm - safevideo

  Ship RAG based LLM web apps in seconds.

• The Power of Noise: Redefining Retrieval for RAG Systems, arXiv, 2401.14887, arxiv, pdf, cication: -1

  Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, Fabrizio Silvestri

• RAGatouille - bclavie

• simple-rag - lamini-ai

• pdftochat - Nutlope

  Chat with your PDFs with AI · (pdftochat)

• RAGxplorer - gabrielchua

  Visualise and explore your RAG documents

• RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture, arXiv, 2401.08406, arxiv, pdf, cication: -1

  Aman Gupta, Anup Shirgaonkar, Angels de Luis Balaguer, Bruno Silva, Daniel Holstein, Dawei Li, Jennifer Marsman, Leonardo O. Nunes, Mahsa Rouzbahman, Morris Sharp

• Improving Text Embeddings with Large Language Models, arXiv, 2401.00368, arxiv, pdf, cication: -1

  Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei

• QAnything - netease-youdao

  Question and Answer based on Anything.

• embedchain - embedchain

  The Open Source RAG framework

• Retrieval-Augmented Generation for Large Language Models: A Survey, arXiv, 2312.10997, arxiv, pdf, cication: -1

  Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Haofen Wang · (rag-survey - tongji-kgllm)

  · (mp.weixin.qq)

• CodeFuse-DevOps-Model - codefuse-ai

  DevOps-Models is a series of industrial-first LLMs for theDevOps domain. Asking it for any question in the DevOps domain to get solution!

• codefuse-chatbot - codefuse-ai

  An open-sourced AI assistant/agents for the full-life cycle of AI native software developing, supporting chat interactions plus knowledge base, invoking tools, sandbox execution, etc. · (qbitai)

• Context Tuning for Retrieval Augmented Generation, arXiv, 2312.05708, arxiv, pdf, cication: -1

  Raviteja Anantha, Tharun Bethi, Danil Vodianik, Srinivas Chappidi

• TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents, arXiv, 2312.01279, arxiv, pdf, cication: -1

  James Enouen, Hootan Nakhost, Sayna Ebrahimi, Sercan O Arik, Yan Liu, Tomas Pfister

• LongContext_vs_RAG_NeedleInAHaystack - A-Roucher

  Comparing retrieval abilities from GPT4-Turbo and a RAG system on a toy example for various context lengths

• Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models, arXiv, 2311.09210, arxiv, pdf, cication: -1

  Wenhao Yu, Hongming Zhang, Xiaoman Pan, Kaixin Ma, Hongwei Wang, Dong Yu

• SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models, arXiv, 2311.09818, arxiv, pdf, cication: -1

  Shicheng Liu, Jialiang Xu, Wesley Tjangnaka, Sina J. Semnani, Chen Jie Yu, Monica S. Lam · (suql - stanford-oval)

• Learning to Filter Context for Retrieval-Augmented Generation, arXiv, 2311.08377, arxiv, pdf, cication: -1

  Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md Rizwan Parvez, Graham Neubig · (filco - zorazrw)

• gpt-crawler - BuilderIO

  Crawl a site to generate knowledge files to create your own custom GPT from a URL

• Langchain-Chatchat - chatchat-space

  Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM 等语言模型的本地知识库问答 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM) QA app with langchain

• privateGPT - imartinez

  Interact with your documents using the power of GPT, 100% privately, no data leaks

• KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval, arXiv, 2310.15511, arxiv, pdf, cication: -1

  Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Ghosh Peshawaria, Ranjita Naik, Besmira Nushi

• Langchain-Chatchat - chatchat-space

  Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM 等语言模型的本地知识库问答 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM) QA app with langchain

• DocsGPT - arc53

  GPT-powered chat for documentation, chat with your documents · (qbitai)

• LMDX: Language Model-based Document Information Extraction and Localization, arXiv, 2309.10952, arxiv, pdf, cication: -1

  Vincent Perot, Kai Kang, Florian Luisier, Guolong Su, Xiaoyu Sun, Ramya Sree Boppana, Zilong Wang, Jiaqi Mu, Hao Zhang, Nan Hua

• PDFTriage: Question Answering over Long, Structured Documents, arXiv, 2309.08872, arxiv, pdf, cication: 3

  Jon Saad-Falcon, Joe Barrow, Alexa Siu, Ani Nenkova, David Seunghyun Yoon, Ryan A. Rossi, Franck Dernoncourt

• sec-insights - run-llama

  A real world full-stack application using LlamaIndex

• simplyretrieve - rcgai

  An Easy-to-use Private and Lightweight Retrieval-Centric Generative AI Tool. Create chat tool with your documents and open-source LLMs, highly customizable.

• FastGPT - labring

  A platform that uses the OpenAI API to quickly build an AI knowledge base, supporting many-to-many relationships.

• factool - gair-nlp

  A fact-checking tool that detects factual errors.

• Llama-2-Open-Source-LLM-CPU-Inference - kennethleungty

  Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

• danswer - danswer-ai

  Ask Questions in natural language and get Answers backed by private sources. Connects to tools like Slack, GitHub, Confluence, etc.

• quivr - StanGirard

  🧠 Dump all your files and thoughts into your private GenerativeAI Second Brain and chat with it 🧠

• chatgpt-retrieval - techleadhd

• localGPT - PromtEngineer

  Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.

• privateGPT - imartinez

  Interact privately with your documents using the power of GPT, 100% privately, no data leaks

Embedding

• sqlite-vec - asg017

  A vector search SQLite extension that runs anywhere!

• Piccolo2: General Text Embedding with Multi-task Hybrid Loss Training, arXiv, 2405.06932, arxiv, pdf, cication: -1

  Junqin Huang, Zhongjie Hu, Zihao Jing, Mengya Gao, Yichao Wu · (huggingface)

• Generating Synthetic Data for Fine-Tuning Custom Embedding Models

• NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models, arXiv, 2405.17428, arxiv, pdf, cication: -1

  Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping · (huggingface)

• Piccolo2: General Text Embedding with Multi-task Hybrid Loss Training, arXiv, 2405.06932, arxiv, pdf, cication: -1

  Junqin Huang, Zhongjie Hu, Zihao Jing, Mengya Gao, Yichao Wu · (huggingface)

• snowflake-arctic-embed-m - Snowflake 🤗

  · (snowflake)

• Pile-T5 | EleutherAI Blog

  · (huggingface) · (improved-t5 - EleutherAI)

• LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders, arXiv, 2404.05961, arxiv, pdf, cication: -1

  Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, Siva Reddy

• Gecko: Versatile Text Embeddings Distilled from Large Language Models, arXiv, 2403.20327, arxiv, pdf, cication: -1

  Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding

  · (jiqizhixin)

• FlagEmbedding - FlagOpen

  Retrieval and Retrieval-augmented LLMs · (huggingface)

• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval

• Is Cosine-Similarity of Embeddings Really About Similarity?, arXiv, 2403.05440, arxiv, pdf, cication: -1

  Harald Steck, Chaitanya Ekanadham, Nathan Kallus

• echo-embeddings - jakespringer

• 🪆 Introduction to Matryoshka Embedding Models

• Multilingual E5 Text Embeddings: A Technical Report, arXiv, 2402.05672, arxiv, pdf, cication: -1

  Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei · (unilm - microsoft)

• contrastors - nomic-ai

  Train Models Contrastively in Pytorch · (huggingface) · (mp.weixin.qq)

---

• Smaller, Faster, Cheaper: Introducing Jina Rerankers Turbo and Tiny

• Advanced RAG 10: Corrective Retrieval Augmented Generation (CRAG) | by Florian June | Apr, 2024 | AI Advances

• World's Most Accurate RAG? Langchain/Pinecone, LlamaIndex and EyeLevel Duke it Out

• Unlocking the Power of Multi-Document Agents with LlamaIndex | by Ankush k Singal | Apr, 2024 | AI Advances

  · (twitter)

• Cheap RAGs up for grabs: How we cut LLM costs without sacrificing accuracy? | Pathway

• Introducing RAG 2.0 - Contextual AI

• dspy-gradio-rag - diicellman

  RAG example using DSPy, Gradio, FastAPI

• excellent demo of a highly capable LLM-RAG setup

• Towards Long Context RAG

• The First Place Solution of WSDM Cup 2024: Leveraging Large Language Models for Conversational Multi-Doc QA, arXiv, 2402.18385, arxiv, pdf, cication: -1

  Yiming Li, Zhao Zhang

  · (WSDM-Cup-2024 - zhangzhao219)

• chunk_visualizer - m-ric 🤗

• Stanford CS25: V3 I Retrieval Augmented Language Models - YouTube

• Retrieval Augmented Generation (RAG) for LLMs | Prompt Engineering Guide

• A Cheat Sheet and Some Recipes For Building Advanced RAG | by Andrei | Jan, 2024 | LlamaIndex Blog

• Build a search engine, not a vector DB

---

• 4W字RAG技术总结和串讲
• RAG效果评估经验
• 微调与RAG的优缺点分析
• Langchain中改进RAG能力的3种常用的扩展查询方法
• 通过4个任务比较LangChain和LlamaIndex
• self-RAG｜大模型决策的典型案例探究
• RAG研发真实图鉴：一周出Demo，半年用不好
• 大模型RAG的迭代路径
• 大模型RAG问答技术架构及核心模块回顾
• 【大模型外挂知识库(RAG)优化】如何炼成强大的向量化召回模型 - 知乎
• RAG调优方案
• RAG+GPT-4 Turbo让模型性能飙升！更长上下文不是终局，「大海捞针」实验成本仅4%
• 问答场景常用大模型解决方案

Code Interpreter

• APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets, arXiv, 2406.18518, arxiv, pdf, cication: -1

  Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng · (apigen-pipeline.github) · (huggingface)

• instructor - jxnl

  structured outputs for llms

• FuzzTypes - genomoncology

  Pydantic extension for annotating autocorrecting fields.

• function-calling-eval - interstellarninja

  A framework for evaluating function calls made by LLMs

• Hermes-Function-Calling - NousResearch

• phidata - phidatahq

  Build AI Assistants using function calling

• open-interpreter - KillianLucas

  OpenAI's Code Interpreter in your terminal, running locally

GPTs

• awesome-prompts - ai-boost

  Curated list of chatgpt prompts from the top-rated GPTs in the GPTs Store. Prompt Engineering, prompt attack & prompt protect. Advanced Prompt Engineering papers.

• BlackFriday-GPTs-Prompts - friuns2

  List of free GPTs that doesn't require plus subscription

• GPTs - linexjlin

  leaked prompts of GPTs

• rags - run-llama

• GPT-Baker - abidlabs 🤗

• gpts-works - all-in-aigc

  A Third-party GPTs store

• gpt-crawler - BuilderIO

  Crawl a site to generate knowledge files to create your own custom GPT from a URL

• Awesome-GPTs - ai-boost

  Curated list of awesome GPTs 👍.

• Awesome-GPT-Agents - fr0gger

  A curated list of GPT agents for cybersecurity

• Awesome-GPT-Store - Anil-matcha

  A collection of major GPTS available in public

• awesome-gpts - taranjeet

  Collection of all the GPTs created by the community

• opengpts - langchain-ai

Plugins

• GPT-4调用插件40次都没成功，果断放弃，无效调用、拒绝回答时有发生 | 机器之心k

Other

• Featured GPTs | Best Curated Custom GPTs List for your Daily Tasks
• Discover the Best GPTs
• AI of the day by SamurAI
• 各路大神献出自定义GPT，24小时Top 9名单在这 | 机器之心

Evaluation

• CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents, arXiv, 2407.01511, arxiv, pdf, cication: -1

  Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian · (crab.camel-ai) · (crab - camel-ai)

• MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains, arXiv, 2407.18961, arxiv, pdf, cication: -1

  Guoli Yin, Haoping Bai, Shuang Ma, Feng Nan, Yanchao Sun, Zhaoyang Xu, Shen Ma, Jiarui Lu, Xiang Kong, Aonan Zhang · (axlearn - apple)

• snowflake-arctic-embed-m-v1.5 - Snowflake 🤗

• PersonaGym: Evaluating Persona Agents and LLMs, arXiv, 2407.18416, arxiv, pdf, cication: -1

  Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpurohit, Ameet Deshpande, Karthik Narasimhan, Vishvak Murahari

• tau-bench - sierra-research

  Code and Data for Tau-Bench

• stark - snap-stanford

  Official Code of "STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases"

• MMInA: Benchmarking Multihop Multimodal Internet Agents, arXiv, 2404.09992, arxiv, pdf, cication: -1

  Ziniu Zhang, Shulin Tian, Liangyu Chen, Ziwei Liu · (mmina.cliangyu)

• AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents, arXiv, 2401.13178, arxiv, pdf, cication: -1

  Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, Junxian He · (AgentBoard - hkust-nlp)

• codefuse-devops-eval - codefuse-ai

  Industrial-first evaluation benchmark for LLMs in the DevOps/AIOps domain.

• GAIA: a benchmark for General AI Assistants, arXiv, 2311.12983, arxiv, pdf, cication: -1

  Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, Thomas Scialom · (huggingface)

• Testing Language Model Agents Safely in the Wild, arXiv, 2311.10538, arxiv, pdf, cication: -1

  Silen Naihin, David Atkinson, Marc Green, Merwane Hamadi, Craig Swift, Douglas Schonholtz, Adam Tauman Kalai, David Bau

• BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents, arXiv, 2308.05960, arxiv, pdf, cication: 7

  Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinecke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpit · (BOLAA - salesforce)

• mlagentbench - snap-stanford

• smartplay - microsoft

  SmartPlay is a benchmark for Large Language Models (LLMs). It is designed to be easy to use, and to provide a wide variety of games to test agents on.

• AgentBench: Evaluating LLMs as Agents, arXiv, 2308.03688, arxiv, pdf, cication: 9

  Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang

Other

• How to evaluate an LLM-powered RAG application automatically. - YouTube

• What's next for AI agentic workflows ft. Andrew Ng of AI Fund - YouTube

  · (jiqizhixin)

• Nexus_Function_Calling_Leaderboard - Nexusflow 🤗

• Learning few-shot imitation as cultural transmission | Nature Communications

  · (mp.weixin.qq)

• Rapidly build an application in Gradio power by a Generative AI Agent | Google Cloud Blog

• 吴恩达：AI智能体工作流今年将有巨大进展，可能超过下一代基础模型 | 机器之心

• 从第一性原理看大模型Agent技术

• 万字长文！何谓Agent，为何Agent？

• 首个获得驾照的AI！Agent担任私人助理样样精通，还能帮助考试作弊

• 多智能体(Agents)协作框架：人工智能的下一个方向和挑战

• Agent 将是 AI 最大的赛道！

Vector Database

• awesome-vector-database - dangkhoasdc

  A curated list of awesome works related to high dimensional structure/vector search & database

• How to choose your vector database in 2023?

  · (youtube)

Other

• GPT成功背后的秘密--向量数据库简介 - 知乎
• 7个向量数据库对比：Milvus、Pinecone、Vespa、Weaviate、Vald、GSI 和 Qdrant - 墨天轮

Extra reference

• awesome-large-multimodal-agents - jun0wanan

• llm-agent-survey - paitesanshi

• awesome-ai-agents - e2b-dev

  A list of AI autonomous agents

• generative_agents - joonspk-research

  Generative Agents: Interactive Simulacra of Human Behavior