[Feature Request] Document toolkits

[Wendong-Fan]

Required prerequisites

• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Motivation

1. Has the ability to process universal document types whether they are in urls or local files. It is supported by other bottom-level toolkits such as video toolkit, audio toolkit, etc. It uses GraphRAG to process raw text and query.

2. Support text modification, changing existing text to improve quality, adjust tone, or better suit a specific audience. This is a fundamental task in content creation and editing. (low priority)

some implementation already done by MengKang

Solution

No response

Alternatives

No response

Additional context

No response

[Wendong-Fan]

lead: @willshang76 ; support & review: @Aaron617 , @AveryYay

[Aaron617]

After testing the current document toolkit on GAIA, me and @Ralph-Zhou found several problems:

• After processing longer content, it is basically converted into a RAG task. However, the current RAG capability is still somewhat behind agents like H2O. (current reference repository: https://github.com/shibing624/ChatPDF)
• Function extract_webpage_content needs enhancement. Currently it may miss some content (for example, the table is not processed well).