feat: Add structured data parsing #1364

liuxukun2000 · 2024-12-27T00:43:34Z

Description

#1325

Motivation and Context

close #1325

I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds core functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation (update in the documentation)
Example (update in the folder of example)

Implemented Tasks

Subtask 1
Subtask 2
Subtask 3

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

I have read the CONTRIBUTION guide. (required)
My change requires a change to the documentation.
I have updated the tests accordingly. (required for a bug fix or a new feature)
I have updated the documentation accordingly.

#1304)

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

… output. (#1305) Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Co-authored-by: Simon <simon.xianming@ext.deposit-solution.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

…1206) Co-authored-by: Isaac Jin <whale3ye@gmail.com>

… proxy support (#1323)

… in test units to enhance test robustness (#1128) Co-authored-by: yh 林 <yhlin@yhdebijibendiannao.local> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

#1318) Co-authored-by: Harry Ye <116691547+harryeqs@users.noreply.github.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: “yifeng.wang” <“3038880699@qq.com;q:wqqgit config --global user.name “yifeng.wang”git config --global user.email “3038880699@qq.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Signed-off-by: Caelum Forder <caelum119@gmail.com>

… QWQ (#1369) Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Signed-off-by: Caelum Forder <caelum119@gmail.com>

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: “yifeng.wang” <“3038880699@qq.com;q:wqqgit config --global user.name “yifeng.wang”git config --global user.email “3038880699@qq.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

…f cook… (#1385) Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

MuggleJinx

The rest looks cool to me.

MuggleJinx · 2025-01-03T14:09:59Z

examples/loaders/panda_example.py

@@ -46,5 +58,3 @@ def main():



Add the expected output of the print statement as comments.

review-notebook-app · 2025-01-03T17:39:56Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Wendong-Fan · 2025-01-06T15:33:16Z

close this pr as #1395 opened

Wendong-Fan and others added 30 commits December 11, 2024 00:50

docs: update customer_service_Discord_bot_using_SambaNova_with_agenti… (

0de7531

#1304)

feat: Add llama3.3_70B model (#1306)

a60a682

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

fix: outdated code snippets for loaders module in doc (#1310)

99aedcd

feat: support ollama.com structured-outputs Support ollama structured…

ed92297

… output. (#1305) Co-authored-by: Wendong <w3ndong.fan@gmail.com>

feat: Reward model (#1271)

b3a2fde

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

feat: integrate sglang in camel (#1255)

2063923

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

Fix nemotron name (#1314)

5af18e6

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

feat: Integrate stripe (#1261)

7d9c906

Co-authored-by: Simon <simon.xianming@ext.deposit-solution.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

chore: Update some versions (#1311)

d40f5a0

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

feat: Integrate e2b. (#1284)

7e20640

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

feat: add data collector for dataset generation (#1193)

33c2787

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

feat: set structured output from ChatAgent using openai beta client (#…

d9fec49

…1206) Co-authored-by: Isaac Jin <whale3ye@gmail.com>

Fix: Import anthropic package when used since it's optional (#1319)

1659bd9

chore: new version release and test fix (#1320)

a3df42a

chore: update README and CONTRIBUTING (#1322)

7f6f970

feat: openai tokencounter disallowed_special setting & google scholar…

76afef8

… proxy support (#1323)

docs: add chunkr cookbook and hermes data gen cookbook (#1328)

fd1c38b

chore: Add mock LLM for test purposes; Enable multiple step() calls…

6890945

… in test units to enhance test robustness (#1128) Co-authored-by: yh 林 <yhlin@yhdebijibendiannao.local> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

docs: update agent_with_chunkr_for_pdf_parsing (#1331)

1b1564b

fix: Bump up unstructured io version (#1329)

111975f

feat: increate timeout for LLMs, release new version (#1335)

70df478

feat: Integrate fish audio (#1287)

d779a06

feat: Implement outlines converter model for structured output (#1211) (

89691b8

#1318) Co-authored-by: Harry Ye <116691547+harryeqs@users.noreply.github.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

docs: add cookbook contribution guideline (#1337)

90d4c6d

feat: Support OpenAI O1 model (#1340)

3afaaf7

fix: logger setting for print_text_animated (#1341)

1af14ee

fix: Remove strict:ture for OpenAI.chat.completions.create (#1344)

65420f1

release: Version 0.2.14 (#1346)

ee68e9f

Fix: Remove leading newlines from tool calls (#1350)

cb65696

Signed-off-by: Caelum Forder <caelum119@gmail.com>

Wendong-Fan and others added 15 commits December 29, 2024 19:02

fix: enhancment for tool benchmark integration (#1370)

986e06a

docs: updated the local model for discord bot cookbook from llama3 to…

4c27173

… QWQ (#1369) Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

fix: api_keys_required (#1277)

487ed43

docs: cited self-instruct paper (#1372)

c9ef72c

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com> Co-authored-by: Wendong <w3ndong.fan@gmail.com>

fix: typo (#1380)

da5b74d

Signed-off-by: Caelum Forder <caelum119@gmail.com>

Add hint about installing CAMEL from source / topic branch (#1381)

bfbdd72

Signed-off-by: Caelum Forder <caelum119@gmail.com>

feat: Integrate OpenBB Into Camel (#1300)

1b4afc8

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

Fix: circular import of FunctionTool and ChatAgent (#1379)

4b39748

Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

chore:change the name of CoTDataGenerator and update the share link o…

1e09067

…f cook… (#1385) Co-authored-by: Wendong <w3ndong.fan@gmail.com>

index fix

3ea2b05

feat: nebula graph add time label (#1383)

fdc727b

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

chore: sglang doc, logger setting, version release (#1317)

0d5071b

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

fix: doc file name

6752840

add test, rewrite code

937e1b2

liuxukun2000 requested a review from Wendong-Fan January 3, 2025 07:00

MuggleJinx reviewed Jan 3, 2025

View reviewed changes

examples/loaders/panda_example.py

@@ -46,5 +58,3 @@ def main():

Copy link

Collaborator

MuggleJinx Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the expected output of the print statement as comments.

amaan-ai20 and others added 2 commits January 4, 2025 01:20

feat: Add Dappier toolkit (#1393)

bfd2a71

chore: docstring format dappier (#1394)

ced27fd

Wendong-Fan force-pushed the feat/structured_loader branch from d8028b9 to 937e1b2 Compare January 3, 2025 17:44

Wendong-Fan and others added 7 commits January 4, 2025 01:48

Merge branch 'master' into feat/structured_loader

3d80abb

fix: incorrect role assigning in rewardmodelfilter (#1390)

e262fd1

Co-authored-by: Wendong <w3ndong.fan@gmail.com> Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>

chore: update ruff version

6cb00f5

fix: update anthropic (#1392)

fdf08dd

Co-authored-by: Wendong <w3ndong.fan@gmail.com>

clean up commit history

9b13192

remove unused vars

9fa3cc5

Merge branch 'master' into feat/structured_loader

9235a3e

Wendong-Fan closed this Jan 6, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add structured data parsing #1364

feat: Add structured data parsing #1364

liuxukun2000 commented Dec 27, 2024 •

edited by Wendong-Fan

Loading

MuggleJinx left a comment

MuggleJinx Jan 3, 2025

review-notebook-app bot commented Jan 3, 2025

Wendong-Fan commented Jan 6, 2025

feat: Add structured data parsing #1364

feat: Add structured data parsing #1364

Conversation

liuxukun2000 commented Dec 27, 2024 • edited by Wendong-Fan Loading

Description

Motivation and Context

Types of changes

Implemented Tasks

Checklist

MuggleJinx left a comment

Choose a reason for hiding this comment

MuggleJinx Jan 3, 2025

Choose a reason for hiding this comment

review-notebook-app bot commented Jan 3, 2025

Wendong-Fan commented Jan 6, 2025

liuxukun2000 commented Dec 27, 2024 •

edited by Wendong-Fan

Loading