Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add LangChain ReAct impl to examples/FinanceBench #347

Merged
merged 50 commits into from
Sep 15, 2024
Merged
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
3832d67
create langchain react agent
Sep 1, 2024
a778f6f
fix path
Sep 1, 2024
0e19938
refactor
Sep 1, 2024
5f38090
refactor
Sep 1, 2024
d6a2aa2
rename examples/FinanceBench/langchain_react module
TheVinhLuong102 Sep 1, 2024
a360e12
Revert "rename examples/FinanceBench/langchain_react module"
TheVinhLuong102 Sep 1, 2024
fd98a56
add examples/FinanceBench/export-multi-runs module
TheVinhLuong102 Sep 1, 2024
4536191
optimize running time
Sep 2, 2024
1b9ef02
make the code easier to use
Sep 2, 2024
40ed13f
handle bugs
Sep 3, 2024
0839f57
Merge branch 'examples/FinanceBench' of https://GitHub.com/Aitomatic/…
TheVinhLuong102 Sep 3, 2024
4c3378c
add WiP examples/FinanceBench/langchain_react module
TheVinhLuong102 Sep 3, 2024
1b1ceb2
update WiP examples/FinanceBench/langchain_react module
TheVinhLuong102 Sep 3, 2024
35cf6ea
fix bug agent ReAct
Sep 3, 2024
109b088
Merge branch 'example-langchain-react-agent' of github.com:aitomatic/…
Sep 3, 2024
5fa7800
change backbone to gpt4o
Sep 3, 2024
e299454
fix bug
Sep 3, 2024
75cff1d
add optional LangChain dependencies
TheVinhLuong102 Sep 3, 2024
d11db23
add optional FAISS dependency
TheVinhLuong102 Sep 3, 2024
d8bdf0d
update examples/FinanceBench LangChain ReAct implementation
TheVinhLuong102 Sep 3, 2024
f606c67
update examples/FinanceBench LangChain ReAct implementation
TheVinhLuong102 Sep 3, 2024
37a8285
update examples/FinanceBench LangChain ReAct implementation
TheVinhLuong102 Sep 3, 2024
90fe183
add logging to capture reasoning steps
Sep 3, 2024
006b27f
show intermediate steps
Sep 3, 2024
a605987
update examples/FinanceBench financebench_id_01935 ground truth
TheVinhLuong102 Sep 3, 2024
572235e
update examples/FinanceBench financebench_id_00476 ground truth
TheVinhLuong102 Sep 3, 2024
849b800
update examples/FinanceBench financebench_id_01351 ground truth
TheVinhLuong102 Sep 3, 2024
d824fa2
update examples/FinanceBench financebench_id_00685 ground truth
TheVinhLuong102 Sep 3, 2024
589f0bc
update examples/FinanceBench financebench_id_01346 ground truth
TheVinhLuong102 Sep 3, 2024
06a0c75
update examples/FinanceBench financebench_id_01484 ground truth
TheVinhLuong102 Sep 3, 2024
2e87e8e
update examples/FinanceBench financebench_id_04171 ground truth
TheVinhLuong102 Sep 4, 2024
27b32ee
update examples/FinanceBench financebench_id_03282 ground truth
TheVinhLuong102 Sep 4, 2024
ca7f517
update examples/FinanceBench financebench_id_01476 ground truth
TheVinhLuong102 Sep 4, 2024
4ddc90f
update examples/FinanceBench financebench_id_00651 ground truth
TheVinhLuong102 Sep 4, 2024
6668c6b
update examples/FinanceBench financebench_id_01928 ground truth
TheVinhLuong102 Sep 4, 2024
e2290fd
feat: eval consistency
Sep 4, 2024
40f9cfe
Merge branch 'example-langchain-react-agent' of github.com:aitomatic/…
Sep 4, 2024
ae2895b
update examples/FinanceBench financebench_id_00540 ground truth
TheVinhLuong102 Sep 4, 2024
8f2c521
update examples/FinanceBench financebench_id_00678 ground truth
TheVinhLuong102 Sep 4, 2024
9536d86
update processing 5 answers
Sep 13, 2024
3e2d26a
Merge branch 'example-langchain-react-agent' of github.com:aitomatic/…
Sep 13, 2024
99f773c
update examples/FinanceBench financebench_id_01198 ground truth
TheVinhLuong102 Sep 14, 2024
acb91d1
update examples/FinanceBench financebench_id_00070 ground truth
TheVinhLuong102 Sep 14, 2024
6393518
update examples/FinanceBench financebench_id_02416 ground truth
TheVinhLuong102 Sep 15, 2024
3de18ce
add WiP examples/FinanceBench/eval.eval_accuracy_and_consistency_wrt_…
TheVinhLuong102 Sep 15, 2024
8a04f59
update examples/FinanceBench/eval.eval_accuracy_and_consistency_wrt_g…
TheVinhLuong102 Sep 15, 2024
b45bf23
update examples/FinanceBench/eval.eval_accuracy_and_consistency_wrt_g…
TheVinhLuong102 Sep 15, 2024
7f8b786
temporarily moving LangChain & consistency implementations in example…
TheVinhLuong102 Sep 15, 2024
d3e8973
update PyLint linting configs
TheVinhLuong102 Sep 15, 2024
e797c0e
Merge pull request #348 from aitomatic/lint
TheVinhLuong102 Sep 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
refactor
William Nguyen authored and William Nguyen committed Sep 1, 2024
commit 0e199388fce04dff22e9a5c592d469606e08306e
4 changes: 2 additions & 2 deletions examples/FinanceBench/langchain-react-agent/call-agent.py
Original file line number Diff line number Diff line change
@@ -38,7 +38,7 @@ def ask_question(react_agent, query):
response = react_agent.invoke({"query": query})
return response['result']

def main(queries, pdf_paths):
def prediction(queries, pdf_paths):
print("Loading and preparing PDFs...")
documents = load_and_prepare_pdfs(pdf_paths)

@@ -74,7 +74,7 @@ def process_questions(data_file, pdf_paths, output_file):

queries = df['question'].tolist()

question_answer_pairs = main(queries, pdf_paths)
question_answer_pairs = prediction(queries, pdf_paths)

drop_columns = ["answer", "OpenAI-Asst", "OpenAI-Asst---CORRECTNESS"]