-
Notifications
You must be signed in to change notification settings - Fork 4.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Add] Added function calling support to GeminiClient #2793
base: main
Are you sure you want to change the base?
Conversation
I am learning python just for the past few days. Kindly give me feedback on what I can improve in this. |
Thanks for your great PR @arjun-g !! There are two tiny stuff missing: test cases and tutorial notebook. |
…el. Added test case for function_call.
@BeibinLi added a test case. Let me know if this is suffice |
@arjun-g - A notebook example of using this would be great to have. |
@BeibinLi added a notebook as well. |
Hi, @r4881t This is unrelated to function calling. This is a problem with vertexai implementation itself. Will give a separate PR for this. |
I'll do some quick tests now... |
Okay, running through a set of test programs (which I've put on my repository), it looks good, however there was an issue with parallel tool calling (when the model suggests two or more tool-calls in one response). It's only handling one tool-call suggestion. I'll add my code in to correct that. My repo with test programs that you can use to test: I ignored the streaming ones as streaming isn't supported yet in the code |
Nice work @arjun-g :)... just need to run Let us know if you need help with that. |
|
GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
---|---|---|---|---|---|
12853598 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
10404693 | Triggered | Generic High Entropy Secret | e43a86c | test/oai/test_utils.py | View secret |
10404693 | Triggered | Generic High Entropy Secret | bdb40d7 | test/oai/test_utils.py | View secret |
10404693 | Triggered | Generic High Entropy Secret | 954ca45 | test/oai/test_utils.py | View secret |
10404693 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
10404662 | Triggered | Generic CLI Secret | eff19ac | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | 06a0a5d | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | 0524c77 | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | d7ea410 | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | e43a86c | .github/workflows/dotnet-build.yml | View secret |
10404662 | Triggered | Generic CLI Secret | 841ed31 | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | 802f099 | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | 9a484d8 | .github/workflows/dotnet-build.yml | View secret |
10404662 | Triggered | Generic CLI Secret | e973ac3 | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | 89650e7 | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | e07b06b | .github/workflows/dotnet-release.yml | View secret |
10404662 | Triggered | Generic CLI Secret | abe4c41 | .github/workflows/dotnet-build.yml | View secret |
10404662 | Triggered | Generic CLI Secret | 7362fb9 | .github/workflows/dotnet-release.yml | View secret |
12853599 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
10404694 | Triggered | Generic High Entropy Secret | e43a86c | test/oai/test_utils.py | View secret |
10404694 | Triggered | Generic High Entropy Secret | 954ca45 | test/oai/test_utils.py | View secret |
10404694 | Triggered | Generic High Entropy Secret | bdb40d7 | test/oai/test_utils.py | View secret |
10404695 | Triggered | Generic High Entropy Secret | abad9ff | test/oai/test_utils.py | View secret |
10404695 | Triggered | Generic High Entropy Secret | 954ca45 | test/oai/test_utils.py | View secret |
10404695 | Triggered | Generic High Entropy Secret | c7bb588 | test/oai/test_utils.py | View secret |
10404695 | Triggered | Generic High Entropy Secret | b97b99d | test/oai/test_utils.py | View secret |
10404695 | Triggered | Generic High Entropy Secret | e43a86c | test/oai/test_utils.py | View secret |
12853600 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
12853601 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
10493810 | Triggered | Generic Password | 49e8053 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 501610b | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 49e8053 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 501610b | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | d422c63 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 97fa339 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 49e8053 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | d422c63 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 97fa339 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | d422c63 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 97fa339 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 501610b | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10404696 | Triggered | Generic High Entropy Secret | 954ca45 | test/oai/test_utils.py | View secret |
10404696 | Triggered | Generic High Entropy Secret | bdb40d7 | test/oai/test_utils.py | View secret |
10404696 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
10404696 | Triggered | Generic High Entropy Secret | e43a86c | test/oai/test_utils.py | View secret |
10422482 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
10422482 | Triggered | Generic High Entropy Secret | bdb40d7 | test/oai/test_utils.py | View secret |
12853602 | Triggered | Generic High Entropy Secret | 79dbb7b | test/oai/test_utils.py | View secret |
11616921 | Triggered | Generic High Entropy Secret | a86d0fd | notebook/agentchat_agentops.ipynb | View secret |
11616921 | Triggered | Generic High Entropy Secret | 394561b | notebook/agentchat_agentops.ipynb | View secret |
11616921 | Triggered | Generic High Entropy Secret | 3eac646 | notebook/agentchat_agentops.ipynb | View secret |
11616921 | Triggered | Generic High Entropy Secret | f45b553 | notebook/agentchat_agentops.ipynb | View secret |
11616921 | Triggered | Generic High Entropy Secret | 6563248 | notebook/agentchat_agentops.ipynb | View secret |
12853598 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
12853598 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
10404693 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
10404693 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
10404693 | Triggered | Generic High Entropy Secret | 0a3c6c4 | test/oai/test_utils.py | View secret |
10404693 | Triggered | Generic High Entropy Secret | 76f5f5a | test/oai/test_utils.py | View secret |
10404662 | Triggered | Generic CLI Secret | 954ca45 | .github/workflows/dotnet-build.yml | View secret |
12853599 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
12853599 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
10404694 | Triggered | Generic High Entropy Secret | 76f5f5a | test/oai/test_utils.py | View secret |
10404694 | Triggered | Generic High Entropy Secret | 0a3c6c4 | test/oai/test_utils.py | View secret |
10404695 | Triggered | Generic High Entropy Secret | 3b79cc6 | test/oai/test_utils.py | View secret |
10404695 | Triggered | Generic High Entropy Secret | 11baa52 | test/oai/test_utils.py | View secret |
12853600 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
12853600 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
12853601 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
12853601 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
10493810 | Triggered | Generic Password | 3b79cc6 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 11baa52 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 11baa52 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10493810 | Triggered | Generic Password | 3b79cc6 | notebook/agentchat_pgvector_RetrieveChat.ipynb | View secret |
10404696 | Triggered | Generic High Entropy Secret | 0a3c6c4 | test/oai/test_utils.py | View secret |
10404696 | Triggered | Generic High Entropy Secret | 76f5f5a | test/oai/test_utils.py | View secret |
10404696 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
10404696 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
10422482 | Triggered | Generic High Entropy Secret | 2b3a9ae | test/oai/test_utils.py | View secret |
10422482 | Triggered | Generic High Entropy Secret | c03558f | test/oai/test_utils.py | View secret |
and 16 others.
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secrets safely. Learn here the best practices.
- Revoke and rotate these secrets.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2793 +/- ##
==========================================
- Coverage 32.45% 25.59% -6.87%
==========================================
Files 93 99 +6
Lines 10109 10261 +152
Branches 2172 2358 +186
==========================================
- Hits 3281 2626 -655
- Misses 6544 7325 +781
- Partials 284 310 +26
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Is there anything else you need here @arjun-g? This is an important component for the gemini client I'd love to make use of in a more official capacity. I'm happy to help wrap up. |
@robraux I am held up in a work. Will update the PR in couple of days, |
It would be really nice to have this feature 🙂 |
Looking forward to having this feature 🙂 |
@arjun-g I have prepared a pull request to your repo that would resolve the merge conflicts, synchronize updates from the upstream autogen repo, and also add support for tool config with function calling. Only changes to gemini.py , test_gemini.py , and openai_utils.py need to be reviewed, as the rest is changed due to synchronization with the official autogen repo except for agentchat_gemini.ipynb, where the difference is only automated formatting, which is not visible to the user anyway and thus irrelevant. I hope this would save you some effort in case you still do not have time to finish this PR 🙂 Link to the PR: arjun-g#1 |
Why are these changes needed?
I have added function calling ability to GeminiClient so that we can use autogen tools with Gemini as well.
Related issue number
Completes "Function Calling" roadmap item in #2387
Checks