[BFCL] Improved tree-sitter java, javascript installation (ShishirPat…

…il#505)

**Purpose**:
- One step addressing ShishirPatil#501 

**Changes**:
- remove steps of cloning repo, symlinks, builds
- add `tree-sitter-java` and `tree-sitter-javascript` in
requirements.txt with pip install

**Test**:
- Tested end-to-end BFCL generation and checked on
gorilla-openfunctions-v2 and llama3-70B.

Co-authored-by: Shishir Patil <30296397+ShishirPatil@users.noreply.github.com>

berkeley-function-call-leaderboard/README.md

@@ -21,25 +21,6 @@ pip install -r requirements.txt # Inside gorilla/berkeley-function-call-leaderbo
 pip install vllm # If you have vLLM supported GPU(s) and want to run our evaluation data against self-hosted OSS models.
 ```
 
-### Evaluation Checker Setup (only required for Java and JavaScript test categories)
-
-We use `tree-sitter` for AST parsing of Java and JavaScript function calls.
-
-The git clones need to be under the `gorilla/berkeley-function-call-leaderboard/eval_checker` directory.
-
-```bash
-cd eval_checker # Navigate into gorilla/berkeley-function-call-leaderboard/eval_checker
-git clone https://github.com/tree-sitter/tree-sitter-java.git
-git clone https://github.com/tree-sitter/tree-sitter-javascript.git
-```
-
-Now, move back to `gorilla/berkeley-function-call-leaderboard`, and create two symbolic links to the `tree-sitter-java` and `tree-sitter-javascript` directories. This is required to run `openfunctions_evaluation.py`.
-
-```bash
-cd .. # Navigate into gorilla/berkeley-function-call-leaderboard
-ln -s eval_checker/tree-sitter-java tree-sitter-java
-ln -s eval_checker/tree-sitter-javascript tree-sitter-javascript
-```
 
 ## Prepare Evaluation Dataset
 

berkeley-function-call-leaderboard/model_handler/java_parser.py

@@ -1,13 +1,9 @@
 import json
 from tree_sitter import Language, Parser
+import tree_sitter_java
+
+JAVA_LANGUAGE = Language(tree_sitter_java.language(), "java")
 
-Language.build_library(
-    # Store the library in the `build` directory
-    "build/tree_sitter.so",
-    # Include one or more languages
-    ["./tree-sitter-java"],
-)
-JAVA_LANGUAGE = Language("build/tree_sitter.so", "java")
 parser = Parser()
 parser.set_language(JAVA_LANGUAGE)
 

berkeley-function-call-leaderboard/model_handler/js_parser.py

@@ -1,10 +1,9 @@
 import json
 from tree_sitter import Language, Parser
+import tree_sitter_javascript
 
-# Load your language grammar and create a parser
-Language.build_library("build/tree_sitter_js.so", ["./tree-sitter-javascript"])
+JS_LANGUAGE = Language(tree_sitter_javascript.language(), "javascript")
 
-JS_LANGUAGE = Language("build/tree_sitter_js.so", "javascript")
 parser = Parser()
 parser.set_language(JS_LANGUAGE)
 

berkeley-function-call-leaderboard/requirements.txt

@@ -9,3 +9,5 @@ anthropic~=0.29.0
 openai
 numpy
 cohere~=5.2.5
+tree-sitter-java==0.21.0
+tree-sitter-javascript==0.21.4