[BFCL] Improved tree-sitter java, javascript installation

berkeley-function-call-leaderboard/README.md

@@ -24,24 +24,6 @@ pip install vllm # If you have vLLM supported GPU(s) and want to run our evaluat
 ```
 If you plan to evaluate on OSS models, we are using vLLM for inference and refer to https://github.com/vllm-project/vllm for detail. We recommend to inference on at least V100s, A100s, and latest GPUs that are supported by vLLM. 
 
-### Checker Setup (required for Java, JavaScript test categories)
-We use `tree-sitter` to do the AST parsing for Java and JavaScript test categories. Thus, you need to install `tree-sitter`.
-
-The git clones need to be under the `/berkeley-function-call-leaderboard/eval_checker` folder.
-
-```bash
-cd ./eval_checker
-git clone https://github.com/tree-sitter/tree-sitter-java.git
-git clone https://github.com/tree-sitter/tree-sitter-javascript.git
-```
-
-Now, move back to `/berkeley-function-call-leaderboard` by `cd ..`, and create two symbolic links to the `tree-sitter-java` and `tree-sitter-javascript` directories. This is required to run `openfunctions_evaluation.py`.
-
-```
-ln -s eval_checker/tree-sitter-java tree-sitter-java
-ln -s eval_checker/tree-sitter-javascript tree-sitter-javascript
-```
-
 ## Prepare Evaluation Dataset
 
 To download the evaluation dataset from huggingface, from the current directory `./berkeley-function-call-leaderboard`, run the following command:

berkeley-function-call-leaderboard/model_handler/java_parser.py

@@ -1,13 +1,9 @@
 import json
 from tree_sitter import Language, Parser
+import tree_sitter_java
+
+JAVA_LANGUAGE = Language(tree_sitter_java.language(), "java")
 
-Language.build_library(
-    # Store the library in the `build` directory
-    "build/tree_sitter.so",
-    # Include one or more languages
-    ["./tree-sitter-java"],
-)
-JAVA_LANGUAGE = Language("build/tree_sitter.so", "java")
 parser = Parser()
 parser.set_language(JAVA_LANGUAGE)
 

berkeley-function-call-leaderboard/model_handler/js_parser.py

@@ -1,10 +1,9 @@
 import json
 from tree_sitter import Language, Parser
+import tree_sitter_javascript
 
-# Load your language grammar and create a parser
-Language.build_library("build/tree_sitter_js.so", ["./tree-sitter-javascript"])
+JS_LANGUAGE = Language(tree_sitter_javascript.language(), "javascript")
 
-JS_LANGUAGE = Language("build/tree_sitter_js.so", "javascript")
 parser = Parser()
 parser.set_language(JS_LANGUAGE)
 

berkeley-function-call-leaderboard/requirements.txt

@@ -9,3 +9,5 @@ anthropic~=0.29.0
 openai
 numpy
 cohere~=5.2.5
+tree-sitter-java==0.21.0
+tree-sitter-javascript==0.21.4