Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BFCL] some tiny fix in possible_answer #786

Merged
merged 1 commit into from
Nov 25, 2024

Conversation

zhangch-ss
Copy link
Contributor

I found an error ["Invalid value for parameter 'name': 'John Doe'. Expected one of ['j', 'o', 'h', 'n', '', 'd', 'o', 'e']."], in BFCL_v3_live_multiple_score.json.
this is because "John Doe" is not a list.
{"id": "live_multiple_189-83-0", "ground_truth": [{"sendHttpRequest": {"method": ["POST"], "url": ["https://httpbin.org/post"], "headers": [{"Content-Type": ["application/json"]}], "data": [{"name": "John Doe", "email": "john.doe@example.com"}]}}]}
I made a list of all similar errors and fixed them

ERROR:root:java_67:[{'APP_NAME': 'PhotoEditor'}, {'IMAGES_ROOT': '/path/to/images'}, {'BACKGROUND_IMAGE_FOLDER': 'background'}, {'BACKGROUND_IMAGE': 'custom-bg.png'}] value is not a list: PhotoEditor
ERROR:root:java_67:[{'APP_NAME': 'PhotoEditor'}, {'IMAGES_ROOT': '/path/to/images'}, {'BACKGROUND_IMAGE_FOLDER': 'background'}, {'BACKGROUND_IMAGE': 'custom-bg.png'}] value is not a list: /path/to/images
ERROR:root:java_67:[{'APP_NAME': 'PhotoEditor'}, {'IMAGES_ROOT': '/path/to/images'}, {'BACKGROUND_IMAGE_FOLDER': 'background'}, {'BACKGROUND_IMAGE': 'custom-bg.png'}] value is not a list: background
ERROR:root:java_67:[{'APP_NAME': 'PhotoEditor'}, {'IMAGES_ROOT': '/path/to/images'}, {'BACKGROUND_IMAGE_FOLDER': 'background'}, {'BACKGROUND_IMAGE': 'custom-bg.png'}] value is not a list: custom-bg.png
ERROR:root:java_68:[{'/Applications/MyApp.app': 'MyApp'}] value is not a list: MyApp
ERROR:root:javascript_5:[{'useStateSelector': 'useStateSelectorHook', 'useDispatchAction': 'useDispatchActionHook'}] value is not a list: useStateSelectorHook
ERROR:root:javascript_5:[{'useStateSelector': 'useStateSelectorHook', 'useDispatchAction': 'useDispatchActionHook'}] value is not a list: useDispatchActionHook
ERROR:root:live_multiple_189-83-0:[{'name': 'John Doe', 'email': 'john.doe@example.com'}] value is not a list: John Doe
ERROR:root:live_multiple_189-83-0:[{'name': 'John Doe', 'email': 'john.doe@example.com'}] value is not a list: john.doe@example.com

@HuanzhiMao HuanzhiMao added the BFCL-Dataset BFCL Dataset-Related Issue label Nov 25, 2024
Copy link
Collaborator

@HuanzhiMao HuanzhiMao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @zhangch-ss and good catch!

@HuanzhiMao HuanzhiMao changed the title some tiny fix in possible_answer [BFCL] some tiny fix in possible_answer Nov 25, 2024
@HuanzhiMao HuanzhiMao merged commit 716ea6e into ShishirPatil:main Nov 25, 2024
HuanzhiMao added a commit that referenced this pull request Dec 7, 2024
This PR updates the leaderboard to reflect the change in score due to
the following PR merge:

1. #747 
2. #770 
3. #768 
4. #750 
5. #763 
6. #772 
7. #777 
8. #778 
9. #786 
10. #787 
11. #697 
12. #718 
13. #755 
14. #796 
15. #789 
16. #804 
17. #808 
18. #809
19. #811 
20. #810 

Models were evaluated using checkpoint commit d7e52e5.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BFCL-Dataset BFCL Dataset-Related Issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants