Skip to content
This repository has been archived by the owner on Oct 2, 2024. It is now read-only.

Investigate scheduled test failure #539

Closed
rogthefrog opened this issue Aug 30, 2024 · 0 comments · Fixed by #546
Closed

Investigate scheduled test failure #539

rogthefrog opened this issue Aug 30, 2024 · 0 comments · Fixed by #546
Assignees

Comments

@rogthefrog
Copy link
Contributor

https://github.com/mlcommons/modelgauge/actions/runs/10637406145/job/29491210629

=========================== short test summary info ============================
FAILED plugins/validation_tests/test_object_creation.py::test_all_suts_can_evaluate[gpt2] - Failed: Timeout >45.0s
FAILED plugins/validation_tests/test_object_creation.py::test_all_suts_can_evaluate[llama-2-7b] - modelgauge.general.APIException: Unexpected API failure (400): {"error":{"message":"Unable to access non-serverless model meta-llama/Llama-2-7b-hf. Please visit https://api.together.ai/models/meta-llama/Llama-2-7b-hf to create and start a new dedicated endpoint for the model.","type":"invalid_request_error","param":null,"code":"model_not_available"}}
FAILED plugins/validation_tests/test_object_creation.py::test_all_suts_can_evaluate[llama-2-7b-chat] - modelgauge.general.APIException: Unexpected API failure (400): {"error":{"message":"Unable to access non-serverless model meta-llama/Llama-2-7b-chat-hf. Please visit https://api.together.ai/models/meta-llama/Llama-2-7b-chat-hf to create and start a new dedicated endpoint for the model.","type":"invalid_request_error","param":null,"code":"model_not_available"}}

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant