-
Notifications
You must be signed in to change notification settings - Fork 35
Open
Description
Currently in benchmarks, we don't support sub-agent delegation.
benchmarks/benchmarks/swtbench/run_infer.py
Lines 260 to 265 in 680ce0f
| # TODO: we can enable condenser and security analyzer later | |
| # and have them configurable via EvalMetadata | |
| # condenser=get_default_condenser( | |
| # llm=self.metadata.llm.model_copy(update={"service_id": "condenser"}) | |
| # ), | |
| # security_analyzer=LLMSecurityAnalyzer(), |
We'd like to support this and try, e.g. SWE-Bench with agent delegation on.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels