-
Notifications
You must be signed in to change notification settings - Fork 34
Open
Description
Currently context condensation is off in benchmarks:
benchmarks/benchmarks/swtbench/run_infer.py
Lines 260 to 265 in 680ce0f
| # TODO: we can enable condenser and security analyzer later | |
| # and have them configurable via EvalMetadata | |
| # condenser=get_default_condenser( | |
| # llm=self.metadata.llm.model_copy(update={"service_id": "condenser"}) | |
| # ), | |
| # security_analyzer=LLMSecurityAnalyzer(), |
We should enable it and make it configurable.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels