Tool choice required

[matheper]

This pull request refactors how runtime generation parameters (such as temperature, max_tokens, and tool_choice) are passed to and handled by LLM (Large Language Model) instances. The main improvement is to make the LLM instantiation more flexible by supporting arbitrary generation arguments via a unified mechanism, and to clarify the default tool usage behavior in HuggingFace and OpenAI LLMs. Additional tests are included to ensure the new instantiation logic works as expected.

LLM Instantiation and Generation Argument Handling:

• Refactored LLM.instantiate in debug_gym/llms/base.py to accept arbitrary runtime generation keyword arguments via **runtime_generate_kwargs, removing the explicit temperature and max_tokens parameters and allowing more flexible configuration. [1] [2]
• Updated scripts/replay.py and scripts/run.py to pass generate_kwargs from the config into LLM.instantiate using the new runtime_generate_kwargs mechanism. [1] [2]
• Added/updated tests in tests/llms/test_base.py to cover the new instantiation logic, including passing both config and runtime generation kwargs.

LLM Generation Behavior:

• Changed the default tool_choice parameter in OpenAILLM.generate to "auto" and ensured it is passed through to the underlying API call, making tool usage more configurable. [1] [2]
• Overrode the generate method in HuggingFaceLLM to default tool_choice to "required" for tool usage, clarifying its intended behavior.

Minor Cleanups:

• Removed an unused import from debug_gym/agents/utils.py and added a missing import in debug_gym/llms/huggingface.py. [1] [2]

[Copilot]

Pull request overview

This PR refactors the LLM instantiation and generation parameter handling to provide more flexibility in configuring runtime generation parameters (such as temperature, max_tokens, and tool_choice). The changes make it easier to pass arbitrary generation kwargs through the system while clarifying the default tool usage behavior for different LLM implementations.

Changes:

• Refactored LLM.instantiate to accept arbitrary runtime generation parameters via **runtime_generate_kwargs instead of explicit parameters
• Made tool_choice configurable in OpenAI and HuggingFace LLM implementations with appropriate defaults
• Removed unused import and added missing import for better code hygiene

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
debug_gym/llms/base.py	Refactored instantiate method to accept **runtime_generate_kwargs instead of explicit temperature and max_tokens parameters
debug_gym/llms/openai.py	Added tool_choice parameter to generate method with default value "auto"
debug_gym/llms/huggingface.py	Added import for LLMResponse and overrode generate method to default tool_choice to "required"
scripts/run.py	Updated to pass LLM config using new kwargs-based approach
scripts/replay.py	Updated to pass LLM config using new kwargs-based approach
debug_gym/agents/utils.py	Removed unused AGENT_REGISTRY import
tests/llms/test_base.py	Updated tests to verify new instantiation behavior with tool_choice parameter

Comments suppressed due to low confidence (1)

debug_gym/llms/base.py:293

• Overridden method signature does not match call, where it is passed too many arguments. Overriding method method OpenAILLM.generate matches the call.
  Overridden method signature does not match call, where it is passed an argument named 'tool_choice'. Overriding method method OpenAILLM.generate matches the call.

    def generate(self, messages, tools, **kwargs) -> LLMResponse:

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.