Set default temperature to None instead of 0.0 by neubig · Pull Request #1989 · OpenHands/software-agent-sdk

neubig · 2026-02-10T17:29:57Z

Summary

Change get_default_temperature to return None by default instead of 0.0. This allows models to use their provider's default temperature setting rather than forcing temperature=0.0 for all models.

Motivation

As discussed in #1913, always setting temperature=0.0 causes issues with some models (like claude-4.6) that don't accept this value. By returning None, we let the provider use their own default temperature, which:

Avoids errors from providers that don't accept certain temperature values
Ensures new models work out of the box without needing explicit configuration
Simplifies the codebase by removing unnecessary default values

Changes

get_default_temperature() now returns float | None instead of float
Default return value changed from 0.0 to None
Models with specific temperature requirements (kimi-k2-thinking, kimi-k2.5) still explicitly return 1.0
Updated field description in LLM class to reflect the new behavior
Updated tests to expect None instead of 0.0

Testing

All 583 LLM tests pass. Pre-commit hooks pass on all modified files.

Fixes #1913

@neubig can click here to continue refining the PR

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	`eclipse-temurin:17-jdk`	Link
python	amd64, arm64	`nikolaik/python-nodejs:python3.12-nodejs22`	Link
golang	amd64, arm64	`golang:1.21-bookworm`	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:887118f-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-887118f-python \
  ghcr.io/openhands/agent-server:887118f-python

All tags pushed for this build

ghcr.io/openhands/agent-server:887118f-golang-amd64
ghcr.io/openhands/agent-server:887118f-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:887118f-golang-arm64
ghcr.io/openhands/agent-server:887118f-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:887118f-java-amd64
ghcr.io/openhands/agent-server:887118f-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:887118f-java-arm64
ghcr.io/openhands/agent-server:887118f-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:887118f-python-amd64
ghcr.io/openhands/agent-server:887118f-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:887118f-python-arm64
ghcr.io/openhands/agent-server:887118f-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:887118f-golang
ghcr.io/openhands/agent-server:887118f-java
ghcr.io/openhands/agent-server:887118f-python

About Multi-Architecture Support

Each variant tag (e.g., 887118f-python) is a multi-arch manifest supporting both amd64 and arm64
Docker automatically pulls the correct architecture for your platform
Individual architecture tags (e.g., 887118f-python-amd64) are also available if needed

Change get_default_temperature to return None by default instead of 0.0. This allows models to use their provider's default temperature setting rather than forcing temperature=0.0 for all models. Models with specific temperature requirements (kimi-k2-thinking, kimi-k2.5) still explicitly return 1.0. Fixes #1913 Co-authored-by: openhands <openhands@all-hands.dev>

neubig · 2026-02-11T00:45:47Z

@OpenHands check the failing CI, merge main, reproduce locally, and fix any issues.

openhands-ai · 2026-02-11T00:45:55Z

I'm on it! neubig can track my progress at all-hands.dev

…re-to-none

Update test to expect temperature=None instead of 0.0, aligning with the PR's change to get_default_temperature() returning None by default. Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2026-02-11T00:51:37Z

Summary

I've successfully addressed the failing CI for PR #1989 by:

Completed Tasks:

✅ Checked the failing CI - Identified that the sdk-tests job was failing with test_llm_config_defaults expecting temperature == 0.0 but getting None
✅ Merged main - Merged origin/main into the fix-default-temperature-to-none branch to incorporate the latest changes
✅ Reproduced locally - Ran the failing test locally and confirmed the issue
✅ Fixed the issue - Updated tests/sdk/config/test_llm_config.py to expect temperature is None instead of temperature == 0.0, aligning with the PR's intended change to get_default_temperature() returning None by default

Change Made:

-    assert config.temperature == 0.0
+    assert config.temperature is None  # None to use provider defaults

Verification:

All 15 tests in test_llm_config.py pass
All 130 tests in test_model_features.py pass

Pushed:

The fix has been pushed to the fix-default-temperature-to-none branch. The CI should now pass.

View full conversation

github-actions · 2026-02-11T00:53:32Z

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk/llm
llm.py	460	86	81%	402, 423–424, 460, 637, 738, 740–741, 769, 819, 830–832, 836–840, 848–850, 860–862, 865–866, 870, 872–873, 875, 898–903, 1026, 1031–1032, 1229–1230, 1239, 1252, 1254–1259, 1261–1278, 1281–1285, 1287–1288, 1294–1303, 1354, 1356
openhands-sdk/openhands/sdk/llm/utils
model_features.py	42	1	97%	32
TOTAL	17838	5369	69%

all-hands-bot

🟡 Taste Rating: Acceptable

Pragmatic fix to a real problem, but breaks existing behavior without proper user migration guidance.

The core change is simple and solves the claude-4.6 incompatibility issue. However, this is a significant behavior change that moves from deterministic outputs (temp=0.0) to provider-dependent defaults. See inline comments for specific concerns.

VERDICT: ✅ Worth merging after addressing migration documentation

KEY INSIGHT: This is the right technical fix, but users who relied on deterministic behavior need explicit guidance on setting temperature=0.0 in their configs.

openhands-sdk/openhands/sdk/llm/utils/model_features.py

tests/sdk/llm/test_model_features.py

openhands-sdk/openhands/sdk/llm/llm.py

openhands-sdk/openhands/sdk/llm/utils/model_features.py

enyst

Thank you for this! I think maybe we can then remove the DEFAULT_TEMPERATURE_MODELS too? That’s there because Kimi is a reasoning model that doesn’t accept effort, if I recall correctly, but it needs temperature 1 like reasoning models.

I had the agent test Kimi:

for 2.5, it worked fine with None
for 2-thinking, no deployments found. (maybe it’s legacy now; we may want to remove it from eval proxy, since it doesn’t appear to work anymore)

Please see PR:

#1994

Addresses review comment asking for more explicit documentation about: - What None means (provider default) - How to get deterministic behavior (explicitly set 0.0) - The typical temperature range Co-authored-by: openhands <openhands@all-hands.dev>

As per enyst's suggestion on PR #1989: - Remove DEFAULT_TEMPERATURE_MODELS entirely - get_default_temperature() now returns None for all models - This simplifies the codebase and lets all models use their provider's default temperature - Previously Kimi 2.5 was tested to work fine with None Co-authored-by: openhands <openhands@all-hands.dev>

enyst · 2026-02-11T23:55:55Z

FWIW, PR 1994 is a PR to this PR. 😅

all-hands-bot

🟡 Taste Rating: Acceptable - Solves a real problem with elegant simplification, but introduces a breaking change that needs careful consideration.

Verdict: Worth merging with minor improvements to address backward compatibility concerns and design consistency.

Key Insight: This trades deterministic defaults for provider flexibility—a pragmatic choice, but one that changes behavior for all existing users.

openhands-sdk/openhands/sdk/llm/utils/model_features.py

openhands-sdk/openhands/sdk/llm/llm.py

all-hands-bot

🔴 Taste Rating: Needs Rework

This PR simplifies the code (good!) but introduces a breaking change that violates the "never break userspace" principle. Users relying on deterministic outputs (temperature=0.0) will now get unpredictable provider defaults.

[CRITICAL ISSUES] - Must Fix

1. Breaking Change Without Documentation

File: openhands-sdk/openhands/sdk/llm/llm.py (lines 180-185)

🔴 Critical: This description doesn't mention that this is a BREAKING CHANGE. Users who relied on temperature=0.0 for deterministic outputs will now get provider-specific defaults (which vary: OpenAI defaults to 1.0, Anthropic to 1.0, etc.).

You've broken userspace without documenting it. At minimum:

Clear migration guide: "To preserve deterministic behavior, explicitly set temperature=0.0"
Deprecation warning in release notes
Consider a compatibility flag for one release cycle

Suggested fix:

temperature: float | None = Field(
    default=None,
    ge=0,
    description=(
        "Sampling temperature for response generation. "
        "Defaults to None (uses provider default, typically 1.0 for most models). "
        "BREAKING CHANGE: Previously defaulted to 0.0 for deterministic outputs. "
        "To restore deterministic behavior, explicitly set temperature=0.0. "
        "Higher values (0.7-1.0) increase creativity and randomness."
    ),
)

2. Unused Parameter - Code Smell

File: openhands-sdk/openhands/sdk/llm/utils/model_features.py (line 183)

🔴 Critical: The _model parameter is now completely ignored (indicated by the underscore prefix). This is a code smell that screams "delete this function entirely."

If the model parameter is never used, why does this function exist? Either:

Make it a module-level constant: DEFAULT_TEMPERATURE = None
Delete it and use None directly where needed
Or explain why we need the indirection

The function signature promises model-specific logic but delivers nothing. That's broken by design.

[IMPROVEMENT OPPORTUNITIES] - Should Fix

3. Misleading Documentation

File: openhands-sdk/openhands/sdk/llm/utils/model_features.py (lines 184-189)

🟠 Important: The docstring says "Return the default temperature for a given model pattern" but the model is completely ignored. This is a lie.

Suggested fix:

def get_default_temperature(_model: str) -> float | None:
    """Return None to allow providers to use their default temperature.
    
    This function exists for backward compatibility and may be removed in future versions.
    To use provider defaults, pass temperature=None when creating LLM instances.
    """
    return None

4. Missing Integration Tests

File: tests/sdk/llm/test_model_features.py (lines 342-346)

🟠 Important: These tests only verify the function returns None. They don't test what matters:

Does passing temperature=None to a provider actually work?
Does it fix the claude-4.6 issue mentioned in #1913?
What temperature does each provider actually use when None is passed?

You're testing the implementation, not the behavior. Add tests that:

Verify LiteLLM/providers accept temperature=None
Confirm claude-4.6 no longer errors
Document what "provider default" means for major providers

VERDICT

❌ Needs rework: The simplification is good in principle, but the breaking change and unused parameter issues must be addressed first.

KEY INSIGHT

Simplifying code by removing special cases is "good taste" - BUT you cannot break existing user expectations in the process. Either preserve backward compatibility with explicit opt-in to new behavior, or clearly document the breaking change and provide a migration path.

As per review comment on PR #1989: - Remove the unused _model parameter since DEFAULT_TEMPERATURE_MODELS was removed - Function now returns None without needing model info - Updated tests to use new function signature Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2026-02-13T01:58:09Z

Looks like there are a few issues preventing this PR from being merged!

GitHub Actions are failing:
- Review Thread Gate

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1989 at branch `fix-default-temperature-to-none`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

openhands-agent added 2 commits February 11, 2026 00:48

Merge remote-tracking branch 'origin/main' into fix-default-temperatu…

b1f915a

…re-to-none

Fix test_llm_config_defaults to expect None temperature

ad69f00

Update test to expect temperature=None instead of 0.0, aligning with the PR's change to get_default_temperature() returning None by default. Co-authored-by: openhands <openhands@all-hands.dev>

neubig marked this pull request as ready for review February 11, 2026 01:19

all-hands-bot reviewed Feb 11, 2026

View reviewed changes

enyst reviewed Feb 11, 2026

View reviewed changes

openhands-sdk/openhands/sdk/llm/utils/model_features.py Outdated Show resolved Hide resolved

enyst mentioned this pull request Feb 11, 2026

Default temperature to None in the SDK OpenHands/benchmarks#408

Open

enyst reviewed Feb 11, 2026

View reviewed changes

openhands-agent added 2 commits February 11, 2026 15:36

neubig added the needs-review label Feb 12, 2026 — with OpenHands Staging

neubig added the review-this This label triggers a PR review by OpenHands label Feb 12, 2026 — with OpenHands AI

all-hands-bot reviewed Feb 12, 2026

View reviewed changes

openhands-sdk/openhands/sdk/llm/utils/model_features.py Outdated Show resolved Hide resolved

openhands-sdk/openhands/sdk/llm/utils/model_features.py Outdated Show resolved Hide resolved

openhands-sdk/openhands/sdk/llm/llm.py Show resolved Hide resolved

all-hands-bot reviewed Feb 12, 2026

View reviewed changes

enyst added behavior-initiative This is related to the system prompt sections and LLM steering. and removed behavior-initiative This is related to the system prompt sections and LLM steering. labels Feb 14, 2026

Conversation

neubig commented Feb 10, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Changes

Testing

Uh oh!

neubig commented Feb 11, 2026

Uh oh!

openhands-ai bot commented Feb 11, 2026

Uh oh!

openhands-ai bot commented Feb 11, 2026

Summary

Completed Tasks:

Change Made:

Verification:

Pushed:

Uh oh!

github-actions bot commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

🟡 Taste Rating: Acceptable

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

enyst left a comment

Choose a reason for hiding this comment

Uh oh!

enyst commented Feb 11, 2026

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

🔴 Taste Rating: Needs Rework

[CRITICAL ISSUES] - Must Fix

1. Breaking Change Without Documentation

2. Unused Parameter - Code Smell

[IMPROVEMENT OPPORTUNITIES] - Should Fix

3. Misleading Documentation

4. Missing Integration Tests

VERDICT

KEY INSIGHT

Uh oh!

openhands-ai bot commented Feb 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

neubig commented Feb 10, 2026 •

edited by github-actions bot

Loading

github-actions bot commented Feb 11, 2026 •

edited

Loading