LLM Generation Config(generate_content_config) does not work.

[simonwei97]

** Please make sure you read the contribution guide and file the issues in the right place. **
Contribution guide.

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

Note

LLM server way: vLLM
Model Info: Qwen2.5-14B-Instruct

❌ Case1: Custom temperature=0.2 with generate_content_config

import datetime
from zoneinfo import ZoneInfo

from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm
from google.genai import types


def get_weather(city: str) -> dict:
    """Retrieves the current weather report for a specified city.

    Args:
        city (str): The name of the city for which to retrieve the weather report.

    Returns:
        dict: status and result or error msg.
    """
    if city.lower() == "new york":
        return {
            "status": "success",
            "report": (
                "The weather in New York is sunny with a temperature of 25 degrees"
                " Celsius (77 degrees Fahrenheit)."
            ),
        }
    else:
        return {
            "status": "error",
            "error_message": f"Weather information for '{city}' is not available.",
        }


def get_current_time(city: str) -> dict:
    """Returns the current time in a specified city.

    Args:
        city (str): The name of the city for which to retrieve the current time.

    Returns:
        dict: status and result or error msg.
    """

    if city.lower() == "new york":
        tz_identifier = "America/New_York"
    else:
        return {
            "status": "error",
            "error_message": (
                f"Sorry, I don't have timezone information for {city}."
            ),
        }

    tz = ZoneInfo(tz_identifier)
    now = datetime.datetime.now(tz)
    report = (
        f'The current time in {city} is {now.strftime("%Y-%m-%d %H:%M:%S %Z%z")}'
    )
    return {"status": "success", "report": report}


root_agent = Agent(
    model=LiteLlm(
        model="openai/Qwen2.5-14B-Instruct",
        api_base = "http://127.0.0.1:8080/v1",
        api_key="EMPTY",
    ),
    name="weather_time_agent",
    description=(
        "Agent to answer questions about the time and weather in a city."
    ),
    instruction=(
        "You are a helpful agent who can answer user questions about the time and weather in a city."
    ),
    tools=[get_weather, get_current_time],
+   generate_content_config=types.GenerateContentConfig(
+      temperature=0.2, # More deterministic output
+      max_output_tokens=250
    )
)

adk web

• Input: Hello
• Output: Hello! How can I assist you today? Do you want to know the time or weather in a specific city?

Important

LLM server use default temperature=0.7, while generate_content_config does not work (both temperature and max_output_tokens.

---

✅ Case2: Custom temperature=0.2 with generate_content_config and LiteLLM args temperature=0.1

import datetime
from zoneinfo import ZoneInfo

from google.adk.agents import Agent
from google.adk.models.lite_llm import LiteLlm
from google.genai import types


def get_weather(city: str) -> dict:
    """Retrieves the current weather report for a specified city.

    Args:
        city (str): The name of the city for which to retrieve the weather report.

    Returns:
        dict: status and result or error msg.
    """
    if city.lower() == "new york":
        return {
            "status": "success",
            "report": (
                "The weather in New York is sunny with a temperature of 25 degrees"
                " Celsius (77 degrees Fahrenheit)."
            ),
        }
    else:
        return {
            "status": "error",
            "error_message": f"Weather information for '{city}' is not available.",
        }


def get_current_time(city: str) -> dict:
    """Returns the current time in a specified city.

    Args:
        city (str): The name of the city for which to retrieve the current time.

    Returns:
        dict: status and result or error msg.
    """

    if city.lower() == "new york":
        tz_identifier = "America/New_York"
    else:
        return {
            "status": "error",
            "error_message": (
                f"Sorry, I don't have timezone information for {city}."
            ),
        }

    tz = ZoneInfo(tz_identifier)
    now = datetime.datetime.now(tz)
    report = (
        f'The current time in {city} is {now.strftime("%Y-%m-%d %H:%M:%S %Z%z")}'
    )
    return {"status": "success", "report": report}


root_agent = Agent(
    model=LiteLlm(
        model="openai/Qwen2.5-14B-Instruct",
        api_base = "http://127.0.0.1:8080/v1",
        api_key="EMPTY",
+       temperature=0.1,
    ),
    name="weather_time_agent",
    description=(
        "Agent to answer questions about the time and weather in a city."
    ),
    instruction=(
        "You are a helpful agent who can answer user questions about the time and weather in a city."
    ),
    tools=[get_weather, get_current_time],
+   generate_content_config=types.GenerateContentConfig(
+      temperature=0.2, # More deterministic output
+      max_output_tokens=250
    )
)

adk web

• Input: Hello
• Output: Hello! How can I assist you today? Do you want to know the time or weather in a specific city?

Important

LLM server use temperature=0.1, which is same as LiteLLM args. On the other hand, generate_content_config also does not work (both temperature and max_output_tokens.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: macOS
• Python version(python -V): python 3.12
• ADK version(pip show google-adk): 1.2.1

Additional context
Add any other context about the problem here.

[simonwei97]

Root code maybe src/google/adk/models/lite_llm.py, func generate_content_async .

Parameter completion_args not contains generate config like top_k, temperature, etc, while func argsllm_request contains generate_content_config which parameter we expected.

adk-python/src/google/adk/models/lite_llm.py

Lines 647 to 771 in 068df04

	async def generate_content_async(
	self, llm_request: LlmRequest, stream: bool = False
	) -> AsyncGenerator[LlmResponse, None]:
	"""Generates content asynchronously.
	Args:
	llm_request: LlmRequest, the request to send to the LiteLlm model.
	stream: bool = False, whether to do streaming call.
	Yields:
	LlmResponse: The model response.
	"""
	self._maybe_append_user_content(llm_request)
	logger.debug(_build_request_log(llm_request))
	messages, tools, response_format = _get_completion_inputs(llm_request)
	completion_args = {
	"model": self.model,
	"messages": messages,
	"tools": tools,
	"response_format": response_format,
	}
	completion_args.update(self._additional_args)
	if stream:
	text = ""
	# Track function calls by index
	function_calls = {} # index -> {name, args, id}
	completion_args["stream"] = True
	aggregated_llm_response = None
	aggregated_llm_response_with_tool_call = None
	usage_metadata = None
	fallback_index = 0
	for part in self.llm_client.completion(**completion_args):
	for chunk, finish_reason in _model_response_to_chunk(part):
	if isinstance(chunk, FunctionChunk):
	index = chunk.index or fallback_index
	if index not in function_calls:
	function_calls[index] = {"name": "", "args": "", "id": None}
	if chunk.name:
	function_calls[index]["name"] += chunk.name
	if chunk.args:
	function_calls[index]["args"] += chunk.args
	# check if args is completed (workaround for improper chunk
	# indexing)
	try:
	json.loads(function_calls[index]["args"])
	fallback_index += 1
	except json.JSONDecodeError:
	pass
	function_calls[index]["id"] = (
	chunk.id or function_calls[index]["id"] or str(index)
	)
	elif isinstance(chunk, TextChunk):
	text += chunk.text
	yield _message_to_generate_content_response(
	ChatCompletionAssistantMessage(
	role="assistant",
	content=chunk.text,
	),
	is_partial=True,
	)
	elif isinstance(chunk, UsageMetadataChunk):
	usage_metadata = types.GenerateContentResponseUsageMetadata(
	prompt_token_count=chunk.prompt_tokens,
	candidates_token_count=chunk.completion_tokens,
	total_token_count=chunk.total_tokens,
	)
	if (
	finish_reason == "tool_calls" or finish_reason == "stop"
	) and function_calls:
	tool_calls = []
	for index, func_data in function_calls.items():
	if func_data["id"]:
	tool_calls.append(
	ChatCompletionMessageToolCall(
	type="function",
	id=func_data["id"],
	function=Function(
	name=func_data["name"],
	arguments=func_data["args"],
	index=index,
	),
	)
	)
	aggregated_llm_response_with_tool_call = (
	_message_to_generate_content_response(
	ChatCompletionAssistantMessage(
	role="assistant",
	content="",
	tool_calls=tool_calls,
	)
	)
	)
	function_calls.clear()
	elif finish_reason == "stop" and text:
	aggregated_llm_response = _message_to_generate_content_response(
	ChatCompletionAssistantMessage(role="assistant", content=text)
	)
	text = ""
	# waiting until streaming ends to yield the llm_response as litellm tends
	# to send chunk that contains usage_metadata after the chunk with
	# finish_reason set to tool_calls or stop.
	if aggregated_llm_response:
	if usage_metadata:
	aggregated_llm_response.usage_metadata = usage_metadata
	usage_metadata = None
	yield aggregated_llm_response
	if aggregated_llm_response_with_tool_call:
	if usage_metadata:
	aggregated_llm_response_with_tool_call.usage_metadata = usage_metadata
	yield aggregated_llm_response_with_tool_call
	else:
	response = await self.llm_client.acompletion(**completion_args)
	yield _model_response_to_generate_content_response(response)

[simonwei97]

I will fix this issue as possible as I can.

[simonwei97]

Sorry about this issue. I find that my code was reverted by this commit. 1cfc555

[simonwei97]

Should I recommit another PR? @hangfei @genquan9

[genquan9]

Could you help send another PR? thanks!

[genquan9]

i approved #1450 and am merging this change now. thanks!

[genquan9]

The merge finished. thanks again for all your contributions!