Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: properly structure completions in Ollama provider #589

Merged
merged 11 commits into from
Dec 17, 2024
Merged
28 changes: 19 additions & 9 deletions agentops/llms/providers/ollama.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,30 @@

def handle_response(self, response, kwargs, init_timestamp, session: Optional[Session] = None) -> dict:
llm_event = LLMEvent(init_timestamp=init_timestamp, params=kwargs)
if session is not None:
llm_event.session_id = session.session_id

Check warning on line 22 in agentops/llms/providers/ollama.py

View check run for this annotation

Codecov / codecov/patch

agentops/llms/providers/ollama.py#L22

Added line #L22 was not covered by tests

def handle_stream_chunk(chunk: dict):
message = chunk.get("message", {"role": None, "content": ""})

if chunk.get("done"):
llm_event.completion["content"] += message.get("content")
llm_event.end_timestamp = get_ISO_time()
llm_event.model = f'ollama/{chunk.get("model")}'
llm_event.returns = chunk
llm_event.returns["message"] = llm_event.completion
llm_event.prompt = kwargs["messages"]
llm_event.agent_id = check_call_stack_for_agent_id()
self.client.record(llm_event)
self._safe_record(session, llm_event)

Check warning on line 34 in agentops/llms/providers/ollama.py

View check run for this annotation

Codecov / codecov/patch

agentops/llms/providers/ollama.py#L34

Added line #L34 was not covered by tests

if llm_event.completion is None:
llm_event.completion = message
llm_event.completion = {

Check warning on line 37 in agentops/llms/providers/ollama.py

View check run for this annotation

Codecov / codecov/patch

agentops/llms/providers/ollama.py#L37

Added line #L37 was not covered by tests
"role": message.get("role"),
"content": message.get("content", ""),
"tool_calls": None,
"function_call": None,
}
else:
llm_event.completion["content"] += message.get("content")
llm_event.completion["content"] += message.get("content", "")

Check warning on line 44 in agentops/llms/providers/ollama.py

View check run for this annotation

Codecov / codecov/patch

agentops/llms/providers/ollama.py#L44

Added line #L44 was not covered by tests

if inspect.isgenerator(response):

Expand All @@ -47,13 +53,16 @@
return generator()

llm_event.end_timestamp = get_ISO_time()

llm_event.model = f'ollama/{response["model"]}'
llm_event.returns = response
llm_event.agent_id = check_call_stack_for_agent_id()
llm_event.prompt = kwargs["messages"]
llm_event.completion = response["message"]

llm_event.completion = {

Check warning on line 60 in agentops/llms/providers/ollama.py

View check run for this annotation

Codecov / codecov/patch

agentops/llms/providers/ollama.py#L60

Added line #L60 was not covered by tests
"role": response["message"].get("role"),
"content": response["message"].get("content", ""),
"tool_calls": None,
"function_call": None,
}
self._safe_record(session, llm_event)
return response

areibman marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -96,21 +105,22 @@
# Call the original function with its original arguments
init_timestamp = get_ISO_time()
result = original_func["ollama.Client.chat"](*args, **kwargs)
return self.handle_response(result, kwargs, init_timestamp)
return self.handle_response(result, kwargs, init_timestamp, session=kwargs.get("session", None))

Check warning on line 108 in agentops/llms/providers/ollama.py

View check run for this annotation

Codecov / codecov/patch

agentops/llms/providers/ollama.py#L108

Added line #L108 was not covered by tests

# Override the original method with the patched one
Client.chat = patched_function

def _override_chat_async_client(self):
from ollama import AsyncClient

original_func = {}

Check warning on line 116 in agentops/llms/providers/ollama.py

View check run for this annotation

Codecov / codecov/patch

agentops/llms/providers/ollama.py#L116

Added line #L116 was not covered by tests
original_func["ollama.AsyncClient.chat"] = AsyncClient.chat

async def patched_function(*args, **kwargs):
# Call the original function with its original arguments
init_timestamp = get_ISO_time()
result = await original_func["ollama.AsyncClient.chat"](*args, **kwargs)
return self.handle_response(result, kwargs, init_timestamp)
return self.handle_response(result, kwargs, init_timestamp, session=kwargs.get("session", None))

Check warning on line 123 in agentops/llms/providers/ollama.py

View check run for this annotation

Codecov / codecov/patch

agentops/llms/providers/ollama.py#L123

Added line #L123 was not covered by tests

# Override the original method with the patched one
AsyncClient.chat = patched_function
areibman marked this conversation as resolved.
Show resolved Hide resolved
Loading