Unhelpful responses for Tool Calling functionality with Ollama

[aswinkumar1999]

I have been using Goose with Gemini 2.0 Flash/Lite and had a great experience. I tried to use it with Ollama, but it looks like there's a fundamental problem with tool calling in Ollama. I tried various models and got the same kind of unhelpful responses.

Ollama version: 0.6.2 ( on ARM - Running on GH200 from Cloud provider with SSH Tunneling of Port 11434 )
Goose version: 1.0.15

List of models that I tried with (All of them have Tool calling support as per Ollama Tag):

NAME                                     ID              SIZE      MODIFIED          
michaelneale/deepseek-r1-goose:70b       90cc3dd53da8    42 GB     14 minutes ago       
michaelneale/deepseek-r1-goose:latest    425664f4d998    9.0 GB    43 minutes ago       
firefunction-v2:latest                   b1ed6b22fb67    39 GB     59 minutes ago       
llama3-groq-tool-use:70b                 696d50e6fc55    39 GB     About an hour ago    
hermes3:70b                              60ef54cd913f    39 GB     About an hour ago    
mixtral:8x22b                            e8479ee1cb51    79 GB     About an hour ago    
qwen2:72b                                93563ef658b2    41 GB     About an hour ago    
mistral-large:123b                       bbcf36dc47ad    73 GB     About an hour ago    
qwen2.5:72b                              424bad2cc13f    47 GB     About an hour ago    
qwen2.5-coder:32b                        4bd6cbf2d094    19 GB     About an hour ago    
qwq:latest                               009cb3f08d74    19 GB     About an hour ago    
nemotron:latest                          2262f047a28a    42 GB     About an hour ago    
llama3.3:latest                          a6eb4748fd29    42 GB     About an hour ago   

Here's an example from one of my experiments ( Replaced $USER & $HOSTNAME for privacy ) :

$USER@$HOSTNAME:~/goose$ goose session
starting session | provider: google model: gemini-2.0-flash-lite
    logging to /home/$USER/.local/share/goose/sessions/20250323_014149.jsonl
    working directory: /home/$USER/goose

Goose is running! Enter your instructions, or try asking what goose can do.

( O)> List files in current dir

─── shell | developer ──────────────────────────
command: ls -l


total 8
-rw-rw-r-- 1 $USER $USER 5650 Mar 22 23:28 careers.txt

OK. I see one file: `careers.txt`.

( O)> 
$USER@$HOSTNAME:~/goose$ goose configure

This will update your existing config file
  if you prefer, you can edit it directly at /home/$USER/.config/goose/config.yaml

┌   goose-configure 
│
◇  What would you like to configure?
│  Configure Providers 
│
◇  Which model provider should we use?
│  Ollama 
│
●  OLLAMA_HOST is already configured
│  
◇  Would you like to update this value?
│  No 
│
◇  Enter a model from that provider:
│  michaelneale/deepseek-r1-goose:70b
│
◐  Checking your configuration...                                                                                                   └  Configuration saved successfully
$USER@$HOSTNAME:~/goose$ goose session
starting session | provider: ollama model: michaelneale/deepseek-r1-goose:70b
    logging to /home/$USER/.local/share/goose/sessions/20250323_014213.jsonl
    working directory: /home/$USER/goose

Goose is running! Enter your instructions, or try asking what goose can do.

( O)> List files in current dir
Alright, so I need to figure out how to list the files in the current directory. Let me think about what tools are available for this task.

I remember seeing a tool called 'platform__read_resource' and another named 'developer__text_editor'. Hmm, but do these tools actually list files? Or are they used for reading specific resources or editing text?

Wait, I don't see any tool explicitly mentioned that lists directory contents. The 'developer__list_windows' tool is there, but it's for windows titles, not files.

Since none of the available tools seem to handle listing files in a directory, maybe the best approach is to explain how to do this without using a tool or suggest using a terminal command if possible.
</think>

To list the files in the current directory, you can use a terminal command like `ls` (on Unix-based systems) or `dir` (on Windows).
( O)> Can you list all the tools available to you 
Okay, so the user just asked me to list all the tools I have available. Hmm, I remember that earlier they provided a JSON array of some functions. Let me go through that again and try to parse it.

Looking at the JSON, each tool is an object with "name" and "description". The first one is computer_vision, but wait, no—I think the user actually listed tools like platform_read_resource. So I need to go through each of those objects and list them as available tools.

I should make sure to format this in a clear way for the user. Maybe a numbered list would be best, starting from 1 to how many there are. Each entry will have the tool name and a brief description so the user understands what they do.

Wait, let me count how many there are. Starting from computer_vision up until platform_list_resources, looks like about 12 tools. So I can list each with their respective functions in simple terms without any markdown or special formatting.
</think>

Here is a list of tools available to assist you:

1. **platform__read_resource**: Reads resources from an extension to provide context.
2. **platform__list_resources**: Lists resources from specified extensions.
3. **developer__image_processor**: Processes images for conversation use by resizing and converting to PNG.
4. **developer__screen_capture**: Captures screenshots of displays or specific windows.
5. **developer__list_windows**: Lists window titles for screen capture usage.
6. **developer__text_editor**: Edits text files with options to view, write, replace strings, and undo changes.
7. **developer__undo_edit**: Undoes the last edit made in a file.
8. **computer_vision**: Analyzes images using optical character recognition (OCR) and scene understanding.

Each tool is designed for specific tasks, providing functionality from resource management to image processing and text editing. Let me know how I can assist you with these tools!

Thanks!

[zakiali]

Hi @aswinkumar1999, yea, this is something we've seen as well. Tool calling with small models through ollama is just not very good. @alicehau can provide some more context here, but we've got some experimental features with using small models with goose using ollama and a tool-calling shim, which may work better than trying to use the models directly. #1403 is a discussion thread around this and the PR for this feature.

One other thing you can try is to set the OLLAMA_CONTEXT_LENGTH env var. The default we learned is small, and they recently allowed this parameter to be set.

[ahau-square]

@aswinkumar1999 the deepseek-r1 model with tool calling published there is known to not be super reliable. Here's the documentation on the experimental toolshim feature: https://block.github.io/goose/docs/guides/experimental-features

As Zaki mentioned, definitely increase your OLLAMA_CONTEXT_LENGTH when you start your ollama server too.

We also recently released this blogpost about performance with a lot of the open models that might be of interest: https://block.github.io/goose/blog/2025/03/31/goose-benchmark

Deepseek-r1 has tool calling available now via other providers, like openrouter or llama.cpp. So you might consider trying another provider that supports tool calling for that model

[aswinkumar1999]

Thank you @ahau-square, for the benchmarks. This is great work! The suggestions were super helpful, and I got it working.

Here's the steps I followed, adding for anyone else facing the same issue :

1. Updated Ollama & Goose to the latest version.
2. Increased Context Size to 25K ( Refer Ollama FAQ on how to temporary/permanently set this)
3. I used the LLM-Model-VRAM-Calculator and the Goose Benchmark to find balance between VRAM, Model Performance and Context size.

[chrstnwhlrt]

@aswinkumar1999 Thanks for your solution, how much vram do you have available? I'm using a 4080 with 16GB and I just can't get goose to list tools or resources of a custom mcp server (echo example), though the mcp inspector shows everything is working just fine. I created an Ollama file increasing the context window to 30k for some models, but no success either. What model do you recommend?

[chrstnwhlrt]

I tried the exact same setup using oterm and it just works. Somehow goose failed to list tools from my mcp server, oterm list and calls them successfully. I tried with Claude 3.5 as provider and the issue remains the same, so I guess there is something wrong with the tool listing in my setup.

[DOsinga]

I'm going to close this since this does seem to work in general. Feel free to reopen with specific instructions on how to reproduce. Ollama is cool but tricky!