Being able to stop api/pull models download stream

[JulienElkaim]

Feature request

Hi,

I would like a feature to make it possible to cancel a pullModelStream , so Ollama stops downloading the model if we want to.

Motivation

In my Ollama client, I want to give users the right to missclick. Therefore, if they finally decide they don't want the huge models on their computer, they can stop downloading it !

Your contribution

Please share your preferences of implementation, and I will be happy to contribute

[davidmigloz]

Hey @JulienElkaim,

Thanks for bringing this up.

The problem is that the official http package doesn't support cancelling requests yet. Do you have any suggestions?

[JulienElkaim]

Hi,

Interesting for an official http package!
I understand they feel it's negligible most of time... but with models that size the stake is high. As official library they should support these edge cases :/

I would hope we can support it. I tried with curls & ctrl+C and it confirms cancelling http request stops download, see screenshot:

Could we switch http library? I did not dig into your codebase yet so I am not sure how big it is.
If you like the idea, I can do further analysis. It seems dio package support cancellation, If I understand correctly.

• How open would you be to switch to dio?
• Do you prefer the same http package dependency for ALL Ollama calls or can we only switch library for api/pull ?

I can give a try with dio and provide an example next week. See if switching library is worth it before doing anything :)

[davidmigloz]

I agree, it's a pity that such a basic feature is still not supported.

Migrating to another client would have quite some impact, as we generate the OllamaClient from the OpenAPI spec using the openapi_spec generator. So, first we would need to add support for different clients in the generator... And using different clients for different methods would be a bit confusing, I think.

Do you know if there is any alternative that wraps the official http client and adds support for cancelling requests without modifying its API? That would be an ideal solution, as you can always pass your custom http client:

final client = OllamaClient(
  client: CancellableHttpClient(),
);

[davidmigloz]

An example would be https://pub.dev/packages/fetch_client (which is used internally by OllamaClient when targetting web to support streaming). It implements the same http API, but under-the-hood it uses the JS fetch API which supports cancelling requests.

[davidmigloz]

I'll close this issue for now, as it's not actionable at the moment. We'll reconsider it if http package adds the needed functionality or if we decide to migrate to a different http client.