[bounty] make embedded AI work on windows

[louis030195]

screenpipe/screenpipe-app-tauri/src-tauri/src/llm_sidecar.rs

Line 62 in 7922064

serve_command = serve_command.env("OLLAMA_ORIGINS", "*");

ollama on windows needs OLLAMA_ORIGINS=* because its CORS blocks Tauri network protocol

for some reason the thing i did does not work (adding the env in the command)

/bounty 50

definition of done:

• windows users can run embedded AI and use it in the chat or timeline or elsewhere
• e.g. fix it, test it, share screenshot/video to prove it works (maybe use setx or idk)

also it would be good in general to make sure the OLLAMA_ORIGINS=* is sat regardless if the user use embedded AI, but do this code in app code (not CLI)

[linear]

MED-283 [bounty] make embedded AI work on windows

[algora-pbc]

💎 $50 bounty • Screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #682 with your implementation plan
2. Submit work: Create a pull request including /claim #682 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to mediar-ai/screenpipe!

Add a bounty • Share on socials

[tribhuwan-kumar]

ollama/ollama#703 (comment)

i think this is going to fix this issue!

[tribhuwan-kumar]

i'm able to run this but its searching for cublas64_11.dll instead of cublas64_12.dll which doesn't exists (cublas64_11.dll)
maybe i need to also fix this #602

ps: this is awesome SYSTRAN/faster-whisper#535 (comment)

[louis030195]

i'm able to run this but its searching for cublas64_11.dll instead of cublas64_12.dll which doesn't exists (cublas64_11.dll) maybe i need to also fix this #602

ps: this is awesome SYSTRAN/faster-whisper#535 (comment)

how do you ran this ollama?

here's the thing:

screenpipe/screenpipe-app-tauri/scripts/pre_build.js

Line 540 in 7922064

const oldLibs = [

we have to delete old cuda libs because tauri crash if the build has too many files, so we only keep cuda 12 libs

do you have a nvidia gpu? which? maybe only support cuda 11

[tribhuwan-kumar]

how do you ran this ollama?

by adding this line serve_command = serve_command.env("OLLAMA_HOST", "0.0.0.0"); in windows env, even though its not neccesary there is already env for OLLAMA_HOST, here:

screenpipe/screenpipe-app-tauri/src-tauri/src/llm_sidecar.rs

Line 60 in 7922064

#[cfg(target_os = "windows")]

then from app ui started it

we have to delete old cuda libs because tauri crash if the build has too many files, so we only keep cuda 12 libs
do you have a nvidia gpu? which? maybe only support cuda 11,

yes, i tested in rtx 2050 which old, but its cuda capable
i also have rtx 4060, & i think this issue shouldn't cause in rtx 4060

[tribhuwan-kumar]

ollama on windows needs OLLAMA_ORIGINS=* because its CORS blocks Tauri network protocol

for some reason the thing i did does not work (adding the env in the command)

just tested with full lib folder without deleting anything, and it worked like a charm, there is not OLLAMA_HOST or OLLAMA_ORIGIN error.
if there is even an error related to ollama's env, i can't reproduce it :(

[tribhuwan-kumar]

since the problem with build, so we can't keep all the ollama lib files, in that case users must have to install cuda toolkit, (i've tested this) which can be rough for normal windows users also cuda toolkit takes too much space.

maybe we could use something like this tauri-apps/tauri#7372 (comment) this way embedded ai will not depend on cuda toolkit and all the requirement of cublas64_11.dll etc will also satisfied

[tribhuwan-kumar]

this issue isn't causing in rtx4060

[louis030195]

hmm yeah maybe something likle the ffmpeg-sidecar we use that download ollama at runtime and install it?

[tribhuwan-kumar]

hmm yeah maybe something likle the ffmpeg-sidecar we use that download ollama at runtime and install it?

this should work, working on it!!

[louis030195]

@tribhuwan-kumar btw the best would be that it's in the screenpipe core code under the llm feature flag

we had done this with Candle lib but in the end used ollama because there was too much stuff to do
so there is some leftover dead code related to this that you can safely remove and we would use this ollama downloaded/sidecar for all OSes at the core lib level

so CLI users could also have the LLM thing without the app

hope it's clear enough

[tribhuwan-kumar]

@tribhuwan-kumar btw the best would be that it's in the screenpipe core code under the llm feature flag

we had done this with Candle lib but in the end used ollama because there was too much stuff to do so there is some leftover dead code related to this that you can safely remove and we would use this ollama downloaded/sidecar for all OSes at the core lib level

so CLI users could also have the LLM thing without the app

hope it's clear enough

yes, that'd be better. all kind of users can use the embeded ai

[tribhuwan-kumar]

sorry for the late, i had some exams, i'll open a pr soon!