NPU models no longer listed in 0.7.117

[filipw]

I just upgraded to v0.7.117 on Surface Pro 11

All the NPU models have disappeared, foundry model list shows only CPU models

PS C:\Users\filip> foundry model list
Alias                          Device     Task               File Size    License      Model ID
-----------------------------------------------------------------------------------------------
phi-4                          CPU        chat-completion    10.16 GB     MIT          Phi-4-generic-cpu:1
----------------------------------------------------------------------------------------------------------
phi-3.5-mini                   CPU        chat-completion    2.53 GB      MIT          Phi-3.5-mini-instruct-generic-cpu:1
--------------------------------------------------------------------------------------------------------------------------
phi-3-mini-128k                CPU        chat-completion    2.54 GB      MIT          Phi-3-mini-128k-instruct-generic-cpu:2
-----------------------------------------------------------------------------------------------------------------------------
phi-3-mini-4k                  CPU        chat-completion    2.53 GB      MIT          Phi-3-mini-4k-instruct-generic-cpu:2
---------------------------------------------------------------------------------------------------------------------------
mistral-7b-v0.2                CPU        chat-completion    4.07 GB      apache-2.0   mistralai-Mistral-7B-Instruct-v0-2-generic-cpu:2
---------------------------------------------------------------------------------------------------------------------------------------
deepseek-r1-14b                CPU        chat-completion    11.51 GB     MIT          deepseek-r1-distill-qwen-14b-generic-cpu:3
---------------------------------------------------------------------------------------------------------------------------------
deepseek-r1-7b                 CPU        chat-completion    6.43 GB      MIT          deepseek-r1-distill-qwen-7b-generic-cpu:3
--------------------------------------------------------------------------------------------------------------------------------
qwen2.5-0.5b                   CPU        chat-completion    0.80 GB      apache-2.0   qwen2.5-0.5b-instruct-generic-cpu:3
--------------------------------------------------------------------------------------------------------------------------
qwen2.5-1.5b                   CPU        chat-completion    1.78 GB      apache-2.0   qwen2.5-1.5b-instruct-generic-cpu:3
--------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-0.5b             CPU        chat-completion    0.80 GB      apache-2.0   qwen2.5-coder-0.5b-instruct-generic-cpu:3
--------------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-7b               CPU        chat-completion    6.16 GB      apache-2.0   qwen2.5-coder-7b-instruct-generic-cpu:3
------------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-1.5b             CPU        chat-completion    1.78 GB      apache-2.0   qwen2.5-coder-1.5b-instruct-generic-cpu:3
--------------------------------------------------------------------------------------------------------------------------------
phi-4-mini                     CPU        chat-completion    4.80 GB      MIT          Phi-4-mini-instruct-generic-cpu:4
------------------------------------------------------------------------------------------------------------------------
phi-4-mini-reasoning           CPU        chat-completion    4.52 GB      MIT          Phi-4-mini-reasoning-generic-cpu:2
-------------------------------------------------------------------------------------------------------------------------
qwen2.5-14b                    CPU        chat-completion    11.06 GB     apache-2.0   qwen2.5-14b-instruct-generic-cpu:3
-------------------------------------------------------------------------------------------------------------------------
qwen2.5-7b                     CPU        chat-completion    6.16 GB      apache-2.0   qwen2.5-7b-instruct-generic-cpu:3
------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-14b              CPU        chat-completion    11.06 GB     apache-2.0   qwen2.5-coder-14b-instruct-generic-cpu:3

I am using Qualcomm® Hexagon™ NPU Driver version 1.0.0.11 (Sept 4th Release)

At initial startup after installation, the QNN execution provider appears to have been downloaded successfully:

🟢 Service is Started on http://localhost:5272/, PID 13080!
[                                    ]   0.00 % [Time remaining: about 3m20s]    Downloading QNNExecutionProvider       [                                    ]
... lots of progress....
QNNExecutionProvider       [####################################] 100.00 % [Time remaining: about 0s]       Downloading complete!

System Info

Processor Snapdragon(R) X 12-core X1E80100 @ 3.40 GHz, 3417 Mhz, 12 Core(s), 12 Logical Processor(s)
OS Name Microsoft Windows 11 Home
Version 10.0.27950 Build 27950

[natke]

@filipw Thank you for reporting this. Can you please run the following command in a PowerShell window

Get-AppxPackage -AllUsers "*.EP.*"  | Select-Object -ExpandProperty PackageFullName

[filipw]

it returns nothing:

PS C:\Users\filip> Get-AppxPackage -AllUsers "*.EP.*"  | Select-Object -ExpandProperty PackageFullName
PS C:\Users\filip>

Foundry itself is there

PS C:\Users\filip> Get-AppxPackage -AllUsers "*Foundry*" | Select-Object -ExpandProperty PackageFullName
Microsoft.FoundryLocal_0.7.117.26375_arm64__8wekyb3d8bbwe
PS C:\Users\filip>

I assume this implies the execution provider did not install successfully? Can I force-install it?

[leestott]

Same issue on Intel and Qualcomm NPU devices

Devices running windows 25H2
Intel NPU Surface Laptop 6
ARM Device Lenovo 14S Snapdragon X Elite

Foundry Logs clearly shows on CPU or GPU Models are being downloaded see

Arm Device

2025-09-25 08:54:30.435 +08:00 [INF] Starting Foundry Local CLI with 'model list'
2025-09-25 08:54:31.486 +08:00 [INF] Command:ServiceAutoRegister Status:Success Direct:False Time:3ms
2025-09-25 08:54:31.487 +08:00 [INF] UserAgent:foundry-local-CLI/0.7.117+67073234e7 Command:AutoRegisterCertifiedEps Status:Success Direct:True Time:0ms
2025-09-25 08:54:32.543 +08:00 [INF] Model Phi-4-reasoning-generic-cpu:1 does not have a valid prompt template.
2025-09-25 08:54:32.544 +08:00 [INF] Total models fetched across all pages: 18
2025-09-25 08:54:32.545 +08:00 [INF] UserAgent:foundry-local-CLI/0.7.117+67073234e7 Command:ModelList Status:Success Direct:True Time:2069ms
2025-09-25 08:54:32.547 +08:00 [INF] Command:ModelList Status:Success Direct:True Time:2105ms
2025-09-25 08:54:32.547 +08:00 [INF] Stream disconnected
2025-09-25 08:54:46.858 +08:00 [INF] Starting Foundry Local CLI with 'cache ls'
2025-09-25 08:54:46.867 +08:00 [INF] Command:ServiceStart Status:Skipped Direct:False Time:2ms
2025-09-25 08:54:46.892 +08:00 [INF] UserAgent:foundry-local-CLI/0.7.117+67073234e7 Command:ListDownloadedModels Status:Success Direct:True Time:2ms
2025-09-25 08:54:47.803 +08:00 [INF] Model Phi-4-reasoning-generic-cpu:1 does not have a valid prompt template.
2025-09-25 08:54:47.804 +08:00 [INF] Total models fetched across all pages: 18
2025-09-25 08:54:47.804 +08:00 [INF] UserAgent:foundry-local-CLI/0.7.117+67073234e7 Command:ModelList Status:Success Direct:True Time:907ms
2025-09-25 08:54:47.814 +08:00 [INF] Command:CacheList Status:Success Direct:True Time:949ms
2025-09-25 08:54:47.815 +08:00 [INF] Stream disconnected
2025-09-25 09:02:05.584 +08:00 [INF] Starting Foundry Local CLI with 'service restart'
2025-09-25 09:02:05.633 +08:00 [INF] Stopped Inference.Service.Agent PID 7020
2025-09-25 09:02:08.158 +08:00 [INF] Starting service <C:\Program Files\WindowsApps\Microsoft.FoundryLocal_0.7.117.26375_arm64__8wekyb3d8bbwe\Inference.Service.Agent.exe --urls="http://127.0.0.1:0/" --OpenAIServiceSettings:ModelDirPath="C:\Users*****--JsonRpcServer:Run=true --JsonRpcServer:PipeName="inference_agent" --Logging:LogLevel:Default="Information">
2025-09-25 09:02:08.382 +08:00 [INF] Service is started on http://127.0.0.1:53323/, PID 21528!
2025-09-25 09:02:08.382 +08:00 [INF] Command:ServiceRestart Status:Success Direct:True Time:2756ms
2025-09-25 09:02:08.383 +08:00 [INF] Stream disconnected
2025-09-25 09:02:17.984 +08:00 [INF] Starting Foundry Local CLI with 'model list'
2025-09-25 09:02:18.037 +08:00 [INF] Loaded cached model info for 18 models. SavedAt:9/25/2025 8:32:55 AM
2025-09-25 09:02:19.048 +08:00 [INF] UserAgent:foundry-local-CLI/0.7.117+67073234e7 Command:AutoRegisterCertifiedEps Status:Success Direct:True Time:0ms
2025-09-25 09:03:17.226 +08:00 [INF] Registering provider 1/1: QNNExecutionProvider
2025-09-25 09:03:17.243 +08:00 [INF] Successfully autoregistered QNNExecutionProvider
2025-09-25 09:03:17.243 +08:00 [INF] Finished attempt to autoregister certified EPs at 9/25/2025 9:03:17 AM; finished in 00:01:08.8798872
2025-09-25 09:03:17.253 +08:00 [INF] Successfully downloaded and registered the following EPs: QNNExecutionProvider.
Valid EPs: CPUExecutionProvider, QNNExecutionProvider

2025-09-25 09:03:17.253 +08:00 [INF] Command:ServiceAutoRegister Status:Success Direct:False Time:58210ms
2025-09-25 09:03:17.263 +08:00 [INF] Valid devices: CPU
2025-09-25 09:03:17.263 +08:00 [INF] Valid EPs: CPUExecutionProvider, QNNExecutionProvider
2025-09-25 09:03:19.232 +08:00 [INF] Model Phi-4-reasoning-generic-cpu:1 does not have a valid prompt template.
2025-09-25 09:03:19.234 +08:00 [INF] Total models fetched across all pages: 18
2025-09-25 09:03:19.234 +08:00 [INF] UserAgent:foundry-local-CLI/0.7.117+67073234e7 Command:ModelList Status:Success Direct:True Time:1978ms
2025-09-25 09:03:19.239 +08:00 [INF] Command:ModelList Status:Success Direct:True Time:61249ms
2025-09-25 09:03:19.240 +08:00 [INF] Stream disconnected

Intel Device

2025-09-25 00:37:57.380 +01:00 [INF] Starting Foundry Local CLI with '--help'
2025-09-25 00:38:04.183 +01:00 [INF] Starting Foundry Local CLI with 'model list'
2025-09-25 00:38:04.444 +01:00 [INF] Timeout connecting to service
System.TimeoutException: The operation has timed out.
at System.IO.Pipes.NamedPipeClientStream.ConnectInternal(Int32, CancellationToken, Int32) + 0xd5
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread, ExecutionContext, ContextCallback, Object) + 0x3d
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread, ExecutionContext, ContextCallback, Object) + 0x70
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task&, Thread) + 0x6a
--- End of stack trace from previous location ---
at Microsoft.Neutron.Rpc.Client.RpcSessionExtensions.d__2`1.MoveNext() + 0x6f
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.Common.ServiceManagement.d__8.MoveNext() + 0x6f
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.Common.ServiceManagement.d__9.MoveNext() + 0x5f
2025-09-25 00:38:04.445 +01:00 [INF] Starting service <C:\Program Files\WindowsApps\Microsoft.FoundryLocal_0.7.117.26375_x64__8wekyb3d8bbwe\Inference.Service.Agent.exe --urls="http://127.0.0.1:0/" --OpenAIServiceSettings:ModelDirPath="C:\Users***" --JsonRpcServer:Run=true --JsonRpcServer:PipeName="inference_agent" --Logging:LogLevel:Default="Information">
2025-09-25 00:38:05.035 +01:00 [INF] Service endpoints are not yet bound, waiting to retry...
2025-09-25 00:38:05.039 +01:00 [INF] Now listening on: http://127.0.0.1:51354
2025-09-25 00:38:05.040 +01:00 [INF] Application started. Press Ctrl+C to shut down.
2025-09-25 00:38:05.040 +01:00 [INF] Hosting environment: Production
2025-09-25 00:38:05.040 +01:00 [INF] Content root path: C:\Program Files\WindowsApps\Microsoft.FoundryLocal_0.7.117.26375_x64__8wekyb3d8bbwe
2025-09-25 00:38:05.069 +01:00 [INF] Downloading provider 1/1: OpenVINOExecutionProvider
2025-09-25 00:38:05.546 +01:00 [INF] Service is started on http://127.0.0.1:51354/, PID 28088!
2025-09-25 00:38:06.585 +01:00 [INF] UserAgent:foundry-local-CLI/0.7.117+67073234e7 Command:AutoRegisterCertifiedEps Status:Success Direct:True Time:0ms
2025-09-25 00:38:19.380 +01:00 [INF] Registering provider 1/1: OpenVINOExecutionProvider
2025-09-25 00:38:19.405 +01:00 [INF] Successfully autoregistered OpenVINOExecutionProvider
2025-09-25 00:38:19.406 +01:00 [INF] Finished attempt to autoregister certified EPs at 25/09/2025 00:38:19; finished in 00:00:14.3948860
2025-09-25 00:38:19.432 +01:00 [INF] Successfully downloaded and registered the following EPs: OpenVINOExecutionProvider.
Valid EPs: CPUExecutionProvider, OpenVINOExecutionProvider, WebGpuExecutionProvider

2025-09-25 00:38:19.432 +01:00 [INF] Command:ServiceAutoRegister Status:Success Direct:False Time:12852ms
2025-09-25 00:38:19.455 +01:00 [INF] Valid devices: CPU, GPU
2025-09-25 00:38:19.455 +01:00 [INF] Valid EPs: CPUExecutionProvider, OpenVINOExecutionProvider, WebGpuExecutionProvider
2025-09-25 00:38:20.304 +01:00 [INF] Model Phi-4-reasoning-generic-cpu:1 does not have a valid prompt template.
2025-09-25 00:38:20.305 +01:00 [INF] Model Phi-4-reasoning-generic-gpu:1 does not have a valid prompt template.
2025-09-25 00:38:20.305 +01:00 [INF] Total models fetched across all pages: 34
2025-09-25 00:38:20.305 +01:00 [INF] UserAgent:foundry-local-CLI/0.7.117+67073234e7 Command:ModelList Status:Success Direct:True Time:871ms
2025-09-25 00:38:20.312 +01:00 [INF] Command:ModelList Status:Success Direct:True Time:16123ms
2025-09-25 00:38:20.313 +01:00 [INF] Stream disconnected

[leestott]

Downgrading to https://github.com/microsoft/Foundry-Local/releases/tag/v0.6.87 confirms NPU Models are available and can be successfully used.

[jaholme]

@filipw What Windows Insider build Dev/Canary is that device on?

[timenick]

@leestott Windows 25H2 issue is fixed. Could you please try using version 0.7.117 again to see if the EP downloads successfully? Thank you.

[filipw]

I am on Canary, so it updates every few days, currently on latest 27954.1 from September 25th.

[leestott]

Yes this is now working on 25h2

Confirmed NPU Models are now visable

However we now have a QNN driver issue for Phi-4 models see #262

[leestott]

This is also resolved on Windows Intel

Also the Phi-4 models respond well on Intel devices so this indicates a driver issue on ARM Devices as per old issue #136

[natke]

@filipw Are you able to list NPU models on your Qualcomm device now?

[filipw]

This is what I get right now (no models returned, using 0.7.117.26375)

PS C:\Users\filip> foundry model list
Exception: No models were returned from the Azure Foundry catalog.

OK, what if I downgrade?

PS C:\Users\filip> winget uninstall Microsoft.FoundryLocal
Found Foundry Local Model Server [Microsoft.FoundryLocal]
Starting package uninstall...
  ██████████████████████████████  100%
Successfully uninstalled
PS C:\Users\filip> winget install --id=Microsoft.FoundryLocal -v "0.6.87.59034" -e
Found Foundry Local [Microsoft.FoundryLocal] Version 0.6.87.59034
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
This package requires the following dependencies:
  - Packages
      Microsoft.VCLibs.Desktop.14 [>= 14.0.33728.0]
Successfully verified installer hash
Starting package install...
  ██████████████████████████████  100%
Successfully installed

On 0.6.87.59034 it works:

PS C:\Users\filip> foundry model list
🟢 Service is Started on http://127.0.0.1:55651/, PID 21152!
Alias                          Device     Task               File Size    License      Model ID
-----------------------------------------------------------------------------------------------
phi-4                          CPU        chat-completion    10.16 GB     MIT          Phi-4-generic-cpu
--------------------------------------------------------------------------------------------------------
phi-3.5-mini                   CPU        chat-completion    2.53 GB      MIT          Phi-3.5-mini-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------
deepseek-r1-14b                NPU        chat-completion    7.12 GB      MIT          deepseek-r1-distill-qwen-14b-qnn-npu
---------------------------------------------------------------------------------------------------------------------------
deepseek-r1-7b                 NPU        chat-completion    3.71 GB      MIT          deepseek-r1-distill-qwen-7b-qnn-npu
--------------------------------------------------------------------------------------------------------------------------
phi-3-mini-128k                CPU        chat-completion    2.54 GB      MIT          Phi-3-mini-128k-instruct-generic-cpu
---------------------------------------------------------------------------------------------------------------------------
phi-3-mini-4k                  CPU        chat-completion    2.53 GB      MIT          Phi-3-mini-4k-instruct-generic-cpu
-------------------------------------------------------------------------------------------------------------------------
mistral-7b-v0.2                CPU        chat-completion    4.07 GB      apache-2.0   mistralai-Mistral-7B-Instruct-v0-2-generic-cpu
-------------------------------------------------------------------------------------------------------------------------------------
phi-4-mini-reasoning           NPU        chat-completion    2.78 GB      MIT          Phi-4-mini-reasoning-qnn-npu
                               CPU        chat-completion    4.52 GB      MIT          Phi-4-mini-reasoning-generic-cpu
-----------------------------------------------------------------------------------------------------------------------
qwen2.5-0.5b                   CPU        chat-completion    0.80 GB      apache-2.0   qwen2.5-0.5b-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------
qwen2.5-1.5b                   CPU        chat-completion    1.78 GB      apache-2.0   qwen2.5-1.5b-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-0.5b             CPU        chat-completion    0.80 GB      apache-2.0   qwen2.5-coder-0.5b-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-7b               CPU        chat-completion    6.16 GB      apache-2.0   qwen2.5-coder-7b-instruct-generic-cpu
----------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-1.5b             CPU        chat-completion    1.78 GB      apache-2.0   qwen2.5-coder-1.5b-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------------
qwen2.5-14b                    CPU        chat-completion    11.06 GB     apache-2.0   qwen2.5-14b-instruct-generic-cpu
-----------------------------------------------------------------------------------------------------------------------
qwen2.5-7b                     CPU        chat-completion    6.16 GB      apache-2.0   qwen2.5-7b-instruct-generic-cpu
----------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-14b              CPU        chat-completion    11.06 GB     apache-2.0   qwen2.5-coder-14b-instruct-generic-cpu

OK, so let's upgrade to 0.7.117.26375 again

PS C:\Users\filip> winget install Microsoft.FoundryLocal
Found an existing package already installed. Trying to upgrade the installed package...
Found Foundry Local [Microsoft.FoundryLocal] Version 0.7.117.26375
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
This package requires the following dependencies:
  - Packages
      Microsoft.VCLibs.Desktop.14 [>= 14.0.33728.0]
Successfully verified installer hash
Starting package install...
  ████████████████████████████▌   95%
Successfully installed. Restart the application to complete the upgrade.

Now model listing seems to work, but it's a bit suspicious, since the list is identical to the one before, and yesterday after installing 0.7.117.26375 I would get info that EP is being downloaded?

PS C:\Users\filip> foundry model list
Alias                          Device     Task               File Size    License      Model ID
-----------------------------------------------------------------------------------------------
phi-4                          CPU        chat-completion    10.16 GB     MIT          Phi-4-generic-cpu
--------------------------------------------------------------------------------------------------------
phi-3.5-mini                   CPU        chat-completion    2.53 GB      MIT          Phi-3.5-mini-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------
deepseek-r1-14b                NPU        chat-completion    7.12 GB      MIT          deepseek-r1-distill-qwen-14b-qnn-npu
---------------------------------------------------------------------------------------------------------------------------
deepseek-r1-7b                 NPU        chat-completion    3.71 GB      MIT          deepseek-r1-distill-qwen-7b-qnn-npu
--------------------------------------------------------------------------------------------------------------------------
phi-3-mini-128k                CPU        chat-completion    2.54 GB      MIT          Phi-3-mini-128k-instruct-generic-cpu
---------------------------------------------------------------------------------------------------------------------------
phi-3-mini-4k                  CPU        chat-completion    2.53 GB      MIT          Phi-3-mini-4k-instruct-generic-cpu
-------------------------------------------------------------------------------------------------------------------------
mistral-7b-v0.2                CPU        chat-completion    4.07 GB      apache-2.0   mistralai-Mistral-7B-Instruct-v0-2-generic-cpu
-------------------------------------------------------------------------------------------------------------------------------------
phi-4-mini-reasoning           NPU        chat-completion    2.78 GB      MIT          Phi-4-mini-reasoning-qnn-npu
                               CPU        chat-completion    4.52 GB      MIT          Phi-4-mini-reasoning-generic-cpu
-----------------------------------------------------------------------------------------------------------------------
qwen2.5-0.5b                   CPU        chat-completion    0.80 GB      apache-2.0   qwen2.5-0.5b-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------
qwen2.5-1.5b                   CPU        chat-completion    1.78 GB      apache-2.0   qwen2.5-1.5b-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-0.5b             CPU        chat-completion    0.80 GB      apache-2.0   qwen2.5-coder-0.5b-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-7b               CPU        chat-completion    6.16 GB      apache-2.0   qwen2.5-coder-7b-instruct-generic-cpu
----------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-1.5b             CPU        chat-completion    1.78 GB      apache-2.0   qwen2.5-coder-1.5b-instruct-generic-cpu
------------------------------------------------------------------------------------------------------------------------------
qwen2.5-14b                    CPU        chat-completion    11.06 GB     apache-2.0   qwen2.5-14b-instruct-generic-cpu
-----------------------------------------------------------------------------------------------------------------------
qwen2.5-7b                     CPU        chat-completion    6.16 GB      apache-2.0   qwen2.5-7b-instruct-generic-cpu
----------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-14b              CPU        chat-completion    11.06 GB     apache-2.0   qwen2.5-coder-14b-instruct-generic-cpu

Let's try to run a model - unfortunately it fails

PS C:\Users\filip> foundry model run deepseek-r1-7b
Downloading deepseek-r1-distill-qwen-7b-qnn-npu...
[####################################] 100.00 % [Time remaining: about 0s]        22.8 MB/s
🕛 Loading model... [17:24:43 ERR] Failed loading model:deepseek-r1-distill-qwen-7b-qnn-npu
Exception: Failed: Loading model deepseek-r1-distill-qwen-7b-qnn-npu from http://127.0.0.1:55651/openai/load/deepseek-r1-distill-qwen-7b-qnn-npu?ttl=600
Internal Server Error
Failed loading model deepseek-r1-distill-qwen-7b-qnn-npu
Could not find an implementation for EPContext(1) node with name 'QNNExecutionProvider_QNN_part0_5289039109093371165_8_0'
PS C:\Users\filip>

[filipw]

This is in the logs:

2025-09-26 17:29:57.773 +02:00 [INF] Starting Foundry Local CLI with 'model run deepseek-r1-7b'
2025-09-26 17:29:57.783 +02:00 [INF] Command:ServiceStart Status:Skipped Direct:False  Time:2ms
2025-09-26 17:29:58.748 +02:00 [INF] Model Phi-4-reasoning-generic-cpu does not have a valid prompt template.
2025-09-26 17:29:58.750 +02:00 [INF] Model deepseek-r1-distill-qwen-14b-generic-cpu is not supported on Arm64 currently.
2025-09-26 17:29:58.750 +02:00 [INF] Model deepseek-r1-distill-qwen-7b-generic-cpu is not supported on Arm64 currently.
2025-09-26 17:29:58.751 +02:00 [INF] Model Phi-4-mini-instruct-generic-cpu is not supported on Arm64 currently.
2025-09-26 17:29:58.751 +02:00 [INF] Total models fetched across all pages: 17
2025-09-26 17:29:58.781 +02:00 [INF] Command:ModelDownload Status:Skipped Direct:False  Time:997ms
2025-09-26 17:29:58.782 +02:00 [INF] Command:ServiceList Status:Success Direct:False  Time:0ms
2025-09-26 17:29:58.782 +02:00 [INF] Loading model: http://127.0.0.1:55651/openai/load/deepseek-r1-distill-qwen-7b-qnn-npu?ttl=600
2025-09-26 17:29:58.783 +02:00 [INF] Loading model:deepseek-r1-distill-qwen-7b-qnn-npu
2025-09-26 17:31:29.265 +02:00 [ERR] Failed loading model:deepseek-r1-distill-qwen-7b-qnn-npu
2025-09-26 17:31:29.265 +02:00 [INF] Command:ModelLoad Status:Failure Direct:False  Time:90483ms
2025-09-26 17:31:29.265 +02:00 [INF] Command:ModelRun Status:Failure Direct:True  Time:91484ms
2025-09-26 17:31:29.266 +02:00 [INF] Stream disconnected
2025-09-26 17:31:29.273 +02:00 [INF] LogException
Microsoft.AI.Foundry.Local.Common.FLException: Failed: Loading model deepseek-r1-distill-qwen-7b-qnn-npu from http://127.0.0.1:55651/openai/load/deepseek-r1-distill-qwen-7b-qnn-npu?ttl=600
 ---> System.Net.Http.HttpRequestException: Internal Server Error
Failed loading model deepseek-r1-distill-qwen-7b-qnn-npu
Could not find an implementation for EPContext(1) node with name 'QNNExecutionProvider_QNN_part0_5289039109093371165_8_0'
   at Microsoft.AI.Foundry.Local.Common.Utils.EnsureSuccessStatusCode(HttpResponseMessage, String, Func`2) + 0xe4
   --- End of inner exception stack trace ---
   at Microsoft.AI.Foundry.Local.Common.Utils.EnsureSuccessStatusCode(HttpResponseMessage, String, Func`2) + 0x140
   at Microsoft.AI.Foundry.Local.Common.ModelManagement.<LoadModelAsync>d__10.MoveNext() + 0x8b4
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Commands.ModelRunCommand.<<Create>b__1_0>d.MoveNext() + 0x1000
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Common.CommandActionFactory.<>c__DisplayClass0_0`1.<<Create>b__0>d.MoveNext() + 0x238
--- End of stack trace from previous location ---
   at System.CommandLine.NamingConventionBinder.CommandHandler.<GetExitCodeAsync>d__66.MoveNext() + 0x5c
--- End of stack trace from previous location ---
   at System.CommandLine.NamingConventionBinder.ModelBindingCommandHandler.<InvokeAsync>d__11.MoveNext() + 0x6c
--- End of stack trace from previous location ---
   at System.CommandLine.Invocation.InvocationPipeline.<InvokeAsync>d__0.MoveNext() + 0x1f4
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Program.<Main>d__1.MoveNext() + 0x52c

[leestott]

@filipw

Seems to work for me
Processor Snapdragon(R) X Elite - X1E78100 - Qualcomm(R) Oryon(TM) CPU, 3417 Mhz, 12 Core(s), 12 Logical Processor(s)
Installed Physical Memory (RAM) 32.0 GB

Loading personal and system profiles took 4233ms.
 foundry --version
0.7.117+67073234e7
 foundry model run deepseek-r1-7b
Model deepseek-r1-distill-qwen-7b-qnn-npu:1 was found in the local cache.
🕘 Loading model...
🟢 Model deepseek-r1-distill-qwen-7b-qnn-npu:1 loaded successfully

Interactive Chat. Enter /? or /help for help.
Press Ctrl+C to cancel generation. Type /exit to leave the chat.

Interactive mode, please enter your prompt

what is the capital of spain
🧠 Thinking...
🤖
Okay, so I need to figure out the capital of Spain. Hmm, I remember Spain is a country in Europe, right? I think it's in the Iberian Peninsula. I've heard of Madrid before, but I'm not 100% sure if that's the capital or just a big city there. Maybe it's Madrid? I think Barcelona is also pretty famous, but I don't think that's the capital. Let me try to recall any other capitals I know. For example, France's capital is Paris, Italy's is Rome, Germany's is Berlin. So Spain must have a capital too. Since I'm not sure about Madrid, maybe I should think about other Spanish cities. Oh, I think there's a city called Madrid that's a major city and a center of government. Yeah, that makes sense because I've heard it mentioned a lot, especially in news or travel. So I'm going to go with Madrid as the capital of Spain.

The capital of Spain is Madrid.

[natke]

@filipw, can you please try the following steps:

Get-AppxPackage -AllUsers "*.EP.*"  | Select-Object -ExpandProperty PackageFullName

If this shows an EP package is installed, remove it.

Now run this command (cleans up metadata associated with the downloaded package):

remove-appxpackage uup://product/Windows.Workload.ExecutionProvider.QNN.arm64

Run

foundry model list

again, to see the NPU models.

Run one of the models to make sure it is working.