NPU error while using llm-agent-rag-llamaindex on Lunar Lake

[js333031]

Using conda create -n ov-nb-demos python=3.11 to create the env to run the llm-agent-rag-llamaindex notebook

Followed steps described here to setup the env

When "Create OpenVINO Embedding" section is executed, error below appears in the terminal that launched the jupyter env.

[ERROR] 10:17:07.584 [vpux-compiler] Got Diagnostic at loc(fused<{name = "aten::masked_fill/Select", type = "Select"}>["aten::masked_fill/Select"]) : Got non broadcastable dimensions pair : '0' and -9223372036854775808'
loc(fused<{name = "aten::masked_fill/Select", type = "Select"}>["aten::masked_fill/Select"]): error: Got non broadcastable dimensions pair : '0' and -9223372036854775808'
LLVM ERROR: Failed to infer result type(s).
[I 2025-01-25 07:17:16.322 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (1/5), keep random ports

OS Name Microsoft Windows 11 Pro
Version 10.0.26100 Build 26100
System Model Dell XPS 13 9350
BIOS Version/Date Dell Inc. 1.7.1, 12/6/2024
Processor Intel(R) Core(TM) Ultra 7 258V, 2200 Mhz, 8 Core(s), 8 Logical Processor(s)
NPU Driver is 32.0.100.3104 (latest as of Jan 25, '25)

[zulkifli-halim]

Hi @js333031

I wasn't able to reproduce the issue as I could execute the cell without any error. Can you use Python 3.10 to see if the issue is resolved?

[zulkifli-halim]

Hi @js333031, please ignore the previous reply. I wasn't able to run on NPU, and I believe that the notebook cannot run on NPU as of now.

[openvino-dev-samples]

For deploying embedding model on NPU, pls refer the approach in this notebook: llm-rag-llamaindex.ipynb