Cannot get a query to complete, or engage, with my GPU.

[corsairius]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

---

Steps to reproduce

1. Make query, like "How can a cat stop marking inside the house?" Or "What is the population of the US?"
2. Check the GPU with 'nvidia-smi -l' and check for Chat AI responses.
3. Looked in the container logs and the host logs. Couldn't see anything erroring out.

The entire AI deployment has been removed. The containers were removed when stopped, and all of the images were also removed. The Nextcloud Chat with AI isn’t operational.

Then the AppAPI was added along with a Deployment test. Nextcloud Assistant was added as an App. Then the LLM2 was added to implement AI Chat. The system is not yet querying the LLM2 container, and isn’t showing as having engaged the GPU, as seen through the nvidia-smi tool. Also have four workers running.

Expected behaviour

Something.

Actual behaviour

Seems to accrue time. Sometimes an error message for the query (a toast indicator).

Server configuration

Nextcloud 30.0.7
nginx 1.26.2
postgresql 16.6.1
PHP fpm 8.2
Ubuntu 20.04
NVidia 12Gb A2000
128Gb

List of activated apps

- activity: 3.0.0 - announcementcenter: 7.1.0 - app_api: 4.0.6 - assistant: 2.4.0 - bruteforcesettings: 3.0.0 - calendar: 5.2.0 - circles: 30.0.0 - cloud_federation_api: 1.13.0 - comments: 1.20.1 - contacts: 7.0.4 - contactsinteraction: 1.11.0 - dashboard: 7.10.0 - dav: 1.31.1 - deck: 1.14.4 - external: 5.5.2 - externalportal: 1.3.1 - facerecognition: 0.9.60 - federatedfilesharing: 1.20.0 - federation: 1.20.0 - files: 2.2.0 - files_downloadlimit: 3.0.0 - files_external: 1.22.0 - files_pdfviewer: 3.0.0 - files_reminders: 1.3.0 - files_sharing: 1.22.0 - files_trashbin: 1.20.1 - files_versions: 1.23.0 - fileslibreofficeedit: 2.0.1 - firstrunwizard: 3.0.0 - geoblocker: 0.5.16 - groupfolders: 18.1.2 - logreader: 3.0.0 - lookup_server_connector: 1.18.0 - mail: 4.3.1 - memories: 7.5.2 - nextcloud_announcements: 2.0.0 - notes: 4.11.0 - notifications: 3.0.0 - oauth2: 1.18.1 - onlyoffice: 9.7.0 - password_policy: 2.0.0 - photos: 3.0.2 - previewgenerator: 5.8.0 - privacy: 2.0.0 - provisioning_api: 1.20.0 - recognize: 8.2.0 - recommendations: 3.0.0 - related_resources: 1.5.0 - richdocumentscode: 24.4.1302 - serverinfo: 2.0.0 - settings: 1.13.0 - sharebymail: 1.20.0 - spreed: 20.1.5 - support: 2.0.0 - survey_client: 2.0.0 - suspicious_login: 8.0.0 - systemtags: 1.20.0 - tasks: 0.16.1 - text: 4.1.0 - theming: 2.6.0 - twofactor_backupcodes: 1.19.0 - updatenotification: 1.20.0 - user_status: 1.10.0 - user_usage_report: 1.14.0 - viewer: 3.0.0 - weather_status: 1.10.0 - webhook_listeners: 1.1.0-dev - whiteboard: 1.0.5 - workflow_media_converter: 1.13.5 - workflowengine: 2.12.0 ``` If you have access to your command line run e.g.: sudo -u www-data php occ app:list from within your Nextcloud installation folder ```

Nextcloud configuration

{ "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "***REMOVED SENSITIVE VALUE***", "192.168.64.234:443", "192.168.64.234:8888", "localhost:8888" ], "loglevel": 1, "session_lifetime": 86400, "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "pgsql", "version": "30.0.8.1", "overwrite.cli.url": "https:\/\/server_name.net", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "5432", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "enable_previews": true, "enabledPreviewProviders": [ "OC\\Preview\\PNG", "OC\\Preview\\JPEG", "OC\\Preview\\JPG", "OC\\Preview\\GIF", "OC\\Preview\\BMP", "OC\\Preview\\XBitmap", "OC\\Preview\\Movie", "OC\\Preview\\PDF", "OC\\Preview\\MP3", "OC\\Preview\\TXT", "OC\\Preview\\MarkDown", "OC\\Preview\\MP4", "OC\\Preview\\MOV", "OC\\Preview\\Image", "OC\\Preview\\Photoshop", "OC\\Preview\\TIFF", "OC\\Preview\\SVG", "OC\\Preview\\HEIC", "OC\\Preview\\TIFF", "OC\\Preview\\EMF", "OCA\\Richdocuments\\Preview\\EMF" ], "htaccess.RewriteBase": "\/", "installed": true, "mail_smtpmode": "smtp", "mail_smtpsecure": "tls", "mail_sendmailmode": "smtp", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtpauthtype": "LOGIN", "mail_smtpauth": 1, "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "default_phone_region": "US", "maintenance": false, "preview_max_x": 512, "preview_max_y": 512, "jpeg_quality": 60, "theme": "", "updater.release.channel": "stable", "app_install_overwrite": [ "facerecognition", "files_external_ipfs", "duplicatefinder", "caniupdate" ], "memories.exiftool": "\/usr\/share\/nginx\/nextcloud\/apps\/memories\/bin-ext\/exiftool-amd64-glibc", "memories.vod.path": "\/usr\/share\/nginx\/nextcloud\/apps\/memories\/bin-ext\/go-vod-amd64", "memories.index.path.blacklist": "\\\/(@(Recycle|eaDir|Notes( \\([0-9]+\\)*))|Notes( \\([0-9]+\\))*)\\\/", "memories.index.mode": "3", "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg", "memories.vod.ffprobe": "\/usr\/bin\/ffprobe", "memories.exiftool_no_local": true, "memories.db.triggers.fcu": true, "preview_max_memory": 4096, "preview_max_filesize_image": 256, "memcache.local": "\\OC\\Memcache\\APCu", "filelocking.enabled": true, "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 0, "timeout": 0 }, "maintenance_window_start": 100 } }

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your Nextcloud installation folder

Browser

Browser name: Firefox/Chrome/Safari/…

Browser version: 124/125/…

Operating system: Windows/Ubuntu/Mac/…

Browser log

Insert your browser log here, this could for example include:
a) The javascript console log
b) The network log
c) ...

[marcelklehr]

Hello @corsairius
Thank you for your feedback, I'll try to help troubleshoot.
Can you check the logs of the llm2 docker container and paste a portion of them here that covers the time when you made a chat request? It would be interesting if the task hits the app at all. Also, can you paste the output of

./occ app_api:app:list

[corsairius]

App_API list

sudo -u www-data php occ app_api:app:list
ExApps:
llm2 (Local large language model): 2.3.4 [enabled]
stt_whisper2 (Local Whisper Speech To Text): 2.1.3 [enabled]
test-deploy (TestDeploy): 1.1.0 [enabled]

This is just a list of logs. I didn't see anything useful in the latest logs.

docker exec -it 8579c4b047ea /bin/bash
root@llm2:/app/lib# cd /var/log/
root@llm2:/var/log# ls -ltr
total 420
-rw-rw-r-- 1 root utmp      0 Jan 26 02:05 wtmp
-rw-rw---- 1 root utmp      0 Jan 26 02:05 btmp
-rw-rw-r-- 1 root utmp  29492 Jan 26 02:05 lastlog
-rw-r--r-- 1 root root   3232 Jan 26 02:05 faillog
-rw-r--r-- 1 root root  64549 Jan 26 02:05 bootstrap.log
drwxr-xr-x 1 root root   4096 Mar 21 10:05 apt
-rw-r--r-- 1 root root  16141 Mar 21 10:06 alternatives.log
-rw-r--r-- 1 root root 305224 Mar 21 10:06 dpkg.log

I can, however, run the Whisper Speech-to-text app.
I tried another App_API registration, but that didn't move the ball at all. Still the same result. Started looking into the code itself but busy with work stuff.

Thanks!

[marcelklehr]

The logs are in docker and should be accessible via

$ docker logs nc_app_llm2

[corsairius]

Currently receiving this error continuously:

2025-03-30 18:20:47,571 - httpx - INFO - HTTP Request: GET https://server_name.net/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content"

Full server instantiation:

INFO: Started server process [1]
INFO: Waiting for application startup.
TRACE: ASGI [1] Started scope={'type': 'lifespan', 'asgi': {'version': '3.0', 'spec_version': '2.0'}, 'state': {}}
TRACE: ASGI [1] Receive {'type': 'lifespan.startup'}
2025-03-30 18:11:46,038 - httpx - INFO - HTTP Request: GET https://server_name.net/ocs/v1.php/apps/app_api/ex-app/state?format=json "HTTP/1.1 200 OK"
TRACE: ASGI [1] Send {'type': 'lifespan.startup.complete'}
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:23001 (Press CTRL+C to quit)
TRACE: 127.0.0.1:59376 - HTTP connection made
TRACE: 127.0.0.1:59376 - ASGI [2] Started scope={'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('127.0.0.1', 23001), 'client': ('127.0.0.1', 59376), 'scheme': 'http', 'root_path': '', 'headers': '<...>', 'state': {}, 'method': 'GET', 'path': '/heartbeat', 'raw_path': b'/heartbeat', 'query_string': b''}
TRACE: 127.0.0.1:59376 - ASGI [2] Send {'type': 'http.response.start', 'status': 200, 'headers': '<...>'}
INFO: 127.0.0.1:59376 - "GET /heartbeat HTTP/1.1" 200 OK
TRACE: 127.0.0.1:59376 - ASGI [2] Send {'type': 'http.response.body', 'body': '<15 bytes>'}
TRACE: 127.0.0.1:59376 - ASGI [2] Completed
TRACE: 127.0.0.1:59376 - ASGI [3] Started scope={'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('127.0.0.1', 23001), 'client': ('127.0.0.1', 59376), 'scheme': 'http', 'root_path': '', 'headers': '<...>', 'state': {}, 'method': 'POST', 'path': '/init', 'raw_path': b'/init', 'query_string': b''}
TRACE: 127.0.0.1:59376 - ASGI [3] Send {'type': 'http.response.start', 'status': 200, 'headers': '<...>'}
INFO: 127.0.0.1:59376 - "POST /init HTTP/1.1" 200 OK
TRACE: 127.0.0.1:59376 - ASGI [3] Send {'type': 'http.response.body', 'body': '<2 bytes>'}
TRACE: 127.0.0.1:59376 - HTTP connection lost
2025-03-30 18:11:46,721 - httpx - INFO - HTTP Request: GET https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/4f0c246f125fc7594238ebe7beb1435a8335f519/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf "HTTP/1.1 302 Found"
2025-03-30 18:11:46,846 - httpx - INFO - HTTP Request: GET https://cdn-lfs-us-1.hf.co/repos/d6/e9/d6e9318f285870e2a0e3056e22f9c7ec90cd13e14cfde122129ae66af9ad788f/7b064f5842bf9532c91456deda288a1b672397a54fa729aa665952863033557c?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf%3B+filename%3D%22Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf%22%3B&Expires=1743361906&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTc0MzM2MTkwNn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmhmLmNvL3JlcG9zL2Q2L2U5L2Q2ZTkzMThmMjg1ODcwZTJhMGUzMDU2ZTIyZjljN2VjOTBjZDEzZTE0Y2ZkZTEyMjEyOWFlNjZhZjlhZDc4OGYvN2IwNjRmNTg0MmJmOTUzMmM5MTQ1NmRlZGEyODhhMWI2NzIzOTdhNTRmYTcyOWFhNjY1OTUyODYzMDMzNTU3Yz9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=F2IbqoDjFhi3slBzX6usiJWMwNLcC2sYtxgXnJ2omYHwzorfojvkPU-e8fhXeewMadSFm3B93eIpJ54IqZIhpIvlda1hJ3vCwZ2zClaa7GN6LdT1O0836U-HrbLPonfGAQ9gT7BR98BWAtlm2Ea4FdExmad7cg2Cu6IKOPqtAB6JSZoLrpIs9LwEW5x7-GUeQK20YoDTHtBfeDEbHZwYNSQshtZxQygjj6jyrCNd2WOXZf3cM6d7e6cYgCILTpwbr6ClJwxqmP0CDVPJpMkt9-W8UWahcjkWhmIN0S8CsEuIDA3h4l-NQsBTe%7Eoo6r8ETaUm-2fZhD9d7teDwVg21Q__&Key-Pair-Id=K24J24Z295AEI9 "HTTP/1.1 200 OK"
2025-03-30 18:12:12,972 - httpx - INFO - HTTP Request: PUT https://server_name.net/ocs/v1.php/apps/app_api/apps/status/llm2?format=json "HTTP/1.1 200 OK"
TRACE: 127.0.0.1:44812 - HTTP connection made
TRACE: 127.0.0.1:44812 - ASGI [4] Started scope={'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('127.0.0.1', 23001), 'client': ('127.0.0.1', 44812), 'scheme': 'http', 'root_path': '', 'headers': '<...>', 'state': {}, 'method': 'GET', 'path': '/heartbeat', 'raw_path': b'/heartbeat', 'query_string': b''}
TRACE: 127.0.0.1:44812 - ASGI [4] Send {'type': 'http.response.start', 'status': 200, 'headers': '<...>'}
INFO: 127.0.0.1:44812 - "GET /heartbeat HTTP/1.1" 200 OK
TRACE: 127.0.0.1:44812 - ASGI [4] Send {'type': 'http.response.body', 'body': '<15 bytes>'}
TRACE: 127.0.0.1:44812 - ASGI [4] Completed
TRACE: 127.0.0.1:44812 - ASGI [5] Started scope={'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('127.0.0.1', 23001), 'client': ('127.0.0.1', 44812), 'scheme': 'http', 'root_path': '', 'headers': '<...>', 'state': {}, 'method': 'PUT', 'path': '/enabled', 'raw_path': b'/enabled', 'query_string': b'enabled=1'}
2025-03-30 18:12:13,132 - main - INFO - enabled=True
2025-03-30 18:12:13,360 - httpx - INFO - HTTP Request: GET https://server_name.net/ocs/v1.php/cloud/capabilities?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:13,497 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:13,498 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:summary
2025-03-30 18:12:13,631 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:13,632 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:headline
2025-03-30 18:12:13,764 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:13,765 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:topics
2025-03-30 18:12:13,897 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:13,898 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:simplification
2025-03-30 18:12:14,039 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:14,040 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:reformulation
2025-03-30 18:12:14,174 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:14,175 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:contextwrite
2025-03-30 18:12:14,308 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:14,309 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text
2025-03-30 18:12:14,447 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:14,448 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:chat
2025-03-30 18:12:14,583 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:14,584 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:proofread
2025-03-30 18:12:14,722 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:14,723 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:changetone
2025-03-30 18:12:14,858 - httpx - INFO - HTTP Request: POST https://server_name.net/ocs/v1.php/apps/app_api/api/v1/ai_provider/task_processing?format=json "HTTP/1.1 200 OK"
2025-03-30 18:12:14,859 - main - INFO - Registered Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:chatwithtools
TRACE: 127.0.0.1:44812 - ASGI [5] Send {'type': 'http.response.start', 'status': 200, 'headers': '<...>'}
INFO: 127.0.0.1:44812 - "PUT /enabled?enabled=1 HTTP/1.1" 200 OK
TRACE: 127.0.0.1:44812 - ASGI [5] Send {'type': 'http.response.body', 'body': '<12 bytes>'}
TRACE: 127.0.0.1:44812 - ASGI [5] Completed
TRACE: 127.0.0.1:44812 - HTTP connection lost

I have four workers going to get AI Chats going, but doesn't appear to be effective, according to online sources:

#!/bin/sh
echo "Starting Nextcloud AI Worker $1"
cd /usr/share/nginx/nextcloud
sudo -u www-data php occ background-job:worker -t 60 'OC\TaskProcessing\SynchronousBackgroundJob'

[marcelklehr]

Make query, like "How can a cat stop marking inside the house?" Or "What is the population of the US?"

Are you using the "Chat with AI" tab in the assistant UI or the simple "Generate text"? Can you try both, to see if it makes a difference?

[marcelklehr]

2025-03-30 18:20:47,571 - httpx - INFO - HTTP Request: GET https://server_name.net/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content"

This is not en error but merely an info that the app is asking for new tasks but is receiving none from the server.

[corsairius]

2025-03-30 18:20:47,571 - httpx - INFO - HTTP Request: GET https://server_name.net/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content"

This is not en error but merely an info that the app is asking for new tasks but is receiving none from the server.

These messages never changed while querying with a "Chat AI" request, so the error is possibly in sending the Query to the Docker container?

[corsairius]

Make query, like "How can a cat stop marking inside the house?" Or "What is the population of the US?"

Are you using the "Chat with AI" tab in the assistant UI or the simple "Generate text"? Can you try both, to see if it makes a difference?

2025-04-07T19:52:00.277849792Z 2025-04-07 19:52:00,277 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 200 OK" 2025-04-07T19:52:00.278431071Z 2025-04-07 19:52:00,278 - main - INFO - chain: Meta-Llama-3.1-8B-Instruct-Q4_K_M:core:text2text:chat 2025-04-07T19:52:00.723073502Z ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no 2025-04-07T19:52:00.723113720Z ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no 2025-04-07T19:52:00.723126763Z ggml_cuda_init: found 1 CUDA devices: 2025-04-07T19:52:00.723344929Z Device 0: NVIDIA RTX A2000 12GB, compute capability 8.6, VMM: yes 2025-04-07T19:52:06.483142861Z INFO: Started server process [1] 2025-04-07T19:52:06.483213083Z INFO: Waiting for application startup. 2025-04-07T19:52:06.483393905Z TRACE: ASGI [1] Started scope={'type': 'lifespan', 'asgi': {'version': '3.0', 'spec_version': '2.0'}, 'state': {}} 2025-04-07T19:52:06.483435955Z TRACE: ASGI [1] Receive {'type': 'lifespan.startup'} 2025-04-07T19:52:06.663169929Z 2025-04-07 19:52:06,662 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v1.php/apps/app_api/ex-app/state?format=json "HTTP/1.1 200 OK" 2025-04-07T19:52:06.664831461Z TRACE: ASGI [1] Send {'type': 'lifespan.startup.complete'} 2025-04-07T19:52:06.665010498Z INFO: Application startup complete. 2025-04-07T19:52:06.665852772Z INFO: Uvicorn running on http://127.0.0.1:23001 (Press CTRL+C to quit) 2025-04-07T19:52:06.886134630Z 2025-04-07 19:52:06,885 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content" 2025-04-07T19:52:12.069289264Z 2025-04-07 19:52:12,068 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content" 2025-04-07T19:52:17.186327949Z 2025-04-07 19:52:17,185 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content" 2025-04-07T19:52:22.372371367Z 2025-04-07 19:52:22,371 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content" 2025-04-07T19:52:27.557934369Z 2025-04-07 19:52:27,557 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content" 2025-04-07T19:52:32.813629993Z 2025-04-07 19:52:32,813 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content" 2025-04-07T19:52:38.017559536Z 2025-04-07 19:52:38,017 - httpx - INFO - HTTP Request: GET https://xxx/ocs/v2.php/taskprocessing/tasks_provider/next?format=json "HTTP/1.1 204 No Content"

Some new detail, but no output. Still shows execution ten minutes later.

I'll drop down into CPU. Maybe I'm off on using the nVidia A2000 instead of my original nVidia P1000.

It is just odd the the Whisper speech-to-text works.

Removing it all and reverting to CPU mode for LLM2.

No response yet in CPU mode after ten minutes.

[marcelklehr]

These messages never changed while querying with a "Chat AI" request, so the error is possibly in sending the Query to the Docker container?

The ExApp pulls the messages from nextcloud, but yeah, it seems that there is an issue either in asisstant or in your AI config. Can you check the Artificial Intelligence section in the admin settings, and possibly post a screenshot here?

2025-04-07T19:52:00.723344929Z   Device 0: NVIDIA RTX A2000 12GB, compute capability 8.6, VMM: yes
2025-04-07T19:52:06.483142861Z INFO:     Started server process [1] 

This looks like the app crashed for some reason, maybe it is out of memory.

[corsairius]

These messages never changed while querying with a "Chat AI" request, so the error is possibly in sending the Query to the Docker container?

The ExApp pulls the messages from nextcloud, but yeah, it seems that there is an issue either in asisstant or in your AI config. Can you check the Artificial Intelligence section in the admin settings, and possibly post a screenshot here?

2025-04-07T19:52:00.723344929Z   Device 0: NVIDIA RTX A2000 12GB, compute capability 8.6, VMM: yes
2025-04-07T19:52:06.483142861Z INFO:     Started server process [1] 

This looks like the app crashed for some reason, maybe it is out of memory.

Is there a particular process which needs more memory, or is in in the container? The server has 128Gb of memory.

Thanks,
--Joe.

[corsairius]

It appears that the program is certainly dying... This was in the syslog,

Apr 8 21:54:42 nextcloud kernel: [ 5360.708247] traps: python3[118842] trap invalid opcode ip:7f3efd870ec0 sp:7f3f005c6290 error:0 in libggml-cpu.so[7f3efd840000+5d000]
Apr 8 21:54:42 nextcloud dockerd[2986]: time="2025-04-08T21:54:42.638519070-05:00" level=info msg="ignoring event" container=5754c68d52ffc8df13a6b97659c57ff9d228b448677301e970593099e4025240 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Apr 8 21:54:42 nextcloud containerd[2058]: time="2025-04-08T21:54:42.638630250-05:00" level=info msg="shim disconnected" id=5754c68d52ffc8df13a6b97659c57ff9d228b448677301e970593099e4025240 namespace=moby
Apr 8 21:54:42 nextcloud containerd[2058]: time="2025-04-08T21:54:42.638688832-05:00" level=warning msg="cleaning up after shim disconnected" id=5754c68d52ffc8df13a6b97659c57ff9d228b448677301e970593099e4025240 namespace=moby
Apr 8 21:54:42 nextcloud containerd[2058]: time="2025-04-08T21:54:42.638709171-05:00" level=info msg="cleaning up dead shim" namespace=moby
Apr 8 21:54:42 nextcloud systemd[1]: var-lib-docker-overlay2-488c75adb83bb9e74ace9414ca590011018f7c0ad8a11ede42c402fe25ea493a-merged.mount: Succeeded.
Apr 8 21:54:42 nextcloud systemd[3482]: var-lib-docker-overlay2-488c75adb83bb9e74ace9414ca590011018f7c0ad8a11ede42c402fe25ea493a-merged.mount: Succeeded.

The invalid opcode may be for the CPU, but I'm using the GPU. The Xeon on which the server is based does not have AVX instructions. This did, however, work in NC29, and it was moved to NC30 to get Whisper Speech-To-Text to work. The GPU was upgraded to work with larger language models in the 8Gb range.

So why is libggml-cpu.so being called?

Update:
I deleted deployments under AppAPI, and set it to nVidia GPU, and am still receiving this error.

Apr 9 10:40:13 nextcloud kernel: [51291.352766] traps: python3[1445734] trap invalid opcode ip:7f02a0702824 sp:7ffca9d260f8 error:0 in libggml-cpu.so[7f02a0701000+57000

That last update is likely problematic in that I did not recreate the docker image. So when again local to the machine, I will retry the download after removing the docker images.

Yeah, still have that issue with the wrong opcode, and attempting to utilize the CPU instead of the GPU. Is this a basic NextCloud issue?

Apr 9 19:56:52 nextcloud kernel: [84689.535666] traps: python3[2010411] trap invalid opcode ip:7f47e9767ec0 sp:7f47ec5bd290 error:0 in libggml-cpu.so[7f47e9737000+5d000]
Apr 9 19:56:52 nextcloud containerd[2058]: time="2025-04-09T19:56:52.689039057-05:00" level=info msg="shim disconnected" id=a24f5a2486a328a2b56cbd60fbca8b12a3fe1a3497feb7e8e4240f4119eba9fd namespace=moby
Apr 9 19:56:52 nextcloud containerd[2058]: time="2025-04-09T19:56:52.689128005-05:00" level=warning msg="cleaning up after shim disconnected" id=a24f5a2486a328a2b56cbd60fbca8b12a3fe1a3497feb7e8e4240f4119eba9fd namespace=moby
Apr 9 19:56:52 nextcloud containerd[2058]: time="2025-04-09T19:56:52.689164870-05:00" level=info msg="cleaning up dead shim" namespace=moby
Apr 9 19:56:52 nextcloud dockerd[2986]: time="2025-04-09T19:56:52.689192653-05:00" level=info msg="ignoring event" container=a24f5a2486a328a2b56cbd60fbca8b12a3fe1a3497feb7e8e4240f4119eba9fd module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Apr 9 19:56:52 nextcloud systemd[1]: var-lib-docker-overlay2-75940ed407006c096b53df56235ec03784c6530dd7ed342fd22e7f21b75e2e8d-merged.mount: Succeeded.
Apr 9 19:56:52 nextcloud systemd[3482]: var-lib-docker-overlay2-75940ed407006c096b53df56235ec03784c6530dd7ed342fd22e7f21b75e2e8d-merged.mount: Succeeded.

Update 2025-04-10 04:42:
Installed Whisper, and did not receive the libggml-cpu.so python3 trap opcode error. It produced the text from an audio file. The AI Context and AI Chat options did encounter the error. So the GPU was accessed for the audio file and the Chats appeared to request the CPU. That library is called from llama. I also noticed in the logs for the AppAPI resulted in the following:

[marcelklehr]

llm2 requires a CPU with AVX and AVX2 support. This is a requirement from our dependency llama.cpp: ggml-org/llama.cpp#12866

[corsairius]

llm2 requires a CPU with AVX and AVX2 support. This is a requirement from our dependency llama.cpp: ggml-org/llama.cpp#12866

Dell T410:
Operating System: Linux 5.15.0-136-generic x86_64
CPU: Intel(R) Xeon(R) CPU E5649 @ 2.53GHz (24 threads)
Memory: 125.78 GB
Server time:
I'll try a recompile as suggested.

Certainly explains the error, Thanks.

[corsairius]

Looks to be an incoming bug fix which can resolve this.

ggml : add SSE 4.2 variant for CPUs without AVX #12871

Thanks for the assist @marcelklehr. I wasn't there yet.