[Bug]: edge container giving SQLite error on startup #3231

ic1415 · 2024-08-04T16:53:51Z

What happened?

When using the following compose file, I am getting an SQLite error at startup:

services:
  audiobookshelf:
    image: ghcr.io/advplyr/audiobookshelf:edge
    container_name: audiobookshelf
    networks:
      proxy:
    ports:
      - 13378:80
    volumes:
      - /path/to/audiobookshelf:/books
      - /path/to/audio:/audiobooks
      - /path/to/Docker/AudioBookshelf/config:/config
      - /path/to/Docker/AudioBookshelf/meta:/metadata
     #- /var/run/docker.sock:/var/run/docker.sock:ro # optional, for docker integrations
    environment:
      - TZ=America/New_York
      - DOCKER_HOST=socket-proxy
    restart: always
  #logging service pushing to loki
  promtail:
    image: grafana/promtail
    container_name: abs-promtail
    hostname: abs-promtail
    restart: unless-stopped
    volumes:
      - /path/to/Docker/AudioBookshelf/meta/logs/daily:/var/log/abs:ro
      - /path/to/Docker/loki/promtail-config.yml:/etc/promtail-config.yml
    command:
      - '-config.file=/etc/promtail-config.yml'
networks:
  proxy:
    external: true

What did you expect to happen?

Expected that ABS server would start

Steps to reproduce the issue

use above compose file to build edge container
see errors in log

Audiobookshelf version

edge

How are you running audiobookshelf?

Docker

What OS is your Audiobookshelf server hosted from?

Linux

If the issue is being seen in the UI, what browsers are you seeing the problem on?

None

Logs

Config /config /metadata
[2024-08-04 12:52:57.196] INFO: === Starting Server ===
[2024-08-04 12:52:57.197] INFO: [Server] Init v2.11.0
[2024-08-04 12:52:57.198] INFO: [Server] Node.js Version: v20.16.0
[2024-08-04 12:52:57.278] INFO: [BinaryManager] Found valid binary ffmpeg at /ffmpeg
[2024-08-04 12:52:57.278] INFO: [BinaryManager] Updating process.env.FFMPEG_PATH
[2024-08-04 12:52:57.337] INFO: [BinaryManager] Found valid binary ffprobe at /ffprobe
[2024-08-04 12:52:57.338] INFO: [BinaryManager] Updating process.env.FFPROBE_PATH
[2024-08-04 12:52:57.338] INFO: [BinaryManager] Found valid binary unicode at /unicode.so
[2024-08-04 12:52:57.338] INFO: [BinaryManager] Updating process.env.SQLEAN_UNICODE_PATH
[2024-08-04 12:52:57.339] INFO: [Database] Initializing db at "/config/absdatabase.sqlite"
[2024-08-04 12:52:57.377] INFO: [Database] Loading extension /unicode.so
[2024-08-04 12:52:57.377] ERROR: [Database] Failed to load extension /unicode.so [Error: SQLITE_ERROR: Error loading shared library /unicode.so.so: No such file or directory] {
  errno: 1,
  code: 'SQLITE_ERROR'
} (Database.js:236)
[2024-08-04 12:52:57.381] ERROR: [Database] Failed to connect to db [Error: SQLITE_ERROR: Error loading shared library /unicode.so.so: No such file or directory] {
  errno: 1,
  code: 'SQLITE_ERROR'
} (Database.js:214)
[2024-08-04 12:52:57.382] FATAL: [Server] Unhandled rejection: Error: Database connection failed, promise: Promise {
  <rejected> Error: Database connection failed
      at Database.init (/server/Database.js:168:13)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Server.init (/server/Server.js:113:5)
      at async Server.start (/server/Server.js:176:5)
} (Server.js:168)

Additional Notes

No response

The text was updated successfully, but these errors were encountered:

mikiher · 2024-08-04T17:58:26Z

OK, can I ask you to:

open a shell on the container by running the following command:
docker exec -it audiobookshelf /bin/sh
find out if the unicode.so is in the container's root directory, and with the proper permissions, by running:
ls -la /unicode.so
exit the shell by pressing ctrl+d

Please paste the result of the ls command here.

Also, which Linux is your docker running on?

ic1415 · 2024-08-04T19:02:17Z

I can not open a shell in the container. I get:

Error response from the daemon: Container 6cf... is restarting, wait until the container is running

running on raspberry pi OS (Debian bookworm)

katertier · 2024-08-04T19:59:05Z

it's not there, but that's also true for current?

edge:
/ # find . -type f -name "unicode.so"
/ # find . -type f -name "libssl.so.3"
./lib/libssl.so.3

current:
/ # find . -type f -name "unicode.so"
/ # find . -type f -name "libssl.so.3"
./lib/libssl.so.3

mikiher · 2024-08-04T20:14:10Z

it's not there, but that's also true for current?

It's a new binary that's downloaded and installed by Audiobokshelf itself. It's currently only in edge.

I can not open a shell in the container

Ah, sorry - that figures, since your container crashes.
is your raspberry pi 32-bit or 64-bit?

advplyr · 2024-08-04T21:11:22Z

The double file extension in the error seems unusual SQLITE_ERROR: Error loading shared library /unicode.so.so.

mikiher · 2024-08-04T21:25:33Z

My hypothesis is that the file /unicode.so is there (we know it was downloaded successfully because the log says so), but sqlite is unable to load it, and so it tries to add another .so to the name as an alternate path (I know it has this logic), and when that fails, it throws the error.

My suspicion is that the reason it unable to load /unicode.so is becuase it is an armhf OS (32-bit), and the sqlean binaries are only available for arm64. @advplyr do we officially support 32-bit systems? If this is indeed the reason, and if we need to support armhf, I might have to either:

make the unicode extension optional
compile the extension for armhf from source

Since 2 might take a while to achieve, probably I'll need to implement 1.

But before we do all that, let's first hear from @ic1415

advplyr · 2024-08-04T21:35:03Z

We don't officially support 32-bit. That was dropped in v2.8.0 https://github.com/advplyr/audiobookshelf/releases/tag/v2.8.0

advplyr · 2024-08-04T21:37:24Z

If this is a broader issue we'll find out soon because I'm pushing the v2.12.0 release

katertier · 2024-08-04T21:40:32Z

I see the same error on MacOS X, iMac M4, hence I don't think it's caused by a 32bit issue but it might be arm related...

advplyr · 2024-08-04T21:47:32Z

@katertier Are you sure it is the same error? I just tested on mac and found a separate crash on startup due to a bug with dylib

mikiher · 2024-08-04T21:50:19Z

macOS arm64 is supposed to be supported by SQLean (the binaries exist, at least). Can you please add your crash log to the bug?

advplyr · 2024-08-04T21:52:09Z

It works now with that fix for me. I've now tested on all the platforms so I'm going to go ahead with this release

katertier · 2024-08-04T21:52:22Z

I'm pretty sure... :)

2024-08-04 23:50:02 [2024-08-04 21:50:02.630] INFO: === Starting Server ===
2024-08-04 23:50:02 [2024-08-04 21:50:02.634] INFO: [Server] Init v2.11.0
2024-08-04 23:50:02 [2024-08-04 21:50:02.635] INFO: [Server] Node.js Version: v20.16.0
2024-08-04 23:50:02 [2024-08-04 21:50:02.748] INFO: [BinaryManager] Found valid binary ffmpeg at /ffmpeg
2024-08-04 23:50:02 [2024-08-04 21:50:02.749] INFO: [BinaryManager] Updating process.env.FFMPEG_PATH
2024-08-04 23:50:02 [2024-08-04 21:50:02.776] INFO: [BinaryManager] Found valid binary ffprobe at /ffprobe
2024-08-04 23:50:02 [2024-08-04 21:50:02.776] INFO: [BinaryManager] Updating process.env.FFPROBE_PATH
2024-08-04 23:50:02 [2024-08-04 21:50:02.776] INFO: [BinaryManager] Found valid binary unicode at /unicode.so
2024-08-04 23:50:02 [2024-08-04 21:50:02.776] INFO: [BinaryManager] Updating process.env.SQLEAN_UNICODE_PATH
2024-08-04 23:50:02 [2024-08-04 21:50:02.777] INFO: [Database] Initializing db at "/config/absdatabase.sqlite"
2024-08-04 23:50:02 [2024-08-04 21:50:02.802] INFO: [Database] Loading extension /unicode.so
2024-08-04 23:50:02 [2024-08-04 21:50:02.802] ERROR: [Database] Failed to load extension /unicode.so [Error: SQLITE_ERROR: Error loading shared library /unicode.so.so: No such file or directory] {
2024-08-04 23:50:02 errno: 1,
2024-08-04 23:50:02 code: 'SQLITE_ERROR'
2024-08-04 23:50:02 } (Database.js:236)
2024-08-04 23:50:02 [2024-08-04 21:50:02.803] ERROR: [Database] Failed to connect to db [Error: SQLITE_ERROR: Error loading shared library /unicode.so.so: No such file or directory] {
2024-08-04 23:50:02 errno: 1,
2024-08-04 23:50:02 code: 'SQLITE_ERROR'
2024-08-04 23:50:02 } (Database.js:214)
2024-08-04 23:50:02 [2024-08-04 21:50:02.804] FATAL: [Server] Unhandled rejection: Error: Database connection failed, promise: Promise {
2024-08-04 23:50:02 Error: Database connection failed
2024-08-04 23:50:02 at Database.init (/server/Database.js:168:13)
2024-08-04 23:50:02 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-08-04 23:50:02 at async Server.init (/server/Server.js:114:5)
2024-08-04 23:50:02 at async Server.start (/server/Server.js:177:5)
2024-08-04 23:50:02 } (Server.js:169)

katertier · 2024-08-04T21:57:38Z

@advplyr: Thanks! A bit too late for me (midnight is approaching where I am), but I'll check first thing in the morning.

advplyr · 2024-08-04T22:05:55Z

My fix isn't going to fix that error. My mac downloads the .dylib because process.platform output is darwin. Not sure why that wouldn't be the case for your mac

mikiher · 2024-08-04T22:06:20Z

Ah, sorry, and thanks for fixing that!

it would be good to add the following lines after [Server] Node.js Version

    Logger.info('[Server] Platform:', process.platform)
    Logger.info('[Server] Arch:', process.arch)

I believe docker containers on macOS run in a lightweight linux virtual machine, similar to Windows. that's why @katertier's BinaryManager will still download the .so version (and hopefully the arm64 version).

vutsalsinghal · 2024-08-04T23:44:08Z

My hypothesis is that the file /unicode.so is there (we know it was downloaded successfully because the log says so), but sqlite is unable to load it, and so it tries to add another .so to the name as an alternate path (I know it has this logic), and when that fails, it throws the error.

My suspicion is that the reason it unable to load /unicode.so is becuase it is an armhf OS (32-bit), and the sqlean binaries are only available for arm64. @advplyr do we officially support 32-bit systems? If this is indeed the reason, and if we need to support armhf, I might have to either:

make the unicode extension optional

compile the extension for armhf from source

Since 2 might take a while to achieve, probably I'll need to implement 1.

But before we do all that, let's first hear from @ic1415

Same error on 64bit raspberry pi

> uname -a
Linux <hostname> 6.5.0-1020-raspi #23-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun 24 13:20:44 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

I've reverted to v2.11.0 for now

ic1415 · 2024-08-05T00:05:02Z

My hypothesis is that the file /unicode.so is there (we know it was downloaded successfully because the log says so), but sqlite is unable to load it, and so it tries to add another .so to the name as an alternate path (I know it has this logic), and when that fails, it throws the error.

My suspicion is that the reason it unable to load /unicode.so is becuase it is an armhf OS (32-bit), and the sqlean binaries are only available for arm64. @advplyr do we officially support 32-bit systems? If this is indeed the reason, and if we need to support armhf, I might have to either:

make the unicode extension optional

compile the extension for armhf from source

Since 2 might take a while to achieve, probably I'll need to implement 1.

But before we do all that, let's first hear from @ic1415

Catching up, but it is arm64 so no compatibility issues there

Edit: attempted to install 2.12.0 and received the same error. Rolled back to 2.11 for the time being.

advplyr · 2024-08-05T00:24:52Z

Can you share the new logs that were added below "Node.js version" when starting the server? That gives us the variables that are being used to decide which binaries to use

ic1415 · 2024-08-05T00:36:30Z

[2024-08-04 20:30:01.283] INFO: === Starting Server ===
[2024-08-04 20:30:01.287] INFO: [Server] Init v2.12.0
[2024-08-04 20:30:01.288] INFO: [Server] Node.js Version: v20.16.0
[2024-08-04 20:30:01.289] INFO: [Server] Platform: linux
[2024-08-04 20:30:01.289] INFO: [Server] Arch: arm64

mikiher · 2024-08-05T00:36:42Z

Ok, thank you all.

So to summarize what we know at this point, it looks like on linux-arm64, BinaryManager seems to have downloaded some version of unicode.so, but SQLite is unable to load that library.

I'll continue to investigate. In the the meantime, my recommendation would be of course to stay with release 2.11.0 if your system is running on Linux arm64.

Sorry for the hassle.

2fst4u · 2024-08-05T01:08:43Z

Same issue for me on arm64. I have a mixed cluster so I have pinned audiobookshelf to one of my x86 nodes and it started fine so it definitely seems isolated to arm64.

ScuttleSE · 2024-08-05T05:25:26Z

Same here, broken om arm64...

JBaby9783 · 2024-08-05T05:40:04Z

I don't know anything about this error, but I installed 2.12.0 on my M4 iMac and I couldn't access my server. I'm back on 2.11.0.

mikiher · 2024-08-05T09:07:59Z

OK, an update: I submitted a fix for review. Hopefully it will be merged (and released) later today.

devnoname120 · 2024-08-05T10:44:56Z

Same issue on Asahi Linux (aarch64):

Machou · 2024-08-05T11:31:36Z

Same Here :)

uname -a
Linux debian 6.1.0-23-arm64 #1 SMP Debian 6.1.99-1 (2024-07-15) aarch64 GNU/Linux

devnoname120 · 2024-08-05T11:40:53Z

See this PR for a proper fix: #3236

github-actions · 2024-08-05T14:35:13Z

Fixed in v2.12.1.

ic1415 added the bug Something isn't working label Aug 4, 2024

advplyr added a commit that referenced this issue Aug 4, 2024

Fix:Binary manager dylib file ext check #3231

06eaee8

advplyr added a commit that referenced this issue Aug 4, 2024

Add server log for process.platform and process.arch #3231

8ff7b6b

This was referenced Aug 5, 2024

Temporariliy disable sqlite3 unicode extension on linux-arm64 #3235

Closed

[Bug]: audiobookshelf didn't laucn on arm64 with 2.12 #3234

Closed

devnoname120 mentioned this issue Aug 5, 2024

Fix unicode.so loading #3236

Merged

nichwall mentioned this issue Aug 5, 2024

v2.12.0 broken on Arm64 (using docker) #3238

Closed

advplyr closed this as completed in 20eb573 Aug 5, 2024

advplyr closed this as completed in #3236 Aug 5, 2024

advplyr added the awaiting release Issue is resolved and will be in the next release label Aug 5, 2024

github-actions bot removed the awaiting release Issue is resolved and will be in the next release label Aug 5, 2024

ZLoth mentioned this issue Aug 7, 2024

[Enhancement]: Changelog should contain all dot-release notes in a major release #3250

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: edge container giving SQLite error on startup #3231

[Bug]: edge container giving SQLite error on startup #3231

ic1415 commented Aug 4, 2024 •

edited

Loading

mikiher commented Aug 4, 2024

ic1415 commented Aug 4, 2024

katertier commented Aug 4, 2024

mikiher commented Aug 4, 2024 •

edited

Loading

advplyr commented Aug 4, 2024

mikiher commented Aug 4, 2024 •

edited

Loading

advplyr commented Aug 4, 2024

advplyr commented Aug 4, 2024 •

edited

Loading

katertier commented Aug 4, 2024

advplyr commented Aug 4, 2024

mikiher commented Aug 4, 2024

advplyr commented Aug 4, 2024

katertier commented Aug 4, 2024

katertier commented Aug 4, 2024

advplyr commented Aug 4, 2024

mikiher commented Aug 4, 2024 •

edited

Loading

vutsalsinghal commented Aug 4, 2024 •

edited

Loading

ic1415 commented Aug 5, 2024 •

edited

Loading

advplyr commented Aug 5, 2024

ic1415 commented Aug 5, 2024 •

edited

Loading

mikiher commented Aug 5, 2024

2fst4u commented Aug 5, 2024

ScuttleSE commented Aug 5, 2024

JBaby9783 commented Aug 5, 2024

mikiher commented Aug 5, 2024

devnoname120 commented Aug 5, 2024

Machou commented Aug 5, 2024

devnoname120 commented Aug 5, 2024 •

edited

Loading

github-actions bot commented Aug 5, 2024

[Bug]: edge container giving SQLite error on startup #3231

[Bug]: edge container giving SQLite error on startup #3231

Comments

ic1415 commented Aug 4, 2024 • edited Loading

What happened?

What did you expect to happen?

Steps to reproduce the issue

Audiobookshelf version

How are you running audiobookshelf?

What OS is your Audiobookshelf server hosted from?

If the issue is being seen in the UI, what browsers are you seeing the problem on?

Logs

Additional Notes

mikiher commented Aug 4, 2024

ic1415 commented Aug 4, 2024

katertier commented Aug 4, 2024

mikiher commented Aug 4, 2024 • edited Loading

advplyr commented Aug 4, 2024

mikiher commented Aug 4, 2024 • edited Loading

advplyr commented Aug 4, 2024

advplyr commented Aug 4, 2024 • edited Loading

katertier commented Aug 4, 2024

advplyr commented Aug 4, 2024

mikiher commented Aug 4, 2024

advplyr commented Aug 4, 2024

katertier commented Aug 4, 2024

katertier commented Aug 4, 2024

advplyr commented Aug 4, 2024

mikiher commented Aug 4, 2024 • edited Loading

vutsalsinghal commented Aug 4, 2024 • edited Loading

ic1415 commented Aug 5, 2024 • edited Loading

advplyr commented Aug 5, 2024

ic1415 commented Aug 5, 2024 • edited Loading

mikiher commented Aug 5, 2024

2fst4u commented Aug 5, 2024

ScuttleSE commented Aug 5, 2024

JBaby9783 commented Aug 5, 2024

mikiher commented Aug 5, 2024

devnoname120 commented Aug 5, 2024

Machou commented Aug 5, 2024

devnoname120 commented Aug 5, 2024 • edited Loading

github-actions bot commented Aug 5, 2024

ic1415 commented Aug 4, 2024 •

edited

Loading

mikiher commented Aug 4, 2024 •

edited

Loading

mikiher commented Aug 4, 2024 •

edited

Loading

advplyr commented Aug 4, 2024 •

edited

Loading

mikiher commented Aug 4, 2024 •

edited

Loading

vutsalsinghal commented Aug 4, 2024 •

edited

Loading

ic1415 commented Aug 5, 2024 •

edited

Loading

ic1415 commented Aug 5, 2024 •

edited

Loading

devnoname120 commented Aug 5, 2024 •

edited

Loading