Release 3.0.0 Issue - Reading RDB base file on AOF loading...Can't handle RDB format version 12

[KPTheProf]

Current Behavior

After upgrading to release 3.0.0, I've found that Redis Image fails to start with error "Can't handle RDB format version 12".
A recent docker compose pull updated redis:7-alpine to Redis version 7.4.0.
This version is not compatible with the replacement Valkey 8.0.00-rc2 Redis replacement.

Expected Behavior

Valkey 8 does not support the proprietary 7.4.0 Redis database format.

Workaround required or a version that will allow Redis / Valkey to start properly.

Docker Compose Version

Docker Compose version v2.29.2-desktop.2

Docker Version

Client:
 Version:           27.2.0
 API version:       1.47
 Go version:        go1.21.13
 Git commit:        3ab4256
 Built:             Tue Aug 27 14:17:17 2024
 OS/Arch:           windows/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.34.0 (165256)
 Engine:
  Version:          27.2.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.21.13
  Git commit:       3ab5c7d
  Built:            Tue Aug 27 14:15:15 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.20
  GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

The git Revision

ab2a7bc

The git Status

On branch release
Your branch is up to date with 'origin/release'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   configuration/plugins.py
        modified:   env/netbox.env

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        Dockerfile-Plugins
        Netbox.yml
        plugin_requirements.txt
        ssl/

no changes added to commit (use "git add" and/or "git commit -a")

Startup Command

docker compose up --detach

NetBox Logs

netbox-1  | ÔÜÖ´©Å Applying database migrations
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/configuration.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/extra.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/jinjas_filters.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/logging.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/plugins.py'
netbox-1  | Operations to perform:
netbox-1  |   Apply all migrations: account, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, netbox_dns, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
netbox-1  | Running migrations:
netbox-1  |   Applying netbox_dns.0008_view_prefixes... OK
netbox-1  |   Applying netbox_dns.0009_rename_contact_registrationcontact... OK
netbox-1  | ÔÜÖ´©Å Running trace_paths
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/configuration.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/extra.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/jinjas_filters.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/logging.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/plugins.py'
netbox-1  | Found no missing console port paths; skipping
netbox-1  | Found no missing console server port paths; skipping
netbox-1  | Found no missing interface paths; skipping
netbox-1  | Found no missing power feed paths; skipping
netbox-1  | Found no missing power outlet paths; skipping
netbox-1  | Found no missing power port paths; skipping
netbox-1  | Finished.
netbox-1  | ÔÜÖ´©Å Removing stale content types
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/configuration.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/extra.py'
netbox-1  | Traceback (most recent call last):
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django_redis/cache.py", line 29, in _decorator
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/jinjas_filters.py'
netbox-1  |     return method(self, *args, **kwargs)
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/logging.py'
netbox-1  | ­ƒº¼ loaded config '/etc/netbox/config/plugins.py'
netbox-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django_redis/cache.py", line 99, in _get
netbox-1  |     return self.client.get(key, default=default, version=version, client=client)
netbox-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django_redis/client/default.py", line 260, in get
netbox-1  |     raise ConnectionInterrupted(connection=client) from e
netbox-1  | django_redis.exceptions.ConnectionInterrupted: Redis ConnectionError: Error -2 connecting to redis-cache:6379. Name or service not known.
netbox-1  |
netbox-1  | During handling of the above exception, another exception occurred:
netbox-1  |
netbox-1  | Traceback (most recent call last):
netbox-1  |   File "/opt/netbox/netbox/./manage.py", line 10, in <module>
netbox-1  |     execute_from_command_line(sys.argv)
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
netbox-1  |     utility.execute()
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
netbox-1  |     self.fetch_command(subcommand).run_from_argv(self.argv)
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
netbox-1  |     self.execute(*args, **cmd_options)
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
netbox-1  |     output = self.handle(*args, **options)
netbox-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/contrib/contenttypes/management/commands/remove_stale_contenttypes.py", line 101, in handle
netbox-1  |     ct.delete()
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/base.py", line 1183, in delete
netbox-1  |     return collector.delete()
netbox-1  |            ^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/deletion.py", line 463, in delete
netbox-1  |     signals.pre_delete.send(
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django/dispatch/dispatcher.py", line 189, in send
netbox-1  |     response = receiver(signal=self, sender=sender, **named)
netbox-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/netbox/extras/signals.py", line 128, in handle_deleted_object
netbox-1  |     validators = get_config().PROTECTION_RULES.get(model_name, [])
netbox-1  |                  ^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/netbox/netbox/config/__init__.py", line 28, in get_config
netbox-1  |     _thread_locals.config = Config()
netbox-1  |                             ^^^^^^^^
netbox-1  |   File "/opt/netbox/netbox/netbox/config/__init__.py", line 48, in __init__
netbox-1  |     self._populate_from_cache()
netbox-1  |   File "/opt/netbox/netbox/netbox/config/__init__.py", line 71, in _populate_from_cache
netbox-1  |     self.config = cache.get('config') or {}
netbox-1  |                   ^^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django_redis/cache.py", line 92, in get
netbox-1  |     value = self._get(key, default, version, client)
netbox-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django_redis/cache.py", line 36, in _decorator
netbox-1  |     raise e.__cause__
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/django_redis/client/default.py", line 258, in get
netbox-1  |     value = client.get(key)
netbox-1  |             ^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/redis/commands/core.py", line 1821, in get
netbox-1  |     return self.execute_command("GET", name)
netbox-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/redis/client.py", line 545, in execute_command
netbox-1  |     conn = self.connection or pool.get_connection(command_name, **options)
netbox-1  |                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/redis/connection.py", line 1074, in get_connection
netbox-1  |     connection.connect()
netbox-1  |   File "/opt/netbox/venv/lib/python3.11/site-packages/redis/connection.py", line 283, in connect
netbox-1  |     raise ConnectionError(self._error_message(e))
netbox-1  | redis.exceptions.ConnectionError: Error -2 connecting to redis-cache:6379. Name or service not known.

Content of docker-compose.override.yml

services:
  netbox:
    image: netbox:latest-plugins
    ports:
      - "8000:8080"
    # If you want the Nginx unit status page visible from the
    # outside of the container add the following port mapping:
    # - "8001:8081"
    healthcheck:
      # Time for which the health check can fail after the container is started.
      # This depends mostly on the performance of your database. On the first start,
      # when all tables need to be created the start_period should be higher than on
      # subsequent starts. For the first start after major version upgrades of NetBox
      # the start_period might also need to be set higher.
      # Default value in our docker-compose.yml is 60s
      start_period: 120s
    # environment:
      # SKIP_SUPERUSER: "false"
      # SUPERUSER_API_TOKEN: ""
      # SUPERUSER_EMAIL: ""
      # SUPERUSER_NAME: ""
      # SUPERUSER_PASSWORD: ""
    build:
      context: .
      dockerfile: Dockerfile-Plugins
    restart: unless-stopped

  netbox-worker:
    image: netbox:latest-plugins
    build:
      context: .
      dockerfile: Dockerfile-Plugins
    restart: unless-stopped

  netbox-housekeeping:
    image: netbox:latest-plugins
    build:
      context: .
      dockerfile: Dockerfile-Plugins
    restart: unless-stopped

  postgres:
    restart: unless-stopped

  redis:
    restart: unless-stopped

  redis-cache:
    restart: unless-stopped

  tls:
    image: caddy:2-alpine
    depends_on:
      - netbox
    volumes:
      - ./ssl/localhost+2.pem:/etc/ssl/private/localhost.crt:ro,z
      - ./ssl/localhost+2-key.pem:/etc/ssl/private/localhost.key:ro,z
      - ./ssl/Caddyfile:/etc/caddy/Caddyfile:ro
    ports:
      - 80:80 # Allows for http redirection
      - 443:443
    restart: unless-stopped

same here... This error message indicates that is attempting to load an RDB file that is in a newer or incompatible format (version 12) that is not supported by the currently running version (7.9.240).

1:C 04 Sep 2024 13:34:19.489 * oO0OoO0OoO0Oo Valkey is starting oO0OoO0OoO0Oo
redis-1 | 1:C 04 Sep 2024 13:34:19.489 * Valkey version=7.9.240, bits=64, commit=00000000, modified=0, pid=1, just started
redis-1 | 1:C 04 Sep 2024 13:34:19.489 * Configuration loaded
redis-1 | 1:M 04 Sep 2024 13:34:19.490 * monotonic clock: POSIX clock_gettime
redis-1 | 1:M 04 Sep 2024 13:34:19.490 * Running mode=standalone, port=6379.
redis-1 | 1:M 04 Sep 2024 13:34:19.491 * Server initialized
redis-1 | 1:M 04 Sep 2024 13:34:19.492 * Reading RDB base file on AOF loading...
redis-1 | 1:M 04 Sep 2024 13:34:19.492 # Can't handle RDB format version 12
redis-1 | 1:M 04 Sep 2024 13:34:19.492 # Error reading the RDB base file appendonly.aof.48.base.rdb, AOF loading aborted

[KPTheProf]

Hi @AnID-BW
The core data is in the Postgres database, so I'm not sure if this is transient data that gets rebuilt from the Postgres data.
Whenever I've moved data between different instances of Netbox, I've only copied a dump of the Postgres database and I've not experienced any issues.

To work around the issue I first did a "docker compose down" and then deleted the netbox-redis-data and netbox-redis-cache-data volumes. These are recreated after the "docker compose up".

I was then able to get Netbox running.

I have a separate issue where I seem to be getting Netbox 4.0.11 instead of 4.1, but I'm unsure if these issues are related.
#1305