[BUG-python/deployment] Error with Elastic Search on MacOS 15.2 with M4 Macbook

[louisbrulenaudet]

Describe the bug

Everything was working fine until MacOS version 15.2 was upgraded, rendering Elastic Search unusable. This problem is not simply linked to Argilla, but seems to find another reference in the Corretto library (issue here).

Stacktrace and code to create the bug

2024-12-17 09:03:57 # A fatal error has been detected by the Java Runtime Environment:
2024-12-17 09:03:57 #
2024-12-17 09:03:57 # SIGILL (0x4) at pc=0x0000ffff83f3fb5c, pid=7, tid=16
2024-12-17 09:03:57 #
2024-12-17 09:03:57 # JRE version: (21.0.2+13) (build )
2024-12-17 09:03:57 # Java VM: OpenJDK 64-Bit Server VM (21.0.2+13-58, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, serial gc, linux-aarch64)
2024-12-17 09:03:57 # Problematic frame:
2024-12-17 09:03:57 # j java.lang.System.registerNatives()V+0 java.base@21.0.2
2024-12-17 09:03:57 #
2024-12-17 09:03:57 # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
2024-12-17 09:03:57 #
2024-12-17 09:03:57 # An error report file with more information is saved as:
2024-12-17 09:03:57 # /usr/share/elasticsearch/hs_err_pid7.log
2024-12-17 09:03:57 [0.012s][warning][os] Loading hsdis library failed
2024-12-17 09:03:57 #
2024-12-17 09:03:57 # The crash happened outside the Java Virtual Machine in native code.
2024-12-17 09:03:57 # See problematic frame for where to report the bug.
2024-12-17 09:03:57 #

Expected behavior

Normally the docker container should start without trouble, but with this new version, it shows a java error.

Environment

• Argilla Version [2.5.0]:
• ElasticSearch Version [8.12.2]:

Additional context

No response

[louisbrulenaudet]

The problem can be solved by applying this modification to docker-compose.yaml, then docker-compose up -d:

elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.17.0
    environment:
      - ES_JAVA_OPTS=-Xms512m -Xmx512m -XX:UseSVE=0
      - CLI_JAVA_OPTS=-XX:UseSVE=0
      - node.name=elasticsearch
      - cluster.name=es-argilla-local
      - discovery.type=single-node
      - cluster.routing.allocation.disk.threshold_enabled=false
      - xpack.security.enabled=false
    ulimits:
      memlock:
        soft: -1
        hard: -1
    networks:
      - argilla
    volumes:
      - elasticdata:/usr/share/elasticsearch/data/

It remains to be seen whether this modification will bring about any regression.