[bitnami/mongodb] libos.sh: line 344 error when installing ReplicaSet

[RobertFent]

Name and Version

bitnami/mongodb 13.1.7

What steps will reproduce the bug?

basic cluster setup

• Kubernetes version: v1.25.2
• Helm version: v3.9.4
• Cluster setup with kubeadm
• CNI: Weavenet
• Loadbalancer: metallb
• 2 nodes: master and worker

install mongodb with helm

helm install mongodb bitnami/mongodb --set architecture="replicaset",auth.rootPassword=password123

errors

-> Pods not starting

$ kubectl get po
NAME                                  READY   STATUS    RESTARTS      AGE
mongodb-0                             0/1     Pending   0             19s
mongodb-arbiter-0                     0/1     Error     2 (17s ago)   19s
mydenowebsite-chart-b8b74db59-2tgsd   1/1     Running   0             50m
mydenowebsite-chart-b8b74db59-zgtlf   1/1     Running   0             50m
nginx-7965ccc856-98tn4                1/1     Running   0             22h
nginx-7965ccc856-pfq4v                1/1     Running   0             22h

$ kubectl logs mongodb-arbiter-0
mongodb 10:22:58.00
mongodb 10:22:58.01 Welcome to the Bitnami mongodb container
mongodb 10:22:58.01 Subscribe to project updates by watching https://github.com/bitnami/containers
mongodb 10:22:58.02 Submit issues and feature requests at https://github.com/bitnami/containers/issues
mongodb 10:22:58.03
mongodb 10:22:58.03 INFO  ==> ** Starting MongoDB setup **
mongodb 10:22:58.06 INFO  ==> Validating settings in MONGODB_* env vars...
mongodb 10:22:58.37 INFO  ==> Initializing MongoDB...
mongodb 10:22:58.40 INFO  ==> Deploying MongoDB from scratch...
/opt/bitnami/scripts/libos.sh: line 344:    42 Illegal instruction     (core dumped) "$@" > /dev/null 2>&1

Are you using any custom parameters or values?

--set architecture="replicaset",auth.rootPassword=password123

What is the expected behavior?

MongoDB Pods should start with 2 Replicas without any errors

What do you see instead?

MongoDB Pods not starting because of an error in arbiter:

/opt/bitnami/scripts/libos.sh: line 344:    42 Illegal instruction     (core dumped) "$@" > /dev/null 2>&1

[rafariossaa]

Hi,
Could you indicate on witch cpu architecture are you deploying mongodb to ?

[RobertFent]

Hello,
this is the output from my ubuntu server:

❯ uname -a
Linux $HOSTNAME 5.15.0-48-generic #54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Its a 64 bit cpu

[rafariossaa]

Hi,
That indicates which was the architecture of the kernel compiled for.
Could you run: cat /proc/cpuinfo ?

[Aib0t]

Same issue, output of cat /proc/cpuinfo :

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 122
model name      : Intel(R) Celeron(R) N4100 CPU @ 1.10GHz
stepping        : 1
microcode       : 0x2e
cpu MHz         : 2044.107
cache size      : 4096 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 24
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg cx16 xtpr pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave rdrand lahf_lm 3dnowprefetch cpuid_fault cat_l2 pti cdp_l2 ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust smep erms mpx rdt_a rdseed smap clflushopt intel_pt sha_ni xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts umip rdpid md_clear arch_capabilities
vmx flags       : vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs ept_mode_based_exec tsc_scaling
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 2188.80
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

[rafariossaa]

In think that is the issue here, mongodb requires avx instructions and your processor lack that instruction set as you could see in the flags section of cpuinfo

[RobertFent]

Output of cat /proc/cpuinfo:

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 13
model name	: QEMU Virtual CPU version 2.5+
stepping	: 3
microcode	: 0x1
cpu MHz		: 2294.568
cache size	: 16384 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 ht syscall nx lm rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt aes xsave rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti ssbd ibrs ibpb invpcid md_clear arch_capabilities
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 4589.13
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

My cpu also does not support avx instructions. Is there anything I can do?

[rafariossaa]

I think there is not, but maybe in qemu there is something to emulate those instruction.
I would suggest you to ask in qemu and mongodb forums on this regard.

[RobertFent]

Okay, thank you for your help!

[mulder999]

Had same issue with my old Xeon that does not have AVX. Since AVX support was introduced in v5, going back to v4 solved the issue.

[GermanAizek]

@mulder999 @RobertFent,
a more correct option is to rebuild mongodb with patch without experimental optimizations for sandybridge CPU's
https://github.com/GermanAizek/mongodb-without-avx/

[PiotrKuligowski]

I've run to the same issue having mongodb in my k3s cluster, running in a proxmox VM.
I checked if my processor have avx flag, using lscpu | grep avx. It had.
But when running cat /proc/cpuinfo inside some pod, there was not avx flag for my cpu.

Fixed it by changing processor type in VM from qemu64 to host and had to reboot all nodes.

[alejandro-galassi]

I've run to the same issue having mongodb in my k3s cluster, running in a proxmox VM. I checked if my processor have avx flag, using lscpu | grep avx. It had. But when running cat /proc/cpuinfo inside some pod, there was not avx flag for my cpu.

Fixed it by changing processor type in VM from qemu64 to host and had to reboot all nodes.

Had the same issue and was almost a month of troubleshooting.... finally got this thread and got your solution.. thank you!!

[0x1def]

Hi,
we compile binaries and build images without AVX requirement for our needs.
If those artifacts suit you, give it a try.
https://github.com/flakybitnet/mongodb-docker