Certain node.js/amd64 workloads cause container to become unresponsive/100% CPU when using Rosetta

[rouge8]

Description

limactl version 0.19.1, macOS 14.2.1, using Colima 0.6.7 with Rosetta enabled.

On one of our internal projects, an amd64 Docker build seems to hang forever when Rosetta is enabled. I'm also able to reproduce it with the Dockerfile in docker/for-mac#6998, by running docker run --platform linux/amd64 jbinto/rosetta-what. This feels different from #1609 because the VM itself isn't unresposive; I'm able to SSH into the VM and run htop and see that /mnt/lima-rosetta/rosetta /usr/local/bin/node index.js is using 100% CPU.

[AkihiroSuda]

This should be probably reported to Apple

[rouge8]

Interestingly it appears to have been fixed in Docker for Mac so they must have found some workaround...

[Perdjesk]

Observing this issue with a VM type VZ and Rosetta emulation enabled (through colima) and Nextjs build using jest-worker. The release notes of Docker Desktop 4.27.2 (released 2024-02-08) mentions:

Node now uses all the cores available under Rosetta.

Similar hints here: docker/for-mac#7184

This seems to go into the direction that it is a configuration or change at level of the Linux virtual machine

[balajiv113]

From the hints i could see that we could wrap rosetta command with taskset to avoid this issue

I wrote a simple c program to delegate call to rosetta with taskset -c 1 set. Custom program works when calling directly & via binfmt in guest, but this c program is not working when used with binfmt & docker.

[Perdjesk]

Here is case that hopefully is reproducible.

Host is Apple Silicon with Rosetta enabled.

% sw_vers
ProductName:            macOS
ProductVersion:         14.3.1
BuildVersion:           23D60
% arch
arm64
% arch -arch x86_64 uname -m
x86_64

Setup the sample project for NextJS with a Dockerfile.

% nvm use 20
Now using node v20.11.1 (npm v10.2.4)
% node --version
v20.11.1
% npx create-next-app --example with-docker nextjs-docker
% cd nextjs-docker

Setup two VZ VMs one with qemu and one with rosetta emulation.

% colima --version
colima version 0.6.8
% colima -p vz-qemu start --arch aarch64 --cpu 8 --memory 12 --vm-type=vz
% colima -p vz-rosetta start --arch aarch64 --cpu 8 --memory 12 --vm-type=vz --vz-rosetta

Build using VZ with qemu works:

% docker context use colima-vz-qemu
colima-vz-qemu
Current context is now "colima-vz-qemu"

% time docker build --platform linux/amd64  .
...
docker build --platform linux/amd64 .  0.28s user 0.24s system 0% cpu 2:41.29 total

Build using VZ with rosetta hangs.

% docker context use colima-vz-rosetta
colima-vz-rosetta
Current context is now "colima-vz-rosetta"

Hangs on

 Creating an optimized production build ...

The following process is at 100% CPU:

/mnt/lima-rosetta/rosetta /usr/local/bin/node /app/node_modules/next/dist/compiled/jest-worker/processChild.js

From VM

$ ps -ef | grep rosetta
root        2648    2636  0 14:52 ?        00:00:01 /mnt/lima-rosetta/rosetta /usr/local/bin/node /usr/local/bin/npm run build
root        2934    2648  0 14:52 ?        00:00:02 /mnt/lima-rosetta/rosetta /usr/local/bin/node /app/node_modules/.bin/next build
root        2970    2934 99 14:52 ?        00:09:13 /mnt/lima-rosetta/rosetta /usr/local/bin/node /app/node_modules/next/dist/compiled/jest-worker/processChild.js

[balajiv113]

Yes able to reproduce this issue easily

There is a simple docker image and a repo captured here to reproduce this issue
https://github.com/jbinto/rosetta-what