Issue installing on M1 Mac

[humbertowoody]

Describe the bug

Default Locust installation is not working under M1 Mac.

Already read:

• Can't install locust on M1 Mac #2115
• Locust install issue - M1 Mac #1949

I tried switching gevent versions to those shown on those PRs comments but failed to install (clang compiler issues on other packages) on my machine. I include a log of the issue on the Steps to Reproduce, but here's a pip freeze's output:

Brotli==1.0.9
certifi==2022.9.24
charset-normalizer==2.1.1
click==8.1.3
ConfigArgParse==1.5.3
Flask==2.2.2
Flask-BasicAuth==0.2.0
Flask-Cors==3.0.10
gevent==22.10.2
geventhttpclient==2.0.8
greenlet==2.0.1
idna==3.4
itsdangerous==2.1.2
Jinja2==3.1.2
locust==2.13.0
MarkupSafe==2.1.1
msgpack==1.0.4
psutil==5.9.4
pyzmq==24.0.1
requests==2.28.1
roundrobin==0.0.4
six==1.16.0
typing_extensions==4.4.0
urllib3==1.26.12
Werkzeug==2.2.2
zope.event==4.5.0
zope.interface==5.5.1

Expected behavior

Locust to be installed and working on ARM-based Mac.

Actual behavior

Shows an issue with g-event.

Steps to reproduce

$ python3 -m venv ./venv && source ./venv/bin/activate && pip --version && python --version && uname -a && pip install locust && locust


pip 22.2.2 from /Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/pip (python 3.10)
Python 3.10.8
Darwin gojira.local 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct  9 20:14:30 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T8103 arm64
Collecting locust
  Using cached locust-2.13.0-py3-none-any.whl (823 kB)
Collecting psutil>=5.6.7
  Using cached psutil-5.9.4-cp38-abi3-macosx_11_0_arm64.whl (244 kB)
Collecting gevent>=20.12.1
  Using cached gevent-22.10.2-cp310-cp310-macosx_10_9_universal2.whl (1.9 MB)
Collecting Flask-BasicAuth>=0.2.0
  Using cached Flask_BasicAuth-0.2.0-py3-none-any.whl
Collecting typing-extensions>=3.7.4.3
  Using cached typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting geventhttpclient>=2.0.2
  Using cached geventhttpclient-2.0.8-cp310-cp310-macosx_11_0_arm64.whl (42 kB)
Collecting roundrobin>=0.0.2
  Using cached roundrobin-0.0.4-py3-none-any.whl
Collecting Werkzeug>=2.0.0
  Using cached Werkzeug-2.2.2-py3-none-any.whl (232 kB)
Collecting requests>=2.23.0
  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Collecting flask>=2.0.0
  Using cached Flask-2.2.2-py3-none-any.whl (101 kB)
Collecting ConfigArgParse>=1.0
  Using cached ConfigArgParse-1.5.3-py3-none-any.whl (20 kB)
Collecting pyzmq!=23.0.0,>=22.2.1
  Using cached pyzmq-24.0.1-cp310-cp310-macosx_10_15_universal2.whl (1.8 MB)
Collecting Flask-Cors>=3.0.10
  Using cached Flask_Cors-3.0.10-py2.py3-none-any.whl (14 kB)
Collecting msgpack>=0.6.2
  Using cached msgpack-1.0.4-cp310-cp310-macosx_11_0_arm64.whl (69 kB)
Collecting Jinja2>=3.0
  Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting click>=8.0
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting itsdangerous>=2.0
  Using cached itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting Six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting zope.event
  Using cached zope.event-4.5.0-py2.py3-none-any.whl (6.8 kB)
Collecting zope.interface
  Using cached zope.interface-5.5.1-cp310-cp310-macosx_12_0_arm64.whl
Requirement already satisfied: setuptools in ./venv/lib/python3.10/site-packages (from gevent>=20.12.1->locust) (65.4.1)
Collecting greenlet>=2.0.0
  Using cached greenlet-2.0.1-cp310-cp310-macosx_12_0_arm64.whl
Collecting certifi
  Using cached certifi-2022.9.24-py3-none-any.whl (161 kB)
Collecting brotli
  Using cached Brotli-1.0.9-cp310-cp310-macosx_10_9_universal2.whl (786 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting charset-normalizer<3,>=2
  Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
Collecting MarkupSafe>=2.1.1
  Using cached MarkupSafe-2.1.1-cp310-cp310-macosx_10_9_universal2.whl (17 kB)
Installing collected packages: roundrobin, msgpack, brotli, zope.interface, zope.event, urllib3, typing-extensions, Six, pyzmq, psutil, MarkupSafe, itsdangerous, idna, greenlet, ConfigArgParse, click, charset-normalizer, certifi, Werkzeug, requests, Jinja2, gevent, geventhttpclient, flask, Flask-Cors, Flask-BasicAuth, locust
Successfully installed ConfigArgParse-1.5.3 Flask-BasicAuth-0.2.0 Flask-Cors-3.0.10 Jinja2-3.1.2 MarkupSafe-2.1.1 Six-1.16.0 Werkzeug-2.2.2 brotli-1.0.9 certifi-2022.9.24 charset-normalizer-2.1.1 click-8.1.3 flask-2.2.2 gevent-22.10.2 geventhttpclient-2.0.8 greenlet-2.0.1 idna-3.4 itsdangerous-2.1.2 locust-2.13.0 msgpack-1.0.4 psutil-5.9.4 pyzmq-24.0.1 requests-2.28.1 roundrobin-0.0.4 typing-extensions-4.4.0 urllib3-1.26.12 zope.event-4.5.0 zope.interface-5.5.1

[notice] A new release of pip available: 22.2.2 -> 22.3.1
[notice] To update, run: pip install --upgrade pip
Traceback (most recent call last):
  File "/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/bin/locust", line 5, in <module>
    from locust.main import main
  File "/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/locust/__init__.py", line 9, in <module>
    from gevent import monkey
  File "/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/gevent/__init__.py", line 86, in <module>
    from gevent._hub_local import get_hub
  File "/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/gevent/_hub_local.py", line 101, in <module>
    import_c_accel(globals(), 'gevent.__hub_local')
  File "/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/gevent/_util.py", line 148, in import_c_accel
    mod = importlib.import_module(cname)
  File "/opt/homebrew/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/gevent/_gevent_c_hub_local.cpython-310-darwin.so, 0x0002): tried: '/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/gevent/_gevent_c_hub_local.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/gevent/_gevent_c_hub_local.cpython-310-darwin.so' (no such file), '/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/gevent/_gevent_c_hub_local.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

My assumption is that I need to lock some dependencies versions (gevent from what I read) in order to make it work but I'm unsure as to how to find the ones that should/would work on my setup. Also I tried updating pip to the patch version bump, but it made no difference on resolving gevent's version.

Environment

• OS: MacOS Ventura 13.0.1
• Python version: 3.10.8
• Locust version: 2.13.0
• Locust command line that you ran: locust (expected help and/or error message)
• Locust file contents (anonymized if necessary): don't even have a locustfile.py yet, was following installation instructions on locust.io. Thanks in advance!

[cyberw]

I think this is the core of your problem: '/Users/humbertowoody/Proyectos/pago46/swarm-tester/venv/lib/python3.10/site-packages/gevent/_gevent_c_hub_local.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')

Check out https://stackoverflow.com/questions/72289889/how-do-i-fix-the-mach-o-but-wrong-architecture-error-in-pycharm

Or try googling the error message in combination with gevent.

You could also try pip install --only-binary :all: gevent (so pip will only try pre-compiled versions of gevent, if there are any)

I'm sorry people running on apple silicon are having so many issues, but there isn't a lot we can do on the locust side. I'm getting an M1 soon and then I might have some more experience :-P

[humbertowoody]

Hi @cyberw! First of all, thanks for your fast and thorough response. I did what you told me but, to be honest with you, M1/2 is filled with rabbit holes like this one and it's crazy the amount of time and effort that goes with every single issue. I hope for a better and/or easier way to deal with these problems in the future so this doesn't happen anymore :D

For the time being, I have an Ubuntu 22.04 virtual machine running (not a simple installation, you can google about it and it's not as straightforward but it's possible!) with aarch64 (instead of arm64 from M1, I believe it's the same thing but not sure!) and, guess what, it worked without anything else! Crazy how the same architecture, under the same physical system, but different OS gets different outcomes (even if virtualized)!

I don't know if I can call it a fix, but honestly, I have been dealing with these issues on all my tools for the past 2 years and I'm tired. I can try and do more debugging and share more logs in here if you want, but for now, it's working and I'm doing progress with Locust (awesome tool!). Please close the issue if it's appropriate. Again, thank you so much for your response!

Here are the two pip freeze's outputs! They're identical! It's crazy.

[cyberw]

Glad that you found a workaround. Yea, I guess its not the package versions that is the issue, I think it is (for some reason) getting the wrong build (architecture) of them. I guess we can close this for now. I’ll look at it once i get my M1

[humbertowoody]

Hey @cyberw! Good news!

Also, for everyone using an ARM-based Mac, I found the command that not only fixed Locust's installation, but it helped me on other projects, perhaps it's useful in the future:

arch -arm64 pip install --no-cache-dir locust --no-binary :all:

As it compiles everything from source (I assume) there's no way for a dependency to be other architecture! I'm happily back on track working without a VM :) I still wanted to thank you so much! And sorry for the not-locust-specific issue, I hope this can help others running into the same problems in the future :)

[cyberw]

Awesome!