Macos M1 mysqlclient Symbol not found: _mysql_affected_rows ERROR

[kouyalong]

Hello
I'm trying to install mysqlclient with this command pip install mysqlclient
but i'm having this errors
mysql : 8.0.2
python : 3.7.10
Mac: OS version: 11.2.3 M1

ImportError: dlopen(/Users/shiwei/insight_r2_backend/env3/lib/python3.7/site-packages/MySQLdb/_mysql.cpython-37m-darwin.so, 2): Symbol not found: _mysql_affected_rows
Referenced from: /Users/shiwei/insight_r2_backend/env3/lib/python3.7/site-packages/MySQLdb/_mysql.cpython-37m-darwin.so
Expected in: flat namespace

I set
LD_LIBRARY_PATH=/opt/homebrew/lib

[jcbloch]

I'm struggling with a proper solution too.
FWIW, you are using the wrong ENV vars. Per readme, try: MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS
For example (for mysql 5.7):

MYSQLCLIENT_CFLAGS="-I/opt/homebrew/opt/mysql@5.7/include/mysql" MYSQLCLIENT_LDFLAGS="-L/opt/homebrew/opt/mysql@5.7/lib" pip  install -I mysqlclient

That said, above still doesn't produce a working module on my m1 mac-- even though install seems to succeed. It still borks on lines like from MySQLdb import _mysql (can't find _mysql). Nonetheless, I'm pretty sure those ENV vars are being used properly by pip. (You can convince yourself of that by changing MYSQLCLIENT_CFLAGS to the wrong directory and it won't find the required headers). I haven't dug in enough yet, but OTTH, it doesn't seem to be using LD_LIBRARY_PATH or DYLD_FALLBACK_LIBRARY_PATH

If you really have to get this working now, you can use this:, however despite the claims there, pymsql is not 100% a drop in replacement for mysqlclient ( see, eg this )

[jcbloch]

dug in a bit more: no matter how I install mysqlclient (that is regardless of with arch I try), it always builds for intel (that is the required symbols exist but they are for x86_64 not arm_64): Example:

sudo symbols [path to your packages]/MySQLdb/_mysql.cpython-36m-darwin.so -arch x86_64 | grep mysql_affected_rows
                0x00000000000071bc (     0x6) DYLD-STUB$$mysql_affected_rows [DYLD-STUB, LENGTH, NameNList, MangledNameNList, NList]

However, the actual mysqlclient dlyld is for arm64. Maybe this is part of the problem. For example:

sudo symbols /opt/homebrew/Cellar/mysql@5.7/5.7.34/lib/libmysqlclient.dylib -arch arm64  | grep affected_rows
Password:
                0x00000000000067e0 (     0x8) mysql_affected_rows [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts]
                0x0000000000007af4 (     0x8) mysql_stmt_affected_rows [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts]

Seems like the bug is that pip install is not respecting either the current arch, or an arch flag given (arch -arm64 pip install ...) here.

update: yes, pip install complains... ld: warning: ignoring file /opt/homebrew/Cellar/mysql@5.7/5.7.34/lib/libmysqlclient.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64

[jcbloch]

ok-- I think this issue should be closed-- as I don't think there's a good solution and it is not an issue with this module.

Python 3.X, X<9 on an M1 today is for x86_64 (intel) not arm64. The mysqlclient dyld is now native arm64, even for old sql like 5.7. Hence, when intel python calls dlopen(.../MySQLdb/_mysql.cpython-36m-darwin.so) it is going to bork even if that c code is compiled for arm64 (so it can talk to native mysql). Bummer, but the solution here is to install mysql code for intel, not to alter the python module. [update: this does indeed work. I installed mysql-client libs and includes for intel (using brew) and used them when doing the pip install. ]

@kouyalong ?

[nemoDreamer]

@jcbloch could you show how you did this?

I installed mysql-client libs and includes for intel (using brew) and used them when doing the pip install.

[jcbloch]

[I think this is what I did] I installed homebrew in a different location just for x86, something like this:

mkdir homebrew-x86 && curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip 1 -C homebrew-x86

(see https://github.com/Homebrew/brew/blob/664d0c67d5947605c914c4c56ebcfaa80cb6eca0/docs/Installation.md#untar-anywhere)

Then the libs for 5.7 (but should work for other versions too)

arch -x86_64 homebrew-x86/bin/brew install mysql-client@5.7

Which seemed to install the right things; eg:

sudo symbols ~/homebrew-x86/Cellar/mysql-client@5.7/5.7.32/lib/libmysqlclient.dylib -arch x86_64 | grep mysql_affected_rows

finally do the pip install (replace the ... so the paths are correct):

MYSQLCLIENT_CFLAGS="-I/.../homebrew-x86/Cellar/mysql-client@5.7/5.7.32/include/mysql" MYSQLCLIENT_LDFLAGS="-L/.../homebrew-x86/Cellar/mysql-client@5.7/5.7.32/lib -lmysqlclient" arch -x86_64 pip install -I  -vvv mysqlclient

[GrazMir]

Hi,
I followed the explanation above and made sure that all the libraries are compiled for x86_64. However something is still not working.

from MySQLdb import version
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/usr/local/Caskroom/miniconda/base/envs/twoex/lib/python3.7/site-packages/MySQLdb/__init__.py in <module>
     17     from MySQLdb.release import version_info
---> 18     from . import _mysql
     19

ImportError: dlopen(/usr/local/Caskroom/miniconda/base/envs/twoex/lib/python3.7/site-packages/MySQLdb/_mysql.cpython-37m-darwin.so, 0x0002): symbol not found in flat namespace '_mysql_affected_rows'

sudo symbols /usr/local/Caskroom/miniconda/base/envs/twoex/lib/python3.7/site-packages/MySQLdb/_mysql.cpython-37m-darwin.so -arch x86_64 | grep mysql_affected_rows
                0x00000000000071bc (     0x6) DYLD-STUB$$mysql_affected_rows [DYLD-STUB, LENGTH, NameNList, MangledNameNList, NList]

sudo symbols /usr/local/Cellar/mysql@5.7/5.7.36/lib/libmysqlclient.dylib -arch x86_64 | grep mysql_affected_rows
                0x0000000000005887 (     0xd) mysql_affected_rows [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts]

[weasteam]

Thanks @jcbloch

This is what I did:

1. First I use symbols to check each of my programs (mysql, python, _mysql.cpython-37m-darwin.so), and find out mysql is not x86, and I reinstall it.
2. After the reinstall of mysqlclient, I still got the error, it turns out pip will use a cached version when reinstall. I uninstall it and tried with pip install mysqlclient --no-cache-dir, and it worked for me.

[weasteam]

I also found out different shell give out different arch results.

For the same file _mysql.cpython-38-darwin.so, the default zch always gives 'arm64', the ide-terminal I installed for atom always gives 'x86'.

I use the ide-terminal to build all python, virtualenv, and mysqlclint, then it will work. If I install the python from the zch, it won't work.

-- labii.com

[jhsenjaliya]

wonder why python mysqlclient installs ( _mysql.cpython-38-darwin.so ) x86_64 version instead of ARM64.
but finally installing brew for both intel and arm worked for me, where mysql is installed via intel brew.
Also following way I can run everything intel simulated via terminal (open using Rosetta) and arm native via iterm.
if [[ `arch` == "i386" ]] then export HOMEBREW_HOME=/usr/local else export HOMEBREW_HOME=/opt/homebrew fi
then use HOMEBREW_HOME everywhere for the paths.

[amukherjee01]

File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/MySQLdb/init.py", line 18, in
from . import _mysql
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/MySQLdb/_mysql.cpython-310-darwin.so, 2): Symbol not found: _mysql_affected_rows
Referenced from: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/MySQLdb/_mysql.cpython-310-darwin.so
Expected in: flat namespace
in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/MySQLdb/_mysql.cpython-310-darwin.so

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/threading.py", line 946, in run
self._target(*self._args, **self._kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/utils/autoreload.py", line 64, in wrapper
fn(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/core/management/commands/runserver.py", line 115, in inner_run
autoreload.raise_last_exception()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/utils/autoreload.py", line 87, in raise_last_exception
raise _exception[1]
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/core/management/init.py", line 381, in execute
autoreload.check_errors(django.setup)()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/utils/autoreload.py", line 64, in wrapper
fn(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/init.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/apps/registry.py", line 114, in populate
app_config.import_models()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/apps/config.py", line 300, in import_models
self.models_module = import_module(models_module_name)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/contrib/auth/models.py", line 3, in
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/contrib/auth/base_user.py", line 47, in
class AbstractBaseUser(models.Model):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/db/models/base.py", line 122, in new
new_class.add_to_class('_meta', Options(meta, app_label))
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/db/models/base.py", line 326, in add_to_class
value.contribute_to_class(cls, name)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/db/models/options.py", line 207, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/utils/connection.py", line 15, in getattr
return getattr(self._connections[self._alias], item)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/utils/connection.py", line 62, in getitem
conn = self.create_connection(alias)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/db/utils.py", line 204, in create_connection
backend = load_backend(db['ENGINE'])
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/db/utils.py", line 111, in load_backend
return import_module('%s.base' % backend_name)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 15, in
import MySQLdb as Database
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/MySQLdb/init.py", line 24, in
version_info, _mysql.version_info, _mysql.file
NameError: name '_mysql' is not defined

Getting above error with mysqlclient in my django application can anyone help

[yafeiaa]

In my django(2.2.6) app，i meet this question too.

[jocelynxyq]

I got the same question and finally resolve it by homebrew(x86_64)、python(3.7.6 x86_64)、mysql@5.7(by x86_64 homebrew) and brew link/unlink[for me this is the important point].

My installation process is:

1. I never heard the question before, so I install homebrew(arm64), install pyenv and mysql with homebrew(arm64)

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)”
brew install openssl readline sqlite3 xz zlib
brew install pyenv 
brew install mysql@5.7

then i found pyenv cannot build python 3.7.6(which is my project need), so i install homebrew(x86_64) and use it by alias

2. Install homebrew(x86_64) and Python 3.7.6(x86_64)

softwareupdate —-install-rosetta
arch --x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

I also edit my .zshrc file

alias brew86="arch -x86_64 /usr/local/bin/brew"
alias pyenv86="arch -x86_64 pyenv"

Install Python 3.7.6(x86_64) as follow instruction:

brew86 install openssl readline sqlite3 xz zlib
CFLAGS="-I$(brew86 --prefix openssl)/include -I$(brew86 --prefix bzip2)/include -I$(brew86 --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew86 --prefix openssl)/lib -L$(brew86 --prefix readline)/lib -L$(brew86 --prefix zlib)/lib -L$(brew86 --prefix bzip2)/lib" pyenv86 install --patch 3.7.6 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

3. Install mysql-client with pip, and met this question when importing MySQLdb

Warning when install mysql-client[the important point]

  ld: warning: ignoring file /opt/homebrew/lib/libmysqlclient.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64

Error when import MySQLdb:

ImportError: dlopen(/Users/suki/Library/Caches/pypoetry/virtualenvs/server-oKbnXZYb-py3.7/lib/python3.7/site-packages/MySQLdb/_mysql.cpython-37m-darwin.so, 2): Symbol not found: _mysql_affected_rows

I check the architecture for python and _mysql.cpython-37m-darwin.so, all of them are x86_64

Python:

>>> import platform
>>> print(platform.architecture()[0]);print(platform.machine())
64bit
x86_64

_mysql.cpython-37m-darwin.so

sudo symbols /Users/xuyunqi/Library/Caches/pypoetry/virtualenvs/skyline-server-oKbnXZYb-py3.7/lib/python3.7/site-packages/MySQLdb/_mysql.cpython-37m-darwin.so -arch x86_64|grep mysql_affected_rows
                0x0000000000007660 (     0x6) DYLD-STUB$$mysql_affected_rows [DYLD-STUB, LENGTH, NameNList, MangledNameNList, NList]

so that must be my mysql wrong

4. Install mysql mysql-client using homebrew(x86_64)

brew86 install mysql@5.7
brew86 install mysql-client@5.7
brew86 install bzip2

add mysql@5.7(install by x86_64 homebrew) to .zshrc

export PATH="/usr/local/opt/mysql@5.7/bin:$PATH"

5. Brew link affect?

After struggling on same ImportError result, finally I think that may be my brew link affect. So I run the following:

brew86 link mysql@5.7
brew unlink mysql@5.7

6. Reinstall mysql-client with pip

MYSQLCLIENT_CFLAGS="-I/usr/local/Cellar/mysql-client@5.7/5.7.34/include/mysql" MYSQLCLIENT_LDFLAGS="-L/usr/local/Cellar/mysql-client@5.7/5.7.34/lib -lmysqlclient" arch -x86_64 pip install -I -vvv mysqlclient --no-cache-dir

It works! I run my project as expected.