arduino-esp32 projects fail on macOS 11 "Big Sur"

[adamwilt]

Hardware:

Board: DOIT ESP32 DEVKIT V1
Core Installation version: esp32 1.0.3
IDE name: Arduino 1.8.12, 1.8.13
Flash Frequency: N/A
PSRAM enabled: N/A
Upload Speed: N/A
Computer OS: macOS 11.0 Beta (20A5384c) (and earlier betas, too)

Description:

Any attempt to compile an ESP32 project, even an empty one as shown below, fails with:
ValueError: dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found

The same projects compile and run without problems on macOS 10.5.X.

See also https://forum.arduino.cc/index.php?topic=699989.0 for another report of the same problem.

Sketch:

void setup() {
}

void loop() {
}

Debug Messages:

Traceback (most recent call last):
  File "esptool.py", line 57, in <module>
  File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "serial/tools/list_ports.py", line 29, in <module>
  File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "serial/tools/list_ports_posix.py", line 31, in <module>
  File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "serial/tools/list_ports_osx.py", line 32, in <module>
ValueError: dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found
Failed to execute script esptool
exit status 255
/Applications/Arduino.app/Contents/Java/arduino-builder returned 255
Error compiling for board DOIT ESP32 DEVKIT V1.

[lbernstone]

espressif/esptool#540
pyserial/pyserial#509

[huykaamaa]

me too :(

[aungthawdar]

When I compile and upload,
I got the same error on my BigSir Beta version and solved with the following ways.

-esptool is dependent on pyserial (you need to install pyserial latest version) and

-Arduino's default preferences for ESPTool is binary file , it does not use the (esptool.py) source file.
So need to replace default esptool binary file with esptool.py file in system.

How to replace?... :)

1. Click Arduino Menu => Preferences
2. Find [ArduinoLibraryPath]/preferences.txt at the bottom left of Preferences' Settings Tab, Click to Open
   ArduinoLibrary Folder.
3. Find & Open packages/esp32/hardware/esp32/1.x.x Folder and open platform.txt
4. At the line no 7. replace binary tools.esptool_py.cmd=esptool with tools.esptool_py.cmd=esptool.py and save.
5. Open packages/esp32/hardware/esp32/1.x.x/tools folder.
6. Copy esptool.py and paste to ArduinoLibraryFolder/packages/esp32/tools/esptool_py/2.x.x
7. Quit Arduino IDE and open again. have a nice coding. 👍

[robsworld]

@aungthawdar 👍

Thank you for this solution. It works great.
The only think I have to add is you must change the permission of esptool.py with the terminal:

chmod +x /Users/XXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py

[lbernstone]

Does the executable work? If you open a command line and run it, do you get an error (perhaps about python2 not being installed)? We need a solution that can work on osx and big sur.

[SpoonS2K]

@aungthawdar tried your solution and got this error

Traceback (most recent call last): File "/Users/xxxxxxx/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 37, in <module> import serial ImportError: No module named serial

[me-no-dev]

can you guys try the idf-release/v4.2 branch? It has updated esptool that should contain a fix

[TimoWielink]

@me-no-dev seems to be working with 4.2! Perfect Thnx!

[adamwilt]

When I tried replacing esptool with esptool.py in the platform.txt file, I got the serial ImportError: No module named serial error. So I tried the 4.2 branch, but I'm clearly missing some steps. I used the "installation instructions for Mac OS" at https://github.com/espressif/arduino-esp32/blob/idf-release/v4.2/docs/arduino-ide/mac.md, adding the --branch option:
git clone --branch idf-release/v4.2 https://github.com/espressif/arduino-esp32.git esp32 --depth 1
Whether I do this on top of my existing installation, or as the initial ESP32 installation after wiping my existing library files, I still get ValueError: dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found.

What am I doing wrong? What's the proper way to install the idf-release/v4.2 branch?

[Liferme]

When I compile and upload,
I got the same error on my BigSir Beta version and solved with the following ways.

-esptool is dependent on pyserial (you need to install pyserial latest version) and

-Arduino's default preferences for ESPTool is binary file , it does not use the (esptool.py) source file.
So need to replace default esptool binary file with esptool.py file in system.

How to replace?... :)

1. Click Arduino Menu => Preferences
2. Find [ArduinoLibraryPath]/preferences.txt at the bottom left of Preferences' Settings Tab, Click to Open
   ArduinoLibrary Folder.
3. Find & Open packages/esp32/hardware/esp32/1.x.x Folder and open platform.txt
4. At the line no 7. replace binary tools.esptool_py.cmd=esptool with tools.esptool_py.cmd=esptool.py and save.
5. Open packages/esp32/hardware/esp32/1.x.x/tools folder.
6. Copy esptool.py and paste to ArduinoLibraryFolder/packages/esp32/tools/esptool_py/2.x.x
7. Quit Arduino IDE and open again. have a nice coding. 👍

It really works, and we have to combine the method with the command @robsworld provided. When compiling, remember to choose "ESP32 Arduino", NOT "ESP32 Arduino(in sketch book)", and have fun coding!

[me-no-dev]

@TimoWielink thanks for the confirmation :) I have updated the version in Master, so it will go out in 1.0.5 in a few days.

[aungthawdar]

@aungthawdar tried your solution and got this error

Traceback (most recent call last): File "/Users/xxxxxxx/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 37, in <module> import serial ImportError: No module named serial

@SpoonS2K
You need to install pyserial package and be sure to import first

https://pyserial.readthedocs.io/en/latest/pyserial.html
esptool.py used(import) serial library

Here is how to test, you've already installed serial or not.

1. Open terminal
2. type python in that terminal
3. type import serial
4. if no error, you've already installed serial,
   other wise you'll get
   "import serial ImportError: No module named serial

Have a nice code. :P

[tomtomhoward]

Thanks guys, just did the above and got it working on Big Sur for ESP32 in Arduino.
I had to update to 4.2, install python serial, and modify the files.

There's also some playing about getting the CP210x driver working for Dev-Kit ESP32 variants, involving adding the Silicon Labs developed ID to kext in OSX.

[me-no-dev]

@tomtomhoward if you have updated to 4.2, then you should not have needed to change esptool. It is already a patched one.