ESP32 fails to enter download mode on Macbook via USB C Adapters (ESPTOOL-383)

[CraftyMyner]

Operating System

16" Macbook Pro 2021, M1 Max, macOS 12.0.1

Version

latest master

Python Version

3.10.0

Chip Description

ESP32-DevKitC V4, ESP32-WROOM-32D, ESP-PROG

Device Description

Setup 1: ESP32-DevKitC V4 -> USB Micro B to USB-C -> Macbook USB C port (Tried all ports)
Setup 2: ESP32-DevKitC V4 -> USB Micro B to USB A -> USB A to USB-C -> Macbook USB C port (Tried all ports)

Hardware Configuration

Nothing else is attached.

How is Esptool Run

platformio, arduino, and esptool.py from cli

Full Esptool Command Line that Was Run

python3 esptool.py --port /dev/cu.usbserial-10 read_mac

Esptool Output

esptool.py v3.3-dev
Serial port /dev/cu.usbserial-10
Connecting......................................

A fatal error occurred: Failed to connect to Espressif device: Wrong boot mode detected (0x13)! The chip needs to be in download mode.
For troubleshooting steps visit: https://docs.espressif.com/projects/esptool/en/latest/troubleshooting.html

More Information

This appears to be similar to a previous issue: macOS connection failure (ESPTOOL-363) #696

Pressing boot/reset buttons allows chip to enter download mode.
Tried multiple boards, none are able to enter download mode.
Swapped driver for Silicon Labs CP210x VCP driver, still does not work.

Issue does not exist on Macbook Pro 2015 or windows laptop (USB A Ports)

Other Steps to Reproduce

No response

I Have Read the Troubleshooting Guide

• I confirm I have read the troubleshooting guide.

[radimkarnis]

Hello @CraftyMyner,
thanks for reporting and investigating this!

You are right that #696 is probably the same issue. I also use a MacBook and have experienced this behavior with a USB Micro B to USB-C cable. Using a hub solves this.

I will try to capture the DTR/RTS signals and debug this. Until then, we'll keep this open.

[CraftyMyner]

Thanks for looking into this @radimkarnis,

I just did some more testing and it looks like the reset signal works, just not the boot signal. A press of the boot button at the right time will get it to program.

[radimkarnis]

By any chance, do you have a USB2 hub available? There are reports (espressif/esp-idf#2849 (comment)) of failing auto-reset when using USB3.x ports. We could verify this is the same issue.

[CraftyMyner]

MacBook USB C (Thunderbolt 4/ USB 4) to USB Micro B cable - does not work.
MacBook USB C (Thunderbolt 4/ USB 4) to USB A Gen 3 adapter -> standard USB A Gen 2 to USB Micro B cable - does not work.
MacBook USB C (Thunderbolt 4/ USB 4) to USB A Gen 3 adapter -> USB A Gen 3 Hub -> standard USB A Gen 2 to USB Micro B cable - works!

[MMI]

I have a colleague experiencing this same issue with a brand new M1 macbook... they are going shopping for hubs to try.

Apropos of nothing, I had a very similar issue (able to reset but not trigger the ROM bootloader) when using esptool.py in a VM -- which led me to think that it was a problem with the way USB was virtualized. This new problem has me wondering it's more general to MacOS...

[CraftyMyner]

I thought it might be the apple driver but I tried the one from silabs and no change. I got a capture on my scope of the download mode sequence waveforms, there appears to be a slight glitch on the boot line right as the reset goes high for some reason.

[CraftyMyner]

P.S. This is through the same USB C to USB A adapter (that does not work on its own) AND the hub. 100% repeatable.

[MMI]

So for my part, I'd been reading the DTR and RTS lines using a logic analyzer (I don't have a scope) and I have observed
that when DTR is dropped more than 2ms before RTS comes up, the ESP32 will reset in "normal" mode. The trace below shows the desired behaviour (from esptool.py --chip esp32 --port /dev/xxx read_mac)

I'm curious... is it possible to see a trace of both the inputs (RTS,DTR) and the outputs (EN and GPIO0)?

[dobairoland]

Similar issue was reported here.

[MMI]

As has been noted elsewhere, the circuit is sensitive to timing. Those notes tend not to be specific. That's probably because everybody's implementation of that reference "auto-reset" circuit is slightly different -- particularly the capacitor.

On my devices, I have observed that the time between the dropping of DTR and the rising of RTS needs to be less than 2ms in order to trigger a reset into download mode. We have also figured out that changing the capacitor eases that constraint (because the capacitor affects the rise time on the EN pin).

Out of curiousity, is the chain of python, pyserial(or whatever is used) all native M1 compiled or is it running under Rosetta? If Rosetta is involved, I'm wondering if that's what's slowing down the hardware line transitions -> and would a fully native build "fix" the problem?

[256dpi]

I'm experiencing similar issues on my Macbook Pro M1 Max over USB-C. But, on my side, it seems to be an issue with esptool itself. Any release prior to v3.2 allows me to successfully read the chip ID, flash, etc. With v3.2 and master, I always get the "Wrong boot mode detected (0x13)!" error.

@radimkarnis Have there been any significant changes since v3.2 related to boot mode selection logic?

[MMI]

I made the following C program for your experimenting pleasure:

device-bootloader.c

Using it, I get this trace (intel imac):

If this works on your M1 Macs then there's probably a path to fixing esptool.

[radimkarnis]

So, today I tried debugging this and can't reproduce the described behavior anymore. The reset sequence always works, I tried using a multitude of different cables, USB hubs and even devkits with different USB-UART bridge chips.

I used a logic analyzer to capture traces and they look alright - no glitch on the DTR->BOOT/IO0 pin at all.

Can you guys try updating your OS/drivers and see if this maybe got solved on the driver side? I will try to reproduce this with an older Intel MacBook.

[256dpi]

I got a recent ESP32-DevKit-C this week, and was surprised that the automatic boot loader reset didn't work at all. Even with cable combinations I have used successfully with other boards... I took the opportunity to test the script by @MMI and it worked! So I implemented the step sequence by @MMI in esptool and voilà it works there as well. For more details, see my updated PR #718.

[radimkarnis]

So I have tried with an older Intel MacBook Pro running Monterrey and the reset works just fine.

Apart from the conversation in the linked PR, has anyone who was affected tried updating the OS? This seems to have resolved the issue on my side.

Seeing that there are no other reports of this issue makes me believe this was a temporary bug in drivers.

[MMI]

A new M1 laptop (not mine, unfortunately) has come into my radius -- I can confirm that with MacOS 12.1 the problem is reproducible. I have permission to upgrade and will report back when that is done.

[MMI]

Upgrading to 12.2 did not fix the problem.
Updating python did not fix the problem.
Oddly, pip installs esptool 2.8 by default.
Cloning the esptool repo and running 3.3-dev also does not fix the problem (only changes the error message from "timeout waiting for ..." to "wrong boot mode"

I was doing my tests on our hardware so to make sure, I redid the last test on a devkitC that I have kicking around and it too fails.

Oh yeah, to be clear my test is very simple:

esptool.py -p /dev/cu.usbserial-XXXX read_mac

[CraftyMyner]

I can confirm that the issue still exists on my M1 Max MacBook Pro 16". If I use direct USB-C to DevKitC it does not work, however, if I put a USB3 hub in between it works fine. The ESP-Prog (direct connect) seems to complete programming 19 out of 20 times without issue. (Sometimes it fails and needs another attempt)

Possibly related, JTAG debugging seems to be extremely flakey on USB-Prog to the point it basically does not work. I have not gotten further into diagnosing yet.

[Arctic-git]

I recently updated my OS. Prior to that I haven't had any problems.

2016 Intel MBP with MacOs 12.3
Custom ESP32 board with silabs CP2104 connected via an USB-C adapter.
CP210x Macintosh OS VCP Driver 6.0.2

Green: EN pin (scope trigger on rising edge)
Red: BOOT pin

esptool_patch720.mov

Like #712 (comment) mentioned you can see the unwanted pulse on the BOOT line. It varies randomly in length as esptool retries to connect. Esptool only succeeds if the pulse is short enough or not there (0:06).
I also tested 2 other boards with similar behavior (one custom board with cp2104, one Esp32 Lyrat with cp2102). The first was about the same, the latter was better with shorter pulses on average and usually connected after a couple seconds.

I found #718 and replaced my esptool.py (see video). With the modified esptool.py the pulse is gone and all boards connect instantly all the time. Seems like setting RTS and DTR in a single ioctl call fixes the problem for me.
(#718 also seems to affect the reset-to-app sequence which has to be investigated)

[256dpi]

FYI: I recently updated my fork and added the changes in a new branch: https://github.com/256dpi/esptool/tree/dtr-rts-fix. I intend to keep it updated with the upstream esptool until there is an official solution.

[x2ocoder]

esp-idf v4.4.1

Problem with:
M1 Default Driver for CP2104

Confirm this does not fix:
Swapped driver for Silicon Labs CP210x VCP driver, still does not work.

Confirm adding USB Hub fixes.

FYI Reset/Boot lines toggle in both cases, but glitches more without the hub. Suggest merging good fixes after confirmation of operation.

[tomg1970]

Same Problem with 12.4 Macbook Pro M1 2021

[TJC]

Can confirm I've had the same issue on a 2021 Macbook Pro M1.
Didn't work on a USB-C-to-A adapter, but does work on a USB hub that is connected to a USB-C-to-A adapter!

[MMI]

I recently moved my dev work to a Ubuntu 22 box and started having this same problem.

My simple test is: esptool.py -p /dev/ttyUSB0 read_mac which always fails because it fails to switch into bootloader.

AMD based PC, no USB hub -- fails with my product device (which uses an FTDI chip) as well as an ESP32 devkit board.

My production flashing script has my workaround and it works (the workaround is my own ioctl set per my gist and using the --before no_reset and --after no_reset options to esptool/espfuse). I do it this way to avoid having to maintain a my own fork just for this.

My feeling is that it's time to admit that the python code inside esptool.py that handles boot mode switching is subtly broken.

I'm not saying my workaround is 100% correct either but it sure would be nice if something like it were added to the list of strategies tried by esptool since it's clear that I'm not the only one seeing this.

[georgik]

The similar problem could be simulated also on Rust implementation of espflash - https://github.com/esp-rs/espflash

[cataglyphis]

MacBook Air (M1, 2020)
Apple M1
macOS 12.6

• USB-C -> USB Hub -> USB A/Micro -> ESP32 (do not work)
• USB-C -> USB Hub -> USB A Hub -> USB A/Micro -> ESP32 (work)

[beniroquai]

Also report the same issue. It's working with a USB-splitter (i.e. USB-C -> HDMI -> USB-A), but not with a pure USB-C to USB-A adapter.

[jhmaloney]

I can confirm similar behavior on a MacBook Pro 2021 (M1 Pro chip) running Mac OS 12.6.1.

Direct USB cable connection from the USB-C port to ESP-32: fails
Using Satechi hub with 3 USB-A ports + Ethernet: works

Same results with three different ESP-32 boards. As a check of the ESP-32 boards, esptool.py is able to connect to all three boards reliably from a 2015 Macbook Pro.

[oclyke]

Confirming, this is still an issue. On M1 Mac RTS seems to be asserted significantly later than DTR, causing IO0 (BOOT) to fall well after the bootload vs. main program execution decision is made. On my setup (USB-C direct to ESP32 dev kit) we can see IO0 lagging behind EN by approx 3.5ms. When testing the same hardware from another machine (x86 Ubuntu, also potentially with an older version of esptool.py) we see the expected behaviour were IO0 is brought low before EN comes up.

In my opinion this just boils down to a matter of performance differences between the two platforms. PySerial's latency between setting two control lines appears to differ and the ESP32 auto-bootload circuit is timing-sensitive enough to the point that the Mac timing is out of spec.

First image is good timing, second is bad timing.

[oclyke]

Follow-up, I've tested the fix from @256dpi. Its a charm on my machine. It required no additional packages and was a simple code change. I simply added the changes from 9806318 to my local esptool.py file.

The usage of termios and fcntl reduced the latency between setting DTR and RTS from ~4ms to effectively too fast for my logic analyzer to measure. Nice work @256dpi!

I hope this fix will be considered for integration into the main tool - it must be shown to work on many platforms - or perhaps enabled only when certain platforms are detected (i.e. M1 Mac)

Below are two views of the timing after the fix - you can see that IO0 (BOOT) is now low upon startup as expected.

[tofublock]

Same behaviour on Raspberry Pi OS Linux Kernel 5.15.61, and master...256dpi:esptool:dtr-rts-fix fixes it with never a single failure.
Please safeguard against other OSes if necessary and integrate!

[radimkarnis]

We are now assessing multiple ways to make the reset sequence more robust across all platforms - @256dpi's fix being one of them. We will most likely integrate this in the following weeks.

Thank you all for your valuable feedback and patience!

[Jason2866]

@256dpi import fcntl does not work with Windows. The module does not exists for Windows.

[MrNavaStar]

@256dpi import fcntl does not work with Windows. The module does not exists for Windows.

The equivalent of fcntl for windows is win32api. The patch only has one fcntl call, so putting a check in for windows or unix should be easy.

Another option pointed out by this stackoverflow post could just be adding this class to the program:

def fcntl(fd, op, arg=0):
    return 0
        
def ioctl(fd, op, arg=0, mutable_flag=True):
    if mutable_flag:
        return 0
    else:
        return ""
    
def flock(fd, op):
    return
        
def lockf(fd, operation, length=0, start=0, whence=0):
    return

Supposedly it works just like the official module? I'm not sure. But if someone can test on windows its worth a shot.

[Jason2866]

@MrNavaStar Next one is termios which is also not available on Windows ;-)

[MrNavaStar]

@MrNavaStar Next one is termios which is also not available on Windows ;-)

Of course haha, couldn’t have been that easy. A quick google search suggests that pynput should be a drop in replacement for windows.

[HankLloydRight]

Sorry for the bump. Just posting an alternative solution:

I found that adding a 10uf cap between RST and GND was the only thing that solved this for me. And I tried all the other solutions first.