Serial.Begin() hangs if RX line has traffic at boot in versions >= 1.0.5

[mbilsky]

Hardware:

Board: ESP32-WROVER (both dev kit and ESP-EYE)
Core Installation version: v1.0.4 works fine, v1.0.5 and v1.0.6 have the bug
IDE name: Bug validated in Arduino IDE and Platformio
Flash Frequency: 40Mhz
PSRAM enabled: yes
Upload Speed: 115200
Computer OS: Windows 10, Ubuntu

Describe your problem here

If we boot up our code with the serial RX line busy, the ESP32-Wrover hangs. After some hardware debugging (testing on the ESP-WRrover-kit) the issue seems to be happening in Serial.begin(). More specifically, after uartAttachRx(uart, rxPin, inverted) is called in esp32-hal-uart.c . We didn't have this issue in earlier versions of the IDF (we're using Arduino on platformIO and moved from 3.0.0 to 3.X.0). 3.1-3.3 all seem to have the same problem. We can bodge around it by being careful with the other MCUs the ESP32 is talking to but I'd rather just fix the bug :)

The experiment consists of a teensy printing constantly to the serial bus (10 uS downtime between string packets).

It appears that the RX interrupt is firing continuously preventing other code from executing.

I've tried initializing Serial in void loop (in a run once block) to confirm that we can make it through the rest of setup just fine. Same issue happens.

I can also successfully run: Serial.begin(115200,SERIAL_8N1,-1,1); in void setup() to get serial output just fine. Then end the serial and restart with normal Serial.begin(115200) ;

I'm attaching a screenshot showing the stack where it is stuck at esp_intr_alloc()

Sketch: (leave the backquotes for code formatting)

//Change the code below by your sketch
#include <Arduino.h>

void setup() {
Serial.begin(115200);
}

void loop() {
Serial.println("hello world");
delay(100);
}

Debug Messages:

Enable Core debug level: Debug on tools menu of Arduino IDE, then put the serial output here 

It locks up right after PSRAM_enable() prints to the screen

[brawner]

I'm working with @mbilsky, it looks specifically like removing the check for a full queue seems to be part of the problem. When we revert this one-liner our problem is resolved. Would opening a PR for this fix be welcome?

https://github.com/espressif/arduino-esp32/pull/3713/files#diff-3d9c6876ed404db1b65230f87978c06285493ebaba9ceac1d2f45446f8cf8a87L88-R88

[mbilsky]

@brawner Unfortunately after further analysis with the debugger, that still isn't solving the issue.

Any other ideas??

[mbilsky]

The bodge that I figured out is to reset the UART peripheral (needed for soft resets) and start at a low baud rate before changing to match the high speed data line.

The read may not be necessary but it's currently not hurting anyone either.

  periph_module_reset(PERIPH_UART0_MODULE);
  periph_module_reset(PERIPH_UART1_MODULE);
  periph_module_reset(PERIPH_UART2_MODULE);

  Serial.begin(9600, SERIAL_8N1, 3.1);
  while (Serial.available() > 0)
  {
    Serial.read();
  }
  Serial.updateBaudRate(115200);

[RLmonitor]

I'm seeing this too on Serial2. I'll try your "bodge".
I also see it when trying to change to a higher baudrate after boot.
I've had a bit of a look through the HAL code, but have not found anything useful.
My test code below sometimes hangs at the Serial2.begin(). I'm wondering if holding the Rx high during baud change might help.

    Serial.println("Comms reset start");
    Serial2.begin(921600, SERIAL_8N1, 14, 12);   
    Serial.println("Comms reset done");

[RLmonitor]

I was assuming (hoping) that Serial2.begin would flush the Rx buffer. Once I removed all these and used updateBaudRate to shift to the higher data rate, the software was much more stable. So far, no sign of hanging in the Serial code.

[SuGlider]

@mbilsky @RLmonitor - could you try it using the new hardwareSerial of the PR #5549 ?
It is a new UART implementation for Arduino on top of IDF.
Maybe it works for you.

Please let me know about the results.

[mbilsky]

@SuGlider We will test on Monday and let you know the result.

@brawner

[RLmonitor]

@SuGlider I'm trying this out, but am missing some include files. E.G. #include "hal/uart_ll.h"
I may have to try to re-install the IDF connection with Arduino.

[SuGlider]

Note:

In order to apply PR #5549 , it's necessary to use Arduino Core 2.0.0RC version.

When using Arduino IDE, it is a change on the Board Manager to update to version 2.0.0RC.

For PlatformIO, it will be necessary to wait for them to update their servers, which only will happen when we realease officially the 2.0.0 version.

Core 1.0.6 uses IDF version 3.3 which is incompatible with IDF 4.4 used in Arduino Core 2.0.0.

[SuGlider]

@mbilsky @RLmonitor -
I have tested it with a ESP32 sending continuosly a string to ESP32-WROVER with PSRAM active.
ESP32-WROVER RX line busy at boot.
Using Arduino Core 2.0.0 with PR #5549 , I've successfully run the sketches below:

Sketch used in ESP32:

HardwareSerial SenderUART(2);

void setup() {
  SenderUART.begin(BAUD_RATE);
  
  Serial.begin(115200);
  Serial.println();
  Serial.print("Sending String...");
  Serial.println();  
}

void loop (){
  SenderUART.write("UX-");
  SenderUART.flush();
}

Sketch used in ESP32 WROVER:

void setup() {
Serial.begin(115200);
}

void loop() {
Serial.println("hello world");
delay(1000);
}

void serialEvent() {
  Serial.println();
  Serial.print("Received: ");
  while (Serial.available()) {
    char CharSerialRX = (char)Serial.read();
    Serial.print(CharSerialRX);
  }
  Serial.println();
}

ESP32-WROVER Output:

ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x12 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:1240
load:0x40078000,len:13012
load:0x40080400,len:3648
entry 0x400805f8
⸮�`���$⸮LW+QW⸮Y.\&⸮KZX⸮K,\]⸮,'⸮LLW⸮⸮Y](\]⸮⸮equencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz
[   841][D][esp32-hal-psram.c:84] psramInit(): PSRAM enabled

hello world

Received: UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-

hello world

Received: UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-UX-

hello world