-
Notifications
You must be signed in to change notification settings - Fork 7.6k
Hardware serial sometimes doesn't send every data. #6642
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I see you are using PlatformIO. Is your build based on Arduino Core 1.0.6? Would you mind testing it using Arduino IDE with latest Arduino Core (2.0.3-RC1)? Thanks. |
This is my pio ini file
|
I downloaded the new Arduino IDE 2.0.0 and i can only download the 1.0.6 espressif framework. |
Please check this guide: https://docs.espressif.com/projects/arduino-esp32/en/latest/installing.html For using v2.0.3-rc1, you need to change your link in Preferences to Development release link. |
I know that but with PIO i'm already using the latest with this ini file:
|
@zekageri - please check these links regarding PIO setup: |
Thank you, i will try this! |
Wow @SuGlider i got a bunch of warning messages ( like really much ) and then a linker fail c:/users/pc/.platformio/packages/toolchain-xtensa-esp32/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: final link failed: No such file or directory
collect2.exe: error: ld returned 1 exit status
*** [.pio\build\arduino-esp32\firmware.elf] Error 1 I will try to delete platform and packages folder from .platformio and recompile |
Okay, deleted cache, packages and platform folder from .platformio inside users. The sketch compiled, i did upload it to my esp32 but the problem presist. Serial.flush() does not wait for all bytes |
I'm pretty sure that There was an issue related to MODBUS and it was fixed there. Fixed by #6133 Are you sure you are building with the latest Arduino Core version? |
This is really interesting. Is there something wrong with my code? I configured Serial2 now instead of the Serial1 and commented out the microsec delay and i rarely got frame timeout now. If there is file system access or other mid heavy process on other tasks, i got one or two frame timeouts. |
void mBusSetup(){
pinSetup();
Serial2.begin(MBUS_BAUD,SERIAL_8N1,MBUS_RX,MBUS_TX);
}
void mSystem::pinSetup(){
pinMode(MBUS_TX_EN, OUTPUT);
digitalWrite(MBUS_TX_EN, LOW);
}
void mSystem::startWrite(){
digitalWrite(MBUS_TX_EN, HIGH);
}
void mSystem::endWrite(){
Serial2.flush();
//delayMicroseconds(400);
digitalWrite(MBUS_TX_EN, LOW);
}
void mSystem::rawWrite(uint8_t * data, int length){
int crc16 = getCRC_16(data, length);
startWrite();
Serial2.write( data, length );
Serial2.write( lowByte(crc16) );
Serial2.write( highByte(crc16) );
endWrite();
}
#define MBUS_LISTEN_TIMEOUT 150 // Modules will respond back in maximum of 15ms.
void writeReadDevice(int index){
// This function handles which device address to write and assmebles the writable data and calls
// rawWrite. If rawWrite returns, this function waits for the response data.
uint8_t exampleData[] = {0xFF, 0x03, 0x00, 0x06, 0x00, 0x03};
uint8_t responseData[20]; // Maximum response data is 10-13.
rawWrite(exampleData, sizeof(exampleData));
// wait for the response data.
// The modules will respond back if the data is understandable.
long startMs = millis();
while( !Serial1.available() ){
vTaskDelay(1);
if( millis() - startMs >= MBUS_LISTEN_TIMEOUT ){
Serial.println("FRAME TIMEOUT!");
return;
}
}
int respDataCount = 0;
while( Serial1.available() > 0 ){
responseData[respDataCount] = Serial1.read();
respDataCount++;
}
} |
Are you absolutely sure that the output data is broken? |
There is a new API for UART called It works as a sort of ISR... When data arrives and there is short interval of time (as long as about 11 symbols in the current baudrate), it will call right away the callback function. It may help... |
@SuGlider thank you. //Serial1.setRxBufferSize(512); // tried this, doesn't affect anything. data is always read in chunks of 120 bytes
Serial1.begin(9600, SERIAL_8N1, 2, 15); On 1.0.6 i don't see such issues. Could it be an ESP-IDF issue? UPD: ok, what I experience is similar to this: espressif/esp-idf#8369 |
Actually it works for all Serial, Serial1 and Serial2. Just use |
Sure. But it looks like i have problems on the signal lavel. Sorry, it is probably unrelated to this issue. |
It seems OK, if you are sending data through Serial2 and receiving through Serial1. |
I just tested |
Can it be high baud rate problem? ( 115200 ) Or can it be that other tasks interfere, and the pin is not going to low in time? |
void mSystem::startWrite(){
digitalWrite(MBUS_TX_EN, HIGH);
}
void mSystem::endWrite(){
Serial1.flush();
digitalWrite(MBUS_TX_EN, LOW);
}
void mSystem::rawWrite(uint8_t * data, int length){
int crc16 = getCRC_16(data, length);
startWrite();
Serial1.write( data, length );
Serial1.write( lowByte(crc16) );
// Other tasks interrupts this task in here maybe??
Serial1.write( highByte(crc16) );
// OR here?
endWrite();
} |
@zekageri - Just to make sure it is not bad configuration of the development environment, could you please update your PlatformIO version to the latest as described in https://piolabs.com/blog/news/platformio-oss-april-2022-updates.html Thanks! |
Thank you for your response. This results in a strange boot loop when using default_16MB.csv with I opened a new issue for this : Boot loop issue with 16mb csv |
Thank you i will try it now. |
I have got some compile errors with this: src/utilities/modBus.cpp: In member function 'void mSystem::testSerialSetup()':
src/utilities/modBus.cpp:1506:27: error: 'UART_INTR_RXFIFO_FULL' was not declared in this scope
.intr_enable_mask = UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
^~~~~~~~~~~~~~~~~~~~~
src/utilities/modBus.cpp:1506:27: note: suggested alternative: 'UART_BUFFER_FULL'
.intr_enable_mask = UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
^~~~~~~~~~~~~~~~~~~~~
UART_BUFFER_FULL
src/utilities/modBus.cpp:1506:51: error: 'UART_INTR_RXFIFO_TOUT' was not declared in this scope
.intr_enable_mask = UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
^~~~~~~~~~~~~~~~~~~~~
src/utilities/modBus.cpp:1506:51: note: suggested alternative: 'UART_FIFO_OVF'
.intr_enable_mask = UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
^~~~~~~~~~~~~~~~~~~~~
UART_FIFO_OVF*** [.pio\build\esp-wrover-kit\src\utilities\modBus.cpp.o] Error 1 #include "driver/uart.h"
void mSystem::testSerialSetup(){
uart_intr_config_t uart_intr = {
.intr_enable_mask = UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
.rxfifo_full_thresh = 112,
.rx_timeout_thresh = 1,
.txfifo_empty_intr_thresh = 10,
};
uart_intr_config((uart_port_t) 1, &uart_intr);
}
void mSystem::setup(){
Serial1.begin(MBUS_BAUD,SERIAL_8N1,MBUS_RX,MBUS_TX);
testSerialSetup();
} By the way here is all my task initialization. void initTasks(){
fileSys.init(1,2,FILE_SYS_STACK_SIZE);
hsh_modbus.init(-1,4,MODBUS_SYS_STACK_SIZE);
networkSys.init(1,2,NETWORK_SYS_STACK_SIZE);
hshDisplay.init(1,2,DISPLAY_SYS_STACK_SIZE);
hsh_Server.init(1,2,SERVER_SYS_STACK_SIZE);
hsh_Performance.init(1,2,PERFORMANCE_SYS_STACK_SIZE);
hsh_GeoSystem.init(1,2,GEO_SYS_STACK_SIZE);
hsh_timeSystem.init(1,2,TIME_SYS_STACK_SIZE);
zones.init(1,3,ZONES_SYS_STACK_SIZE);
} |
This will work: // Necessary include for testing the fix
#include "driver/uart.h"
void setup() {
// for example, start Serial2 - UART2
Serial2.begin(115200);
// right after starting UART2, add this code:
uart_intr_config_t uart_intr = {
.intr_enable_mask = (0x1<<0) | (0x8<<0), // UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
.rx_timeout_thresh = 1,
.txfifo_empty_intr_thresh = 10,
.rxfifo_full_thresh = 112,
};
uart_intr_config((uart_port_t) 2, &uart_intr); // Two is the UART number for Arduino Serial
} Thanks! |
You code shall be like this: #include "driver/uart.h"
void mSystem::testSerialSetup(){
uart_intr_config_t uart_intr = {
.intr_enable_mask = (0x1<<0) | (0x8<<0), // UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
.rx_timeout_thresh = 1,
.txfifo_empty_intr_thresh = 10,
.rxfifo_full_thresh = 112,
};
uart_intr_config((uart_port_t) 1, &uart_intr);
}
void mSystem::setup(){
Serial1.begin(MBUS_BAUD,SERIAL_8N1,MBUS_RX,MBUS_TX);
testSerialSetup();
} |
Thank you for the suggestions. |
As soon as i run this, i get FRAME_TIMEOUT and no communication after that at all. |
Here is how my code relevant part looks like. #include "driver/uart.h"
boolean mSystem::newWriteToBuffer(int index, uint16_t registerAddr){
int byteCount = modules[index]->writeNumBytes*2;
int writeBufferSize = byteCount + 7;
int respDataCount = 0;
boolean isSuccess = false;
uint8_t writeBuffer[writeBufferSize];
uint8_t respBuffer[150];
writeBuffer[0] = uint8_t(modules[index]->address);
writeBuffer[1] = 0x11;
writeBuffer[2] = highByte(registerAddr);
writeBuffer[3] = lowByte(registerAddr);
writeBuffer[4] = highByte(modules[index]->writeNumBytes);
writeBuffer[5] = lowByte(modules[index]->writeNumBytes);
writeBuffer[6] = uint8_t(byteCount);
// put the buffer data into writeBuffer
for(int i = 0; i < modules[index]->writeNumBytes; i++){
writeBuffer[i*2+7] = highByte(modules[index]->writeBuffer[i]);
writeBuffer[i*2+8] = lowByte(modules[index]->writeBuffer[i]);
}
rawWrite(writeBuffer,writeBufferSize);
long startMS = millis();
while ( !Serial1.available() ){
//vTaskDelay(1);
// READ_WRITE_TIMEOUT == 55ms
if( !Serial1.available() && ( (millis() - startMS) > READ_WRITE_TIMEOUT) ){
Serial.printf("FRAME TIMEOUT at millis: %d module id: %d\n",millis(),modules[index]->id);
return false;
}
}
while ( Serial1.available() ){
respBuffer[respDataCount] = Serial1.read();
respDataCount++;
}
//Serial1.flush();
if( respBuffer[0] == uint8_t(modules[index]->address) ){
uint8_t gotCRC_High = respBuffer[respDataCount-2];
uint8_t gotCRC_Low = respBuffer[respDataCount-1];
uint16_t gotCRC = (gotCRC_Low << 8) | gotCRC_High;
uint16_t calcCRC = getCRC_16(respBuffer,respDataCount-2);
if( gotCRC == calcCRC ){
if( respBuffer[1] == 0x90 || respBuffer[1] == 0x83 ){
Serial.printf("\nNew write failed with code: %02X\n",respBuffer[2]);
return false;
}
for(int i = 0; i < modules[index]->readNumBytes; i++){
modules[index]->readBuffer[i] = (respBuffer[i*2+3] << 8) | respBuffer[i*2+4];
}
return true;
}else{
Serial.printf("new write CRC Mismatch. Got: %04X Calculated: %04X\n",gotCRC,calcCRC);
return false;
}
}
return false;
}
void mSystem::writeReadDevice(int index){
if( modules[index] != 0 ){
if( modules[index]->isActive ){
if( modules[index]->onError ){
if( millis() - modules[index]->wentToError >= 60000 ){//modules[index]->onErrorTryCounter >= 1000 ){
modules[index]->onError = false;
}else{
return;
}
}
uint16_t regWriteAddr = getRegWriteAddr(modules[index]->id);
setThermosDefaultDisplay(index,regWriteAddr);
if( !newWriteToBuffer(index,regWriteAddr) ){
modules[index]->failCounter++;
moduleCommError(index);
}else{
modules[index]->failCounter = 0;
modules[index]->onError = false;
checkModuleFaultCode(index);
}
}
}
}
void mSystem::testSerialSetup(){
uart_intr_config_t uart_intr = {
.intr_enable_mask = (0x1<<0) | (0x8<<0), // UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
.rx_timeout_thresh = 1,
.txfifo_empty_intr_thresh = 10,
.rxfifo_full_thresh = 112,
};
uart_intr_config((uart_port_t) 1, &uart_intr);
}
void mSystem::setup(){
Serial1.begin(MBUS_BAUD,SERIAL_8N1,MBUS_RX,MBUS_TX);
testSerialSetup(); // WITHOUT THIS SERIAL SETUP, IT IS WORKING.
}
void mSystem::loop(){
writeReadDevice(deviceIndex);
deviceIndex++;
if( deviceIndex > deviceCounter ){ deviceIndex = 0; }
}
void mBusLoopTask(void* parameter){
for(;;){
hsh_modbus.loop();
vTaskDelay(2);
}
} |
Please try this change: void mSystem::testSerialSetup(){
uart_intr_config_t uart_intr = {
.intr_enable_mask = (0x1<<0) | (0x8<<0), // UART_INTR_RXFIFO_FULL | UART_INTR_RXFIFO_TOUT,
.rx_timeout_thresh = 1,
.txfifo_empty_intr_thresh = 10,
.rxfifo_full_thresh = 1,
};
uart_intr_config((uart_port_t) 1, &uart_intr);
} It will force the driver to read each byte, one by one very fast. |
This causes CRC mismatches in every response new write CRC Mismatch. Got: 1311 Calculated: 807E
new write CRC Mismatch. Got: 110B Calculated: FFFF
new write CRC Mismatch. Got: 943D Calculated: 97BC
new write CRC Mismatch. Got: C03D Calculated: 4C34
new write CRC Mismatch. Got: 94F1 Calculated: 79B2
new write CRC Mismatch. Got: C03D Calculated: 6FDA
new write CRC Mismatch. Got: 943D Calculated: 965C
new write CRC Mismatch. Got: C03D Calculated: E6A2
new write CRC Mismatch. Got: 943D Calculated: C9C7
new write CRC Mismatch. Got: 11C0 Calculated: FFFF
new write CRC Mismatch. Got: 943D Calculated: EECA
new write CRC Mismatch. Got: C03D Calculated: BF9E
Making config from sun times calculated
new write CRC Mismatch. Got: 943D Calculated: 838A
new write CRC Mismatch. Got: 11C0 Calculated: FFFF
new write CRC Mismatch. Got: 0B3D Calculated: 53E9
new write CRC Mismatch. Got: 943D Calculated: 81E6
new write CRC Mismatch. Got: C03D Calculated: EEE3
new write CRC Mismatch. Got: 943D Calculated: 5DCA
new write CRC Mismatch. Got: 1101 Calculated: FFFF
new write CRC Mismatch. Got: 8CF1 Calculated: 522D
new write CRC Mismatch. Got: C0F1 Calculated: 3C9C
new write CRC Mismatch. Got: 0002 Calculated: 7CA5
new write CRC Mismatch. Got: F110 Calculated: 3021
FS - (copy config) Firmware version: 1.34 It is possible that i have to adjust my code in order for this to work. |
With my workaround to put a bigger delay in the loop it was working.
|
Which version of Arduino Core were you using before updating to 2.0.3? Was that the 2.0.0? |
In my PIO ini i did not modified the framework or the platform. [env:esp-wrover-kit]
platform = espressif32
board = esp-wrover-kit
framework = arduino |
Can i jump between versions with this line in the PIO ini?
If i modify the last version number |
I'm not familiar with PIO... I think that maybe @Jason2866 can help on it. |
I tested with
same in every case |
Platformio actual latest release is 5.0 (which is Arduino core 2.0.3) |
Thank you for the suggestion. What is the difference beetween platform_packages, platform, and framework? Framework should be the ESP IDF, but my default PIO configuration for the framework is arduino, but i have seen arduino_espressif too since Arduino framework is top on IDF. |
|
I see. Thank you for the explanation. I will investigate it, it might be a problem from my side. |
@zekageri Hi, can I consider this as solved? |
Sorry. It seems the update solved the problem. Thank you. |
Board
esp32-wrover-e
Device Description
Custom board, using ESP32-Wrover-E ( 16mb flash 8mb external ram )
Hardware Configuration
Ethernet -> ETH_LAN_8720 chip
RTC Module -> DS3231 chip ( i2c )
Transistor for restarting the device -> 2n2222 or similar. ( GPIO 12 )
Two hardware serial UART:
Version
latest master (checkout manually)
IDE Name
Platform IO
Operating System
Windows 10
Flash frequency
80Mhz
PSRAM enabled
yes
Upload speed
115200
Description
Serial1 sometimes does not send out all the bytes and i got a frame timeout in my communication because the modules that my esp communicates doesnt understand the broken message. Happens mostly on web page load or file upload via HTTP.
Communication runs on an available core in a separate task with 115200 baud rate, tries to write as soon as possible.
Sketch
Debug Message
Other Steps to Reproduce
Try to write frequently on serial1.
I have checked existing issues, online documentation and the Troubleshooting Guide
The text was updated successfully, but these errors were encountered: