Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High CPU load #112

Closed
3 tasks done
holgerlembke opened this issue Mar 5, 2025 · 26 comments · Fixed by #114
Closed
3 tasks done

High CPU load #112

holgerlembke opened this issue Mar 5, 2025 · 26 comments · Fixed by #114
Assignees
Labels
topic: infrastructure Related to project infrastructure type: imperfection Perceived defect in any part of project

Comments

@holgerlembke
Copy link

holgerlembke commented Mar 5, 2025

Describe the problem

When opening the serial monitor, serial-monitor.exe creates a relatively high CPU load.

It seems to be related to a certain board type: esp8266 (3.1.2 https://arduino.esp8266.com/stable/package_esp8266com_index.json).

If I select a different board type, the load is gone.

Something is pumping data, as far as I can see, there is a thread continuously moving data.

Image

To reproduce

(see above)

Expected behavior

(see above)

serial-monitor version

0.14.1

Operating system

Windows

Operating system version

10.0.19045.5487

Additional context

Discussion: https://groups.google.com/a/arduino.cc/g/developers/c/PqosKNzu2GU

Issue checklist

  • I searched for previous reports in the issue tracker
  • I verified the problem still occurs when using the latest version
  • My report contains all necessary details
@holgerlembke holgerlembke added the type: imperfection Perceived defect in any part of project label Mar 5, 2025
@cmaglie cmaglie self-assigned this Mar 5, 2025
@holgerlembke
Copy link
Author

Sidenote: I tried to replace the serial-monitor.exe with my own application (written in C#).

But it is not loaded at all. Why? golang magic?

@cmaglie
Copy link
Member

cmaglie commented Mar 7, 2025

Hi @holgerlembke, thanks for the bug report, and sorry I didn't have the time to look at this yet.

But it is not loaded at all. Why? golang magic?

It's a plain executable it should start normally...

@xxxajk
Copy link

xxxajk commented Mar 7, 2025

Certainly there should be some sort of win equivalent to Linux select() which can check for traffic.

What I have done in anything in my own Linux code (in C), is to do a check with select() with a zero timeout, and if no data, a 1mS sleep() (or other processing for single thread, in which case both have to fail for the sleep()), which always lowered the CPU to nearly zero, and never lost any data, since it can handle 1mS of burst in a single gulp, even at very high speeds.

@holgerlembke
Copy link
Author

holgerlembke commented Mar 7, 2025

no need to hurry, it is not that important.

Just revisited my code... Fixed some very stupid errors, now it is loaded. I think I will spend some time of the weekend with recreating that thing in c#.

@holgerlembke
Copy link
Author

@xxxajk I have no clue about golang, but I would expect a read-function from stdio to pass it to the OS and that should let it wait with zero load. Isn't that so?

@xxxajk
Copy link

xxxajk commented Mar 7, 2025

@xxxajk I have no clue about golang, but I would expect a read-function from stdio to pass it to the OS and that should let it wait with zero load. Isn't that so?

Unsure, but I do know that in order to make programs not eat cycles on linux in C, that using select and sleep on fails was able for some code that I wrote back in about 1995 running on a 386sx totally fine, even when the data was streaming in from 10baseT bidirectionally to/from high-speed UART. Never wrote anything in golang either, but then I'm kind of a control freak when it comes to what's actually running on the CPU, to the point that I have mixed C and ASM a lot...

@per1234
Copy link
Contributor

per1234 commented Mar 8, 2025

Hi @holgerlembke.

If I select a different board type, the load is gone.

I'm not sure I understand what you mean by "select a different board type".

Do you mean that you are only selecting a different board from Arduino IDE's Tools > Board menu, without changing the Tools > Port menu selection? If yes, please tell us which exact Tools > Board menu item you have selected when the fault occurs, and which exact Tools > Board menu item you have selected when the fault does not occur.

Or do you mean that you connect a different board to your computer and select that board and port instead of the board and port of the ESP8266-based board? If yes, please tell us exactly which board you are using when the fault occurs, and which one you are using when the fault does not occur.

@holgerlembke
Copy link
Author

@per1234

I just rechecked it and my observation was wrong. Whatever board I select, load is there.

I edited the bug report.

@per1234
Copy link
Contributor

per1234 commented Mar 8, 2025

Thanks for the clarification. Please tell us which USB to serial bridge chip your Arduino board has. This is a black chip near the USB socket.

The chip will usually be identified by writing on the top. This might say something like "WCH CH340G" or "SILABS CP2102".

Examples:

WCH CH340

CH340 IC

📷

SparkFun - CC BY-SA 4.0 (cropped)

There are some boards on the market that have a USB chip with the same IC package as the WCH CH340, and which are identified by the computer as a CH340. However, these chips don't have the "WCH CH340C ..." labeling like you see on the chip in the picture above:

unlabeled SOIC-16 IC

📷

NodeMCU Lua Lol1n V3.jpg by Popolon - CC BY-SA 4.0 (cropped)

Silicon Labs CP2102

CP2102

📷

SparkFun - CC BY-SA 4.0 (cropped)


If it isn't clear, alternatively you can provide the link to where you bought the board from and we'll see if we can determine the chip from the product listing.

@holgerlembke
Copy link
Author

holgerlembke commented Mar 8, 2025

Device manager says CH340. Chips says CH340C.

Just added a board with a cp9102. It does NOT show the high load.

On a sidenote, my own C# serial-monitor shows traffic even after closing the serial port from the IDE. Not sure about how, why and when... could be my error, needs more research

@per1234
Copy link
Contributor

per1234 commented Mar 8, 2025

Is your board running a sketch that causes it to produce serial output?

Note that the ROM bootloader of the ESP8266 produces output on startup or reset, so in cases where either the sketch code or electrical conditions (e.g., insufficient power supply) causes the board to continuously crash/reset, it is also possible that the board can produce continuous serial output even if there isn't anything in the sketch code that explicitly does so. In this case, you should see the output in the Arduino IDE Serial Monitor if you select "74880" from the baud rate menu in the Serial Monitor panel (the bootloader uses this baud rate even if the sketch is configured for a different rate).

@holgerlembke
Copy link
Author

This

void setup() {}
void loop() {}

produces the load.

@per1234
Copy link
Contributor

per1234 commented Mar 9, 2025

If you press and hold the reset button on the board with the CH340 (or ground the RST pin if your board doesn't have a reset button), does the high serial-monitor.exe process load continue, or does it drop off to a level similar to what you see from the board with the CP9102?

@holgerlembke
Copy link
Author

Yes. Load stays high.

Btw: Board is an esp01s plus those cheap rusty (!) usb-adapters. Some reworked to 4mb. Does not change anything, whatever I use. (That was my first try: grab some original adapter+esp01s.)

Image

@holgerlembke
Copy link
Author

Soooo.

This is a "very early", "just after finishing it", "I'm happy", look at the problem response.

https://github.com/holgerlembke/serial-monitor is a working rewrite in C# of the serial-monitor. It works, at least I see incoming and outgoing stuff. Internal error handling might not be satisfying.

And there is no load.

@cmaglie
Copy link
Member

cmaglie commented Mar 11, 2025

I've been able to reproduce the bug with an esp8266 d1-mini clone. It seems that the CH340 driver does not honor the timeout set on the serial port (that should be infinity), instead returns from the ReadFile a zero-len result without any blocking, this leads to a spin loop that saturates a CPU core. This is CH340-specific, a lot of esp8266-based boards use it, but generally speaking, is not a problem of the esp8266 platform.

I've reproduced the problem with a minimal example in go:

package main

import (
	"fmt"
	"io"
	"os"

	"go.bug.st/serial"
)

func main() {
	if len(os.Args) < 2 {
		fmt.Println("Usage: term <PORT>")
		os.Exit(1)
	}
	portname := os.Args[1]
	port, err := serial.Open(portname, &serial.Mode{
		BaudRate: 9600,
		DataBits: 8,
		Parity:   serial.NoParity,
		StopBits: 1,
		InitialStatusBits: &serial.ModemOutputBits{
			RTS: true,
			DTR: true,
		},
	})
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(2)
	}
	buff := make([]byte, 1024)
	for {
		n, err := port.Read(buff)
		fmt.Println(">>", n, err)
		if err == io.EOF {
			break
		} else if err != nil {
			fmt.Println("Error:", err)
			os.Exit(0)
		} else if n > 0 {
			fmt.Println("READ:", buff[:n])
		}
	}
}

That prints:

>> 0 <nil>
>> 0 <nil>
>> 0 <nil>
>> 0 <nil>
>> 0 <nil>
...

Interestingly, I'm not able to reproduce the problem with python pyserial, so pyserial is doing something that makes the CH340 work properly, I'll compare the win32 calls made by pyserial to see if I found something.

This is a "very early", "just after finishing it", "I'm happy", look at the problem response.

https://github.com/holgerlembke/serial-monitor is a working rewrite in C# of the serial-monitor. It works, at least I see incoming and outgoing stuff. Internal error handling might not be satisfying.

Cool! Nice work on that ;-)

This makes me particularly happy because:

  • You found a way to solve the problem by yourself (in a very DIY way)
  • It means that the pluggable-monitor specification I wrote years ago makes sense :-)

@holgerlembke
Copy link
Author

Nice finding with the CH340. At least in .NET 6/8s System.IO.Ports everything looks fine and event driven, no wait needed. As usual, the TCP connection is not event driven, so I ended up with some sleep(). What makes it more complicated is no real good way to monitor the disconnect/connect of the USB port (other than digging deep into the hardware event system, I do not wanted to do...). And I get no exception. Another miracle for another day.

More confusing, I find this log:

2025-03-11 16:29:56.595 RR:Start
2025-03-11 16:29:56.618 >>HELLO 1 "arduino-cli 1.1.1"
2025-03-11 16:29:56.622 >>DESCRIBE
2025-03-11 16:29:56.623 >>QUIT
2025-03-11 16:29:56.596 RR:Start
2025-03-11 16:29:57.129 >>HELLO 1 "arduino-cli 1.1.1"
2025-03-11 16:29:57.133 >>DESCRIBE
2025-03-11 16:29:57.133 >>QUIT
2025-03-11 16:29:57.182 RR:Start
2025-03-11 16:29:57.206 >>HELLO 1 "arduino-cli 1.1.1"
2025-03-11 16:29:57.210 >>DESCRIBE
2025-03-11 16:29:57.210 >>CONFIGURE bits 8
2025-03-11 16:29:57.211 >>CONFIGURE parity None
2025-03-11 16:29:57.211 >>CONFIGURE stop_bits 1
2025-03-11 16:29:57.212 >>CONFIGURE baudrate 115200
2025-03-11 16:29:57.213 >>CONFIGURE dtr off
2025-03-11 16:29:57.213 >>CONFIGURE rts off
2025-03-11 16:29:57.214 >>OPEN 127.0.0.1:29703 COM4
2025-03-11 16:29:57.216 RR:pumpe starts
2025-03-11 16:29:57.258 RR:pumpe runs
2025-03-11 16:30:17.161 >>CLOSE
2025-03-11 16:30:17.188 RR:Pumpe died
2025-03-11 16:30:17.188 RR:End
2025-03-11 16:30:32.566 RR:Start
2025-03-11 16:30:32.586 >>HELLO 1 "arduino-cli 1.1.1"
2025-03-11 16:30:32.591 >>DESCRIBE
2025-03-11 16:30:32.592 >>QUIT
2025-03-11 16:30:32.650 RR:Start
2025-03-11 16:30:32.673 >>HELLO 1 "arduino-cli 1.1.1"
2025-03-11 16:30:32.677 >>DESCRIBE
2025-03-11 16:30:32.678 >>CONFIGURE bits 8
2025-03-11 16:30:32.679 >>CONFIGURE parity None
2025-03-11 16:30:32.679 >>CONFIGURE stop_bits 1
2025-03-11 16:30:32.680 >>CONFIGURE baudrate 115200
2025-03-11 16:30:32.680 >>CONFIGURE dtr off
2025-03-11 16:30:32.681 >>CONFIGURE rts off
2025-03-11 16:30:32.681 >>OPEN 127.0.0.1:29720 COM4
2025-03-11 16:30:32.683 RR:pumpe starts
2025-03-11 16:30:32.719 RR:pumpe runs
2025-03-11 16:30:42.300 >>CLOSE
2025-03-11 16:30:42.328 RR:Pumpe died
2025-03-11 16:30:42.329 RR:End

It is a simple user interaction sequence: open serial monitor and click upload. So disconnect, compile (:17-:32) and reconnect.

Every time the "RR:Start" indicates a serial-monitor-process is started. I have no idea, why I see those short hello-describe-quit sequences. Especially at the beginning, just two of those cycles until it starts with real connection phase. I feel innocent, I do always the same stuff. :)

Most annoying (Ok, I'm used to that stuff, and it is not so bad) is the complication, that stdin.Read() is a blocking call and there is no async way to do it with stdin. So it needs an extra thread to monitor just stdin.Read()...

@holgerlembke
Copy link
Author

And another log.

2025-03-13 21:55:04.809 RR:Start
2025-03-13 21:55:04.811 RR:Start
2025-03-13 21:55:04.832 >>HELLO 1 "arduino-cli 1.1.1"
2025-03-13 21:55:04.832 >>HELLO 1 "arduino-cli 1.1.1"
2025-03-13 21:55:04.835 >>DESCRIBE
2025-03-13 21:55:04.836 >>DESCRIBE
2025-03-13 21:55:04.836 >>QUIT
2025-03-13 21:55:04.836 >>QUIT
2025-03-13 21:55:04.884 RR:Start
2025-03-13 21:55:04.905 >>HELLO 1 "arduino-cli 1.1.1"
2025-03-13 21:55:04.910 >>DESCRIBE
2025-03-13 21:55:04.911 >>CONFIGURE baudrate 115200
2025-03-13 21:55:04.911 >>CONFIGURE bits 8
2025-03-13 21:55:04.911 >>CONFIGURE parity None
2025-03-13 21:55:04.912 >>CONFIGURE stop_bits 1
2025-03-13 21:55:04.912 >>CONFIGURE dtr on
2025-03-13 21:55:04.912 >>CONFIGURE rts on
2025-03-13 21:55:04.913 >>OPEN 127.0.0.1:18475 COM13
2025-03-13 21:55:04.913 RR:pumpe init
2025-03-13 21:55:04.915 RR:pumpe starts
2025-03-13 21:55:04.952 RR:pumpe runs
2025-03-13 21:55:44.695 >>CLOSE
2025-03-13 21:55:44.726 RR:Pumpe died
2025-03-13 21:55:44.727 RR:End

Originally I wanted to test the hibernate-problem (after wake up from hibernate, the IDE thinks the serial port is connected, but it does not show any incoming data. It needs a close+open cyclus to start logging again.)

So, as you see, serial-monitor is closed at 21:55:44 and never opened again. So at least that explains that. It is not the serial-monitor, it is the IDE forgetting something.

More confusing are the duplicate lines at 21:55:04 etc. It seems like two versions are started. And then, at 21:55:04.884 the IDE starts the real setup ending in an connection.

Reason might be that I have 3 IDE instances open. So the "i did not click on that"-IDEs still try to get the serial params... All in all this seem superfluous because the DESCRIBE answer content is static.

But I'm not the designer. There might be reasons. So in case, I'm just reporting this.

@cmaglie
Copy link
Member

cmaglie commented Mar 17, 2025

More confusing are the duplicate lines at 21:55:04 etc. It seems like two versions are started. And then, at 21:55:04.884 the IDE starts the real setup ending in an connection.

I think it's the IDE that is querying the list of parameters supported from the monitor.
Consider that the serial monitor is just an implementation of a "pluggable" monitor, other kinds of monitors may be implemented in the future with different parameters, maybe not the most efficient way to do so, but in any case, everything looks normal.

@cmaglie
Copy link
Member

cmaglie commented Mar 17, 2025

Back to the CH340 I think I found the reason of the issue, this is the patch I prepared for the serial library in golang with a bit of explanation, it really looks like a bug in the CH340.
After merging the PR on the go-serial library, I will update the serial-monitor tool too.

@cmaglie
Copy link
Member

cmaglie commented Mar 17, 2025

Here are some test builds (at the bottom of the page):
https://github.com/arduino/serial-monitor/actions/runs/13904785383

@cmaglie
Copy link
Member

cmaglie commented Mar 18, 2025

Ok, I've tested again from a clean install and the issue seems fixed, I'm going to release an update to the serial monitor.

Image

cmaglie added a commit that referenced this issue Mar 18, 2025
This upgrade should fix high CPU load with the CH340.
#112
cmaglie added a commit that referenced this issue Mar 18, 2025
This upgrade should fix high CPU load with the CH340.
#112
cmaglie added a commit that referenced this issue Mar 18, 2025
…rial converters) (#114)

* Upgrade go-serial library

This upgrade should fix high CPU load with the CH340.
#112

* Tidy up deps
@holgerlembke
Copy link
Author

Thanks for the work!

I'll stay with my C# solution, because it catches the esp exception on the fly and passes them instantly to ESPEDfGK

@cmaglie
Copy link
Member

cmaglie commented Mar 19, 2025

I'll stay with my C# solution, because it catches the esp exception on the

The serial-monitor 0.15.0 has been released, and the Arduino IDE will upgrade it automatically at the next check of the package_index. If you want to continue to use it you will need to install your serial-monitor.exe replacement again (not a big deal, just wanted to let you know).

@holgerlembke
Copy link
Author

Oh, that reminds me to mention, that I have a 0.10.0, 0.11.0, 0.12.0, 0.13.0 and 0.14.1. Each of it with 3.5 MB going up...

@cmaglie
Copy link
Member

cmaglie commented Mar 19, 2025

You can remove the old ones

@per1234 per1234 added the topic: infrastructure Related to project infrastructure label Apr 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: infrastructure Related to project infrastructure type: imperfection Perceived defect in any part of project
Projects
None yet
4 participants