Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ethernet: High ping on Portenta H7 compared to MKR Shield #955

Open
razvanphp opened this issue Sep 20, 2024 · 8 comments
Open

Ethernet: High ping on Portenta H7 compared to MKR Shield #955

razvanphp opened this issue Sep 20, 2024 · 8 comments

Comments

@razvanphp
Copy link

After migrating our app from MKR WiFi 1010 + ETH shield --> Portenta H7 + Vision Shield, we noticed increased and uneven/unstable ping response times.

Here it is how it looks like on MKR:

64 bytes from 172.30.0.2: icmp_seq=1389 ttl=128 time=0.465 ms
64 bytes from 172.30.0.2: icmp_seq=1390 ttl=128 time=0.315 ms
64 bytes from 172.30.0.2: icmp_seq=1391 ttl=128 time=0.503 ms
64 bytes from 172.30.0.2: icmp_seq=1392 ttl=128 time=0.279 ms
64 bytes from 172.30.0.2: icmp_seq=1393 ttl=128 time=0.657 ms
64 bytes from 172.30.0.2: icmp_seq=1394 ttl=128 time=0.403 ms
64 bytes from 172.30.0.2: icmp_seq=1395 ttl=128 time=0.374 ms
64 bytes from 172.30.0.2: icmp_seq=1396 ttl=128 time=0.304 ms
64 bytes from 172.30.0.2: icmp_seq=1397 ttl=128 time=0.430 ms
64 bytes from 172.30.0.2: icmp_seq=1398 ttl=128 time=0.474 ms
64 bytes from 172.30.0.2: icmp_seq=1399 ttl=128 time=0.463 ms
64 bytes from 172.30.0.2: icmp_seq=1400 ttl=128 time=0.386 ms
64 bytes from 172.30.0.2: icmp_seq=1401 ttl=128 time=0.398 ms
64 bytes from 172.30.0.2: icmp_seq=1402 ttl=128 time=0.399 ms
64 bytes from 172.30.0.2: icmp_seq=1403 ttl=128 time=0.322 ms
64 bytes from 172.30.0.2: icmp_seq=1404 ttl=128 time=0.464 ms
64 bytes from 172.30.0.2: icmp_seq=1405 ttl=128 time=0.394 ms
64 bytes from 172.30.0.2: icmp_seq=1406 ttl=128 time=0.417 ms
64 bytes from 172.30.0.2: icmp_seq=1407 ttl=128 time=0.609 ms
64 bytes from 172.30.0.2: icmp_seq=1408 ttl=128 time=0.429 ms
64 bytes from 172.30.0.2: icmp_seq=1409 ttl=128 time=0.534 ms
64 bytes from 172.30.0.2: icmp_seq=1410 ttl=128 time=0.382 ms
64 bytes from 172.30.0.2: icmp_seq=1411 ttl=128 time=0.621 ms
64 bytes from 172.30.0.2: icmp_seq=1412 ttl=128 time=0.339 ms
64 bytes from 172.30.0.2: icmp_seq=1413 ttl=128 time=0.638 ms
64 bytes from 172.30.0.2: icmp_seq=1414 ttl=128 time=0.335 ms
64 bytes from 172.30.0.2: icmp_seq=1415 ttl=128 time=0.410 ms
64 bytes from 172.30.0.2: icmp_seq=1416 ttl=128 time=0.217 ms
64 bytes from 172.30.0.2: icmp_seq=1417 ttl=128 time=0.463 ms
64 bytes from 172.30.0.2: icmp_seq=1418 ttl=128 time=0.372 ms
64 bytes from 172.30.0.2: icmp_seq=1419 ttl=128 time=0.391 ms
64 bytes from 172.30.0.2: icmp_seq=1420 ttl=128 time=0.603 ms
64 bytes from 172.30.0.2: icmp_seq=1421 ttl=128 time=0.300 ms
64 bytes from 172.30.0.2: icmp_seq=1422 ttl=128 time=0.512 ms
64 bytes from 172.30.0.2: icmp_seq=1423 ttl=128 time=0.378 ms
64 bytes from 172.30.0.2: icmp_seq=1424 ttl=128 time=0.509 ms
64 bytes from 172.30.0.2: icmp_seq=1425 ttl=128 time=0.473 ms
64 bytes from 172.30.0.2: icmp_seq=1426 ttl=128 time=0.345 ms
64 bytes from 172.30.0.2: icmp_seq=1427 ttl=128 time=0.344 ms

And here it is on Portenta H7:

64 bytes from 172.30.0.3: icmp_seq=1 ttl=255 time=7.47 ms
64 bytes from 172.30.0.3: icmp_seq=2 ttl=255 time=2.37 ms
64 bytes from 172.30.0.3: icmp_seq=3 ttl=255 time=1.69 ms
64 bytes from 172.30.0.3: icmp_seq=4 ttl=255 time=4.95 ms
64 bytes from 172.30.0.3: icmp_seq=5 ttl=255 time=4.92 ms
64 bytes from 172.30.0.3: icmp_seq=6 ttl=255 time=4.93 ms
64 bytes from 172.30.0.3: icmp_seq=7 ttl=255 time=0.404 ms
64 bytes from 172.30.0.3: icmp_seq=8 ttl=255 time=1.74 ms
64 bytes from 172.30.0.3: icmp_seq=9 ttl=255 time=0.145 ms
64 bytes from 172.30.0.3: icmp_seq=10 ttl=255 time=1.75 ms
64 bytes from 172.30.0.3: icmp_seq=11 ttl=255 time=0.710 ms
64 bytes from 172.30.0.3: icmp_seq=12 ttl=255 time=1.76 ms
64 bytes from 172.30.0.3: icmp_seq=13 ttl=255 time=0.686 ms
64 bytes from 172.30.0.3: icmp_seq=14 ttl=255 time=1.75 ms
64 bytes from 172.30.0.3: icmp_seq=15 ttl=255 time=0.186 ms
64 bytes from 172.30.0.3: icmp_seq=16 ttl=255 time=1.77 ms
64 bytes from 172.30.0.3: icmp_seq=17 ttl=255 time=0.583 ms
64 bytes from 172.30.0.3: icmp_seq=18 ttl=255 time=1.82 ms
64 bytes from 172.30.0.3: icmp_seq=19 ttl=255 time=4.92 ms
64 bytes from 172.30.0.3: icmp_seq=20 ttl=255 time=2.94 ms
64 bytes from 172.30.0.3: icmp_seq=21 ttl=255 time=1.84 ms
64 bytes from 172.30.0.3: icmp_seq=22 ttl=255 time=0.561 ms
64 bytes from 172.30.0.3: icmp_seq=23 ttl=255 time=1.89 ms
64 bytes from 172.30.0.3: icmp_seq=24 ttl=255 time=0.788 ms
64 bytes from 172.30.0.3: icmp_seq=25 ttl=255 time=1.89 ms
64 bytes from 172.30.0.3: icmp_seq=26 ttl=255 time=4.95 ms
64 bytes from 172.30.0.3: icmp_seq=27 ttl=255 time=4.07 ms
64 bytes from 172.30.0.3: icmp_seq=28 ttl=255 time=3.96 ms
64 bytes from 172.30.0.3: icmp_seq=29 ttl=255 time=2.94 ms
64 bytes from 172.30.0.3: icmp_seq=30 ttl=255 time=1.65 ms
64 bytes from 172.30.0.3: icmp_seq=31 ttl=255 time=4.93 ms

I suspect this is impacting our app's performance, since we need sub ms processing of the data received over ethernet, but not sure how to approach this problem.

Userland code is basically inexistent, as I'm testing just with this:

#include <PortentaEthernet.h>
#include <Ethernet.h>

EthernetClient ethClient;

byte mac[] = { 0xA8, 0x61, 0x0A, 0xAE, 0x44, 0xE0 };
IPAddress ip(172, 30, 0, 3);

void setup() {
  Ethernet.begin(mac, ip);
}

void loop() {}

Initially I though this is because MKR ETH Shield uses W5500 chipset with a built-in hardware TCP/IP stack, while Portenta H7 uses LAN8742AI, an Ethernet PHY that requires a software TCP/IP stack like LwIP.

But today I tried to flash the exact same board with micropython and ping times look perfect:

64 bytes from 192.168.10.252: icmp_seq=10 ttl=255 time=0.075 ms
64 bytes from 192.168.10.252: icmp_seq=11 ttl=255 time=0.086 ms
64 bytes from 192.168.10.252: icmp_seq=12 ttl=255 time=0.082 ms
64 bytes from 192.168.10.252: icmp_seq=13 ttl=255 time=0.069 ms
64 bytes from 192.168.10.252: icmp_seq=14 ttl=255 time=0.072 ms
64 bytes from 192.168.10.252: icmp_seq=15 ttl=255 time=0.092 ms
64 bytes from 192.168.10.252: icmp_seq=16 ttl=255 time=0.094 ms
64 bytes from 192.168.10.252: icmp_seq=17 ttl=255 time=0.095 ms
64 bytes from 192.168.10.252: icmp_seq=18 ttl=255 time=0.085 ms
64 bytes from 192.168.10.252: icmp_seq=19 ttl=255 time=0.067 ms
64 bytes from 192.168.10.252: icmp_seq=20 ttl=255 time=0.094 ms
64 bytes from 192.168.10.252: icmp_seq=21 ttl=255 time=0.071 ms
64 bytes from 192.168.10.252: icmp_seq=22 ttl=255 time=0.086 ms
64 bytes from 192.168.10.252: icmp_seq=23 ttl=255 time=0.084 ms
64 bytes from 192.168.10.252: icmp_seq=24 ttl=255 time=0.069 ms
64 bytes from 192.168.10.252: icmp_seq=25 ttl=255 time=0.105 ms
64 bytes from 192.168.10.252: icmp_seq=26 ttl=255 time=0.084 ms
64 bytes from 192.168.10.252: icmp_seq=27 ttl=255 time=0.095 ms

Are there any settings or optimisations that we could try in LwIP? Seems the issue is not in HW.

Thank you!
R

PS: opened initially in arduino-libraries/Ethernet#275
@razvanphp
Copy link
Author

razvanphp commented Nov 7, 2024 •
edited
Loading

anybody any idea?

I tried increasing some values in mbed_config.h, similar to micropython ones, including LWIP thread priority, ping improved a little bit, but still has spikes of 2-5 ms.

#define MBED_CONF_LWIP_ADDR_TIMEOUT                                             5                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_ADDR_TIMEOUT_MODE                                        1                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_DEBUG_ENABLED                                            0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_DEFAULT_THREAD_STACKSIZE                                 512                                                                                              // set by library:lwip
#define MBED_CONF_LWIP_DHCP_TIMEOUT                                             60                                                                                               // set by library:lwip
#define MBED_CONF_LWIP_ENABLE_PPP_TRACE                                         0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_ETHERNET_ENABLED                                         1                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_IPV4_ENABLED                                             1                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_IPV6_ENABLED                                             0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_IP_VER_PREF                                              4                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_L3IP_ENABLED                                             0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_MBOX_SIZE                                                8                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_MEMP_NUM_TCPIP_MSG_INPKT                                 8                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_MEMP_NUM_TCP_SEG                                         32                                                                                               // set by library:lwip
#define MBED_CONF_LWIP_MEM_SIZE                                                 16000                                                                                            // set by library:lwip[PORTENTA_H7]
#define MBED_CONF_LWIP_ND6_QUEUEING                                             0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_ND6_RDNSS_MAX_DNS_SERVERS                                0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_NETBUF_RECVINFO_ENABLED                                  0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_NUM_NETBUF                                               8                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_NUM_PBUF                                                 8                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_PBUF_POOL_SIZE                                           16                                                                                               // set by library:lwip
#define MBED_CONF_LWIP_PPP_ENABLED                                              0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_PPP_IPV4_ENABLED                                         0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_PPP_IPV6_ENABLED                                         0                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_PPP_THREAD_STACKSIZE                                     768                                                                                              // set by library:lwip
#define MBED_CONF_LWIP_PRESENT                                                  1                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_RAW_SOCKET_ENABLED                                       1                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_SOCKET_MAX                                               4                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_TCPIP_THREAD_PRIORITY                                    osPriorityRealtime                                                                               // set by library:lwip
#define MBED_CONF_LWIP_TCPIP_THREAD_STACKSIZE                                   4096                                                                                             // set by library:lwip
#define MBED_CONF_LWIP_TCP_CLOSE_TIMEOUT                                        1000                                                                                             // set by library:lwip
#define MBED_CONF_LWIP_TCP_ENABLED                                              1                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_TCP_MAXRTX                                               6                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_TCP_MSS                                                  1460                                                                                             // set by library:lwip
#define MBED_CONF_LWIP_TCP_SERVER_MAX                                           4                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_TCP_SND_BUF                                              (8 * TCP_MSS)                                                                                    // set by library:lwip
#define MBED_CONF_LWIP_TCP_SOCKET_MAX                                           4                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_TCP_SYNMAXRTX                                            6                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_TCP_WND                                                  (8 * TCP_MSS)                                                                                    // set by library:lwip
#define MBED_CONF_LWIP_UDP_SOCKET_MAX                                           4                                                                                                // set by library:lwip
#define MBED_CONF_LWIP_USE_MBED_TRACE                                           0

@pennam
Copy link
Contributor

pennam commented Nov 8, 2024

@razvanphp are you getting the same results using WiFi? or WiFi is working good? If it is i would say the root cause is probably at lower level than LwIP.

@multiplemonomials
Copy link

Hi! I just completed rewriting the entire STM32H7 ethernet driver in Mbed. I think there's a good chance it will solve this issue, as the existing one was held together with the digital equivalent of glue and duct tape.

PR here: mbed-ce/mbed-os#438

@razvanphp
Copy link
Author

Did you managed to get better latency with the new driver? Any idea if I can test your branch before mbed merges it?

Not sure how Arduino layer plays on top of mbed, now that mbed is deprecated officially.

@multiplemonomials
Copy link

What board(s) do you have? Currrntly Mbed can be built directly (no arduino) for Portenta H7 and Giga. I also have a WIP version of this arduino core based on Mbed CE, I can try to set it up to work for Portenta

@razvanphp
Copy link
Author

razvanphp commented Mar 17, 2025 •
edited
Loading

Portenta H7. I could replace some files on my machine and clear the cache, it should load it on a fresh build, if it is compatible with arduino ETH API that is built on-top. Would love to test it. Thank you!

@multiplemonomials
Copy link

Yeah if you manually apply my patch, it should work. Also I will work on preparing a version of the mbed ce arduino core for you.

@multiplemonomials
Copy link

multiplemonomials commented Apr 2, 2025 •
edited
Loading

Good news! I was able to prepare a Portenta release of the Mbed CE arduino core. Still have more checking to do, but I was able to confirm that it at least boots, flashes code, and can talk over Ethernet. Please give it a try if you have a chance! https://github.com/mbed-ce/ArduinoCore-mbed-ce/releases/tag/v1.2.0

If everything works, this should be a drop in replacement for the regular Arduino code that is compatible with the same code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@multiplemonomials @razvanphp @pennam