-
Notifications
You must be signed in to change notification settings - Fork 5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Data transfer over USB becomes unreliable after upgrading to v5.4.83-v71+ #4084
Comments
I have managed to narrow down the issue to the particular github commit that causes it. If I do: If I do: The (for me) troublesome commit say something about "Fix USB init for 2835 bootcode.bin". This gives a solution to our issue for now, because the functional Oct 8, 2020 commit is new enough to work on the new pi4 hardware, and old enough not to have the network freeze issue...but I would be surprised if it is not affecting others. |
FYI: the issue is also present in the latest commit I tried: Jan 18, 2021: commit number: 246f5ead8d2d017fd2734d79e1f9a606fb65b72c |
@timg236 does "Fix USB init for 2835 bootcode.bin" affect start.elf and could it have an impact on later USB use? |
Not as far as I can tell. This could would only be used if rpiboot (usb device boot) is used. The change was to the ensure that soft disconnect was still called between bootcode.bin and start.elf on a Pi3 in USb mode. |
There's a separate commit around then which might be squashed. Please could you try 'sudo rpi-update 98d907955a74ba1833f9c31b1fe11b237f8ba599' and apply the attached experimental (untested fix) which disables some USB PHY reset code. |
I tested the files, but unfortunately it did not fix the issue. |
I've run the attached python scripts plus some other SSH and HTTP download soak tests with the latest APT FW release with a CM4 + Pi4. No errors observed. There's another theoretical change attached |
I managed to get a failure after 'pursuading' RNDIS to work on my Windows 10 machine. It's possible that this won't fail on CM4 + IO board. Attached is a possible fix candidate which passed 1000 iterations a couple of times so I'll leave it running over the weekend. I think v2.zip won't fail either but this more targetted |
Thanks so much for the rapid response. I've run some more tests: On clean install:
So the 4084-v5.zip files fix the problem on a fresh install, which is great. I realised that the install I was testing with was quite modified and it seems this enhances the problem. The 4084-v5.zip does not fix it, even though it does work on the clean install. On modified install:
I listed the changes that seem most relevant to the usb:
So for now (since I need my modifications), I'm using the old 50b... commit as a solution. I realise that this could be becoming an ever more challenging issue to debug, but if you think it could be important I'm happy to help if I can. |
Thanks for the test results. V5 restores a single PHY register setup operation which shouldn't be relevant on BCM2711, unfortunately, that code is pretty ancient so getting a definitive answer may take a while. |
firmware: usb: Fix dropouts with USB ethernet gadget See: raspberrypi/linux#4084 firmware: imx477: Allow long exposures for the binned modes. See: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=297521
firmware: usb: Fix dropouts with USB ethernet gadget See: raspberrypi/linux#4084 firmware: imx477: Allow long exposures for the binned modes. See: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=297521
The firmware fix is now available in the latest rpi-update kernel. I've been testing with the 5.10 kernel which is now the default rpi-update kernel. The APT packages are switching to 5.10 very soon. |
Deleted off topic comment |
- firmware: isp: Fix handling of different YUV colour spaces - firmware: poe_hat: Actually close the I2C handle - firmware: platform: Define DVFS modes and change default to be fixed AVS voltage - firmware: arm_loader: Auto-select 64-bit for kernel8.img See: #1193 - firmware: hdmi: Throttle auto-i2c register writes to avoid PWM audio underrun - firmware: platform: Define DVFS modes and change default to be fixed AVS voltage - firmware: arm_loader: Auto-select 64-bit for kernel8.img See: #1193 - firmware: hdmi: Throttle auto-i2c register writes to avoid PWM audio underrun - firmware: video_decode lockup handling - firmware: isp: Initialise extras to avoid vpitch being random - firmware: usb: Fix dropouts with USB ethernet gadget See: raspberrypi/linux#4084 - firmware: imx477: Allow long exposures for the binned modes. See: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=297521 - firmware: arm_dispmanx: Use ALPHA_MIX flag See: https://www.raspberrypi.org/forums/viewtopic.php?t=300769 - firmware: power: Refactor the interface to the PMICs - firmware: platform: vl805: Get BAR2 address from PCIe BAR2 registers - firmware: arm_loader: Return all borrowed DMA channels See: #1541 - firmware: hdmi_2711: Rework I2C driver to NOT use the AUTO-I2C block - firmware: gencmd: Allow groups of clocks/plls to be read together - firmware: power: Fix DA9090 under-voltage detection - firmware: NVME boot support - firmware: brfs: Fix USB bulk-read in start.elf See: Hexxeh/rpi-firmware#258 - firmware: hdmi_2711_i2c: Correct handling of start/stop codes for long read See: #1548 - firmware: video_decode: For VC1/WMV with no signalled header bytes, use start of 1st buffer See: raspberrypi/linux#4113 - firmware: vl805: Remove redundant log statement and fix warning - firmware: power: Fix DA9090 ADC1 register definition - firmware: arm_loader: Only report clocks arm has set, not siblings - firmware: arm_loader: Don't report clocks set as turbo side effect of arm clock - firmware: arm_loader: 2711: gpu clocks are not dependant - firmware: platform: Need to clear cached versions of get_max_clock_internal vars - firmware: Move core to PLLA and support accurate clk108 See: xbmc/xbmc#19263 - firmware: board_info: Separate memory size from OTP field encoding - firmware: power: Swap DA9090 ADC assignments to match XR77004 - firmware: board-info: Fix memsize on 3B+ - firmware: vcfw/power: Add a new latch for power_pad_control See: #1552 - firmware: arm_loader: kernel_old=1 should force kernel_address=0 See: #1561 - firmware: scalerlib: Fix offset applied to x coordinate of YUV10COL image See: https://forum.kodi.tv/showthread.php?tid=361164&pid=3024654#pid3024654 - firmware: isp: Ensure the VRF is locked when setting up video colour denoise See: raspberrypi/rpicam-apps#19 - firmware: isp: Remove custom EV mappings from camera tunings - firmware: Add support for board-type=0xXX conditional filters in bootloader, bootcode and firmware - firmware: Two UART1 patches See: #1566 - firmware: Pi400: Reduce MII clock freq when probing ethernet PHY - firmware: platform: Remove build-time constant for MICROVOLTS_PER_PIP - firmware: dt-blob.dts: Correct HDMI HPD and EMMC_ENABLE for CM4 See: https://www.raspberrypi.org/forums/viewtopic.php?f=29&p=1858516 - firmware: vcfw/hdmi: CUSTOM modes used for FKMS didn't set RGB quant range correctly See: #1580 - firmware: PoE+ HAT support See: raspberrypi/linux#4367 - firmware: arm_loader: Use Pi4 bootloader MAC_ADDRESS if set - firmware: platform: Apply ARM thermal throttling rules on BCM2711 - firmware: bcm_host: Recognise all Pi 4 variants, add BCM2711 See: raspberrypi/userland#695 - firmware: video_decode: Use the ISP instead of vc_image_convert - firmware: hdmi-2711: Wait for HDMI hardware scheduler to activate in HDMI mode - arm_loader: Add message to release firmware framebuffer - firmware: arm_loader: Add rng-seed DT property See: #1595 - firmware: isp: Set the YUV420/YVU420 format stride to 64 byte - firmware: Revert: video_decode: Use the ISP instead of vc_image_convert - firmware: cec: Avoid sending messages with kms See: raspberrypi/linux#4460 - firmware: hdmi_cec: Remove TX/RX SW_INIT on power_on See: Hexxeh/rpi-firmware#267 See: https://www.raspberrypi.org/forums/viewtopic.php?p=1895082#p1895082 - firmware: arm_dt: Limit CMA to 256MB if total_mem < 2GB or gpu_mem > 256MB See: #1603 - firmware: video_decode: Use the ISP instead of vc_image_convert - firmware: video_decode: Correct support for YVU formats using ISP - firmware: firmware: Disable VLL loading from file system See: #1605 - firmware: arm_loader: Make most arm clock requests required See: #1598 - firmware: arm_loader: Consider required flags from GET_CLOCK_RATE See: #1598 - firmware: arm_dt: Load overlays for detected cameras - firmware: Make more use of the user-warnings DT property - firmware: hdmi_2711: Use HDMI block REPEAT_PIXEL instead of PV See: https://forum.libreelec.tv/thread/24415-le-10-beta-for-i4-force-hdmi-resolution - firmware: DSI display autodetection for kms - firmware: arm_loader: Allow hvs interrupt during SET_NOTIFY_DISPLAY_DONE - firmware: arm_display: Allow null buffer in successful call See: raspberrypi/linux#4540 - firmware: video_decode: Ensure all buffers are flushed before port disable completes - firmware: filesystem: sdcard: Fix Hybrid GPT partitions See: #1465 - firmware: tvservice: Add check to warn when running with kms - firmware: arm_loader: Allow non-optional reads of current clock See: #1619 - firmware: dispmanx: Demote null eptr from vcos_verify to no warning See: raspberrypi/linux#4592 - firmware: filesystem: sdcard: Probe FAT type in GPT ESD partitions - firmware: clock-2711: Limit PLLB VCO frequency to the high range - firmware: arm_dt: Export the boot-mode, partition and usb state via device-tree See: #1621 - firmware: video_decode: i/p port enable/disable without o/p active could stall See: RPi-Distro/vlc#48 See: Hexxeh/rpi-firmware#272 See: #1637 - firmware: userland: Reduce debug_sym error messages See: https://forums.raspberrypi.com/viewtopic.php?f=98&t=322238 - firmware: arm_dt: Increase maximum line length to 98 See: raspberrypi/linux#4638 - firmware: arm_loader: Allow VEC clock to be controlled by arm - firmware: platform: Remove licence on VP6, VP8, Theora, and FLAC See: raspberrypi/linux#4661 - firmware: ISP: Fix magenta colour in right hand image of stereo pair See: https://forums.raspberrypi.com/viewtopic.php?t=321089 - firmware: platform: Fix incorrect turbo voltage scaling on Pi0 See: raspberrypi/documentation#2255 - firmware: platform: Declare CM4's SIO_1V8_SEL and SD_PWR_ON See: raspberrypi/Raspberry-Pi-OS-64bit#188 - firmware: hello_fft: Update outdated link to V3D spec - firmware: hello_fft: Remove unused function declaration See: #1645 See: raspberrypi/userland#710 - firmware: dtoverlay: Rebase aliases in overlays like labels - firmware: isp: Set core/vpu min clock to 320Mhz during ISP operation - firmware: arm_loader: Enable watchdog early if wanted See: #1651 - firmware: board_info: Add upstream dtb names for cm1 & 3 - firmware: board_info: Add upstream dtb name for cm4 See: #1660 - firmware: platform: Allow users to disable camera boot HMAC check See: #1657 - firmware: clock: 2711: Fix potential API issue in 2711 VCO setup - firmware: arm_loader: Enable USB MSD boot mode on Zero 2 W - firmware: isp: Fix Rec.709 colour space problems
I think there is a bug in the latest pi4 Linux release, because data transfer over USB becomes unreliable after upgrading from Linux v4.19-v71+ to v5.4.83-v71+ on raspberry pi4.
We are using the USB C port as a RNDIS gadget to get TCP data to a PC. I found the minimum python code required to reproduce the problem, it sets up a socketserver on the pi4, and the client on the PC repeatedly asks for 10MBs data. In Linux v4.19-v71+ it is robust, and I can do the transfer > 5000 times (expected behaviour). In v5.4.83-v71+ exactly the same code works 50 to 500 times then the link starts to break down, I get time out errors (even with time out set to 10 seconds), and frequently after that the link freezes entirely and requires the USB to be disconnected and reconnected for it to return. It even takes down an SSH connection to the pi which is separate software (but connecting over the same USB cable).
I'm unable to use v4.19-v71+ from now because it is not compatible with the latest pi4 hardware, so I hope someone can help - either with a bug fix or workaround suggestions.
To reproduce:
Test code I am using is here:
USBdataTest.zip
Set up the pi4 as a usb gadget:
To /boot/config.txt add:
dtoverlay=dwc2
To /boot/cmdline.txt add:
modules-load=dwc2,g_ether
and use the /etc/network/interfaces file provided.
On the pi4 run:
python3 lanTestPi2.py
On a windows PC run:
python3 lanClientTest.py
It can be reproduced with or without an external PSU for the pi.
Other information:
PC OS: Windows 10
Pi4 boot loader: Sep 3 2020
Python version: 3.7
I upgraded by using:
sudo apt update
sudo apt full-upgrade
Logs:
Time out error (from PC recv command).
No error from pi4, but PC can no longer connect to it.
Please let me know if there is any other information I can provide, or if I can do any tests for you to narrow down the problem.
The text was updated successfully, but these errors were encountered: