RBPi3B+ USB VBUS reset during initialization

[arisoft-fi]

During the Kernel initialization, USB power is dropped off and back on causing some devices not to reset properly during the boot/reboot. For example some SSD drives may fail totally and HDD drives emits a noticeable power failure reset sound just before mounting the root.

Steps required to reproduce the issue
Install rpi-4.19.y kernel to USB drive, no SD-card
Add an indicator for VBUS power. For example, LED light powered from a USB port.

Expected behaviour
If the VBUS is already powered by the bootloader it should be left on by the kernel.
Or the power cycling should be done properly. For example, waiting a full second before applying the power back on.

Actual behaviour
During the boot process the bootloader apply VBUS power on and about 0.3 seconds after the kernel starts, the VBUS is powered off for some milliseconds before it put back on causing some devices to fail during boot/reboot.

System
Raspberry Pi 3B+
Raspberry Pi reference 2019-04-08
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, e910bf55ac92e27c6e77043e8995bcb5e0a2d742, stage2
vcgencmd version
Mar 27 2019 15:46:32
Copyright (c) 2012 Broadcom
version 2e98b31d18547962e564bdf88e57b3df7085c29b (clean) (release) (start_cd)
4.19.42-v7+ #1219 SMP Tue May 14 21:20:58 BST 2019 armv7l GNU/Linux

Additional context
The power is reset while executing dwc_otg_driver_init
linux/drivers/usb/host/dwc_otg/dwc_otg_driver.c
Line 1108: retval = platform_driver_register(&dwc_otg_driver);

Despite using several hours for tracing this down I could't find the actual line where the power is cut off. Maybe it is somewhere inside the platform level code.
Adding one second sleep after this line helps for keeping the power off longer.

retval = platform_driver_register(&dwc_otg_driver);
    printk(KERN_INFO "dwc_otg: VBUS power cycling\n");
dwc_mdelay(1000);

[arara0901]

Provide information I know.
I guess that this issue is caused by the bootloader.
I observed vbus by the oscilloscope with the following pattern.

• Normal startup
• Kernel skip (bootloader only)
  I observed the same vbus drop during both initials.

CH1 is vbus. CH2 is 3.3v

These results show that the vbus drop does not involve the kernel.
It seems looking at the drop pattern, the bootloader does not Implement to be intentionally low.
I think the bootloader has this problem and fixing it will solve it.

Despite using several hours for tracing this down I could't find the actual line where the power is cut off. Maybe it is somewhere inside the platform level code.

I guess this code doesn't exist in the kernel.

[arisoft-fi]

The bootloader definitely plays some role with this but what I discovered is that the VBUS resets 0.3 seconds after the kernel was loaded. I recorded a video showing the display and a LED powered from the VBUS to measure the actual timing referenced to kernel debug message timestamps.

I "solved" the main issue by adding a sleep in the dwc_otg_driver to make the reset longer which allows my SSD drive to start again properly. The best solution is to prevent powering off because it may have other side effects on the SSD drive.

I guess that the power glitch may be a reason for some similar reports that some USB drives do not boot without kernel installed on SD-card.

[arara0901]

I'm glad you found a workaround.

The best solution is to prevent powering off because it may have other side effects on the SSD drive.

I think so, too.

I have a few questions.

I discovered is that the VBUS resets 0.3 seconds after the kernel was loaded.

Do you have evidence that vbus was reset after the kernel was loaded?
If yes, could you show me?

How did you know the vbus kernel timestamps when the LED powered from the VBUS was turned on / off?

[arisoft-fi]

Here are images taken from the video I recorded during a reboot.
On the left side of the screen you can see a green LED which is connected to VBUS
As you can see the VBUS is powered when the kernel starts
About 0.3s later the VBUS power disappears.

[arara0901]

Thanks for your infomation.

I discovered details of vbus drop timing.
After dwc_otg_driver send USB_HCD power on message to GPU firmware(video core), vbus drops.
The GPU firmware needs to be fixed.

Additinal infomation

• In the case of sdcard boot, this problem does not occur because vbus is turned on for the first time after this message is sent.
• Refer to the following for messages between the GPU firmware and kernel.
  https://github.com/raspberrypi/firmware/wiki/Mailbox-property-interface
• GPU firmware returned a successful response when vbus dropped.
• The following function sends a USB_HCD power on message.
  This function is called in the context of platform_driver_register.
  linux/drivers/soc/bcm/raspberrypi-power.c
  static int rpi_domain_on(struct generic_pm_domain *domain)

I think this is not a raspberrypi/linux issue, this is raspberrypi/firmware issue.

[arisoft-fi]

This explains why I didn't find the cause from the device driver code.
Do you sugges to wait for the fimrware update or using a custom kernel workaround as I did?
The power on delay could be added to the kernel module parameters event this is not the best solution.

[popcornmix]

@P33M arm requests arm_power_request(ARM_POWER_USB)
which goes through usb_open which has:
// If USB was previously powered on then turn it off first
which is presumably the issue reported here. What should we do?
Is skipping the power off / power on in this state safe here, or does stuff need to be reset?

[P33M]

@ghollingworth added the block unpower/repower when MSD boot went in - the disconnect (and thus Vbus drop) is necessary to get the bus into a state that Linux recognises (apparently a USB reset doesn't work).

The cycle time for the controller power cycle is 2ms - increasing it would likely fix the issue at the cost of slowing down boot, but removing the cycle would break MSD boot.

The cycle delay should probably be a config.txt parameter that can be tuned.

[arara0901]

@popcornmix , @P33M thanks for information.

@ghollingworth
Does the controller power cycle mean the interval between USB HCD off/on?
I think this issue can be resolved by changing the cycle time to an adjustable Config.txt parameter.

Please give the opinion of @ghollingworth and @arisoft-fi .

[ghollingworth]

Yes a config.txt parameter would work to make it longer...

[6by9]

The bootloader/firmware uses the USB for loading start*.elf and kernelX.img into RAM.
Can't it shut the USB down after it has loaded those files and before starting the kernel? I seem to recall we have a filesys_close or similar once done with everything.

(Does this reset also affect network boot?)

[FD-]

Is it possible this issue also affects the Pi Zero? I'm seeing some odd behaviour with Realtek 8152B ethernet adapters failing to enumerate with "device descriptor read/64, error -110" after a cold boot.

UPDATE: Never mind, I didn't realise this only affects network/USB boot. Plus, the Pi Zero's USB host VBUS seems permanently connected to the upstream VBUS. So, unrelated issue.