Pi5 failed to contact RP1 firmware

[llllllllqq]

Describe the bug

dmesg says:

...
rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware [ 3.957885] rp1-pio: probe of 1f00178000.pio failed with error -2
...

Although nothing strange occurred.
The OS still works fine without reboot.

Steps to reproduce the behaviour

It randomly occurs after sudo reboot --halt and then manually press the on-board power button to power on (with default eeprom config).

It also randomly occurs when use raspi-config to change power off behavior to "full power off" and then reboot/poweroff. (Sorry I cannot remember whether it is reboot or poweroff)

Update: It also occurs after a suddenly plug-out the usb-c power cable and then plug-in back.
It seems only poweroff-related behaviors will trigger this bug. I tested 30 times reboot, no bugs occurred.
Update2: reboot can indeed also trigger this bug, only less frequent.

Device (s)

Raspberry Pi 5

System

Raspberry Pi reference 2024-11-19 Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 891df1e21ed2b6099a2e6a13e26c91dea44b34d4, stage4

2025/01/22 00:16:51
Copyright (c) 2012 Broadcom version a7753063 (release) (embedded)

Linux rpi5 6.6.74+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.74-1+rpt1 (2025-01-27) aarch64 GNU/Linux

Logs

1.txt

Additional context

Can it be safely ignored?

[pelwell]

That's interesting - it looks like a dependency/ordering issue, and not one I've seen before:

[    3.922665] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    3.922671] rp1-pio: probe of 1f00178000.pio failed with error -2
...
[    3.932284] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

I get:

[   13.445656] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[   13.550428] rp1-pio 1f00178000.pio: Created instance as pio0

The code for communicating with the firmware exists in the firmware module, so I thought that would enforce an ordering. Some investigation is required.

[llllllllqq]

That's interesting - it looks like a dependency/ordering issue, and not one I've seen before:

[    3.922665] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    3.922671] rp1-pio: probe of 1f00178000.pio failed with error -2
...
[    3.932284] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

I get:

[   13.445656] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[   13.550428] rp1-pio 1f00178000.pio: Created instance as pio0

The code for communicating with the firmware exists in the firmware module, so I thought that would enforce an ordering. Some investigation is required.

Thanks for your reply.
Here is my dmesg when the rp1&pio works fine.
I hope it can help to investigate this issue.

normal.txt

[pelwell]

Yes - in the working case the order is reversed:

[    3.955929] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[    3.961971] vc4-drm axi:gpu: bound 107c580000.hvs (ops vc4_hvs_ops [vc4])
[    3.967397] rp1-pio 1f00178000.pio: Created instance as pio0

[aBUGSworstnightmare-rpi]

[ 5.557565] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware [ 5.557571] rp1-pio: probe of 1f00178000.pio failed with error -2 [ 5.558534] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

Seeing that on reboot as well /custom kernel on a Pi5 DUT, trying to get an overlay to work)

Linux Pi5uSDDUT 6.6.74-v8-16k+ #1 SMP PREEMPT Sun Feb 2 10:41:23 CET 2025 aarch64 GNU/Linux Revision : c04170 Serial : cb30eb1fd7e7e672 Model : Raspberry Pi 5 Model B Rev 1.0

[pelwell]

It's really strange. I can't get it to fail in that way without hacks - even with a massive delay in the firmware driver, the pio driver waits. Anyway, I'm just testing a fix.

[aBUGSworstnightmare-rpi]

seems quite random. Seen it frequently on reboots (running 'sudo reboot' on CLI)

[mayo228]

is there a way to fix this? I keep getting this error and I can't boot the OS, should I be worried that I will loose data on my raspberry?

[pelwell]

I don't understand why this affecting some people while I can't reproduce it without hacking the driver, but regardless, #6645 should fix the issue. After about 40 minutes you should be able to install a trial kernel with sudo rpi-update pulls/6645, after first backing up any important data.

[pelwell]

And no, you shouldn't worry - it would only affect you if you wanted to use piolib to drive the GPIOs.

[aBUGSworstnightmare-rpi]

I have no idea if and how below is related but I just wanted to state that I'm not making use of piolib for accessing GPIO's.

[   13.678330] ieee80211 phy0: brcmf_p2p_send_action_frame: Unknown Frame: category 0xa, action 0x8
[   15.374753] Bluetooth: RFCOMM TTY layer initialized
[   15.374765] Bluetooth: RFCOMM socket layer initialized
[   15.374772] Bluetooth: RFCOMM ver 1.11
[   27.660651] brcmfmac: brcmf_set_channel: set chanspec 0xd022 fail, reason -52
[   27.768681] brcmfmac: brcmf_set_channel: set chanspec 0xd026 fail, reason -52
[   27.872656] brcmfmac: brcmf_set_channel: set chanspec 0xd02a fail, reason -52
[   27.976661] brcmfmac: brcmf_set_channel: set chanspec 0xd02e fail, reason -52
[   29.636663] brcmfmac: brcmf_set_channel: set chanspec 0xd090 fail, reason -52
[   29.636875] brcmfmac: brcmf_set_channel: set chanspec 0xd095 fail, reason -52
[   29.637196] brcmfmac: brcmf_set_channel: set chanspec 0xd099 fail, reason -52
[   29.637400] brcmfmac: brcmf_set_channel: set chanspec 0xd09d fail, reason -52
[   29.637594] brcmfmac: brcmf_set_channel: set chanspec 0xd0a1 fail, reason -52
[   29.637791] brcmfmac: brcmf_set_channel: set chanspec 0xd0a5 fail, reason -52

[pelwell]

I have no idea if and how below is related

You should know that WiFi and Bluetooth information and warning messages are unrelated.

[llllllllqq]

It seems after applying sudo rpi-update pulls/6645, this issue disappeared. (I tested about 10 times reboot/halt/suddenly poweroff). Thanks for your work! If anything strange found, I'll post here.

BTW, can I safely stay at this firmware and wait for the next stable release offered by apt? Or do I need a sudo apt install --reinstall raspi-firmware to restore first before the next stable release?

[pelwell]

You should be fine with this kernel until the next proper release - I'm not aware of any new issues.

[randomMesh]

I can confirm the error happens on my PI 5 too:

[    0.692347] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    0.692370] rp1-pio: probe of 1f00178000.pio failed with error -2

I am running the latest firmware:

BOOTLOADER: up to date
   CURRENT: Wed 22 Jan 00:16:51 UTC 2025 (1737505011)
    LATEST: Wed 22 Jan 00:16:51 UTC 2025 (1737505011)
   RELEASE: latest (/usr/lib/firmware/raspberrypi/bootloader-2712/latest)
            Use raspi-config to change the release.

[helpful-emu57]

A week ago I also started receiving these errors during bootup:

rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware

At the same time I also started getting protocol errors for the wireless mouse and wireless keyboard
logitech-hidpp-device - hidpp_root_get_ protocol_version: received protocol error 0x08
(Although - both devices continued to work appropriately.)

Changing to wired devices there are no mouse or keyboard errors but still have the rp1-pio error

I have performed the "sudo rpi-update pulls/6645" and no change was observed.

[pelwell]

What does rpi-eeprom-update show?

[theepicflyer]

I'm getting the same error too but maybe a little a different. One installation that I had piolib working on, stopped working last week or so. A fresh OS install on a separate SD card on the same Pi does work consistently though.

Not working install:

[    1.032795] /axi/pcie@120000/rp1: Fixed dependency cycle(s) with /axi/pcie@120000/rp1
[    1.051186] /axi/pcie@120000/rp1: Fixed dependency cycle(s) with /axi/pcie@120000/rp1
[    2.163309] rp1 0000:01:00.0: bar0 len 0x4000, start 0x1f00410000, end 0x1f00413fff, flags, 0x40200
[    2.172393] rp1 0000:01:00.0: bar1 len 0x400000, start 0x1f00000000, end 0x1f003fffff, flags, 0x40200
[    2.181659] rp1 0000:01:00.0: enabling device (0000 -> 0002)
[    2.187947] rp1 0000:01:00.0: chip_id 0x20001927
[    2.196677] genirq: irq_chip rp1_irq_chip did not update eff. affinity mask of irq 100
[    2.957783] rp1-pio 1f00178000.pio: error -ENOENT: failed to contact RP1 firmware
[    2.965577] rp1-pio: probe of 1f00178000.pio failed with error -2
[    5.861008] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55

notworking.txt

Working install:

[    1.032788] /axi/pcie@120000/rp1: Fixed dependency cycle(s) with /axi/pcie@120000/rp1
[    1.051175] /axi/pcie@120000/rp1: Fixed dependency cycle(s) with /axi/pcie@120000/rp1
[    2.167264] rp1 0000:01:00.0: bar0 len 0x4000, start 0x1f00410000, end 0x1f00413fff, flags, 0x40200
[    2.176349] rp1 0000:01:00.0: bar1 len 0x400000, start 0x1f00000000, end 0x1f003fffff, flags, 0x40200
[    2.185614] rp1 0000:01:00.0: enabling device (0000 -> 0002)
[    2.191893] rp1 0000:01:00.0: chip_id 0x20001927
[    2.200581] genirq: irq_chip rp1_irq_chip did not update eff. affinity mask of irq 100
[    4.932575] rp1-firmware rp1_firmware: RP1 Firmware version eb39cfd516f8c90628aa9d91f52370aade5d0a55
[    4.960712] rp1-pio 1f00178000.pio: Created instance as pio0

working.txt

Let me know if there is any other info I can provide that will help.

[pelwell]

The important changes to resolve this failure went into #6645. Although it was merged to 6.6.74, that happened a week after your kernel was built (2025-02-04 vs 2025-01-27).