Assistance reading diagnostics after failure- 3.3
-
@jay_s_uk grabbing them now
-
Here is the relevant section. SD Card failure?
Aug 16 11:18:25 oozeBot kernel: [188744.509778] mmc1: Timeout waiting for hardware interrupt. Aug 16 11:18:25 oozeBot kernel: [188744.509850] brcmfmac: mmc_submit_one: CMD53 sg block write failed -110 Aug 16 11:18:25 oozeBot kernel: [188744.509869] brcmfmac: brcmf_sdio_txfail: sdio error, abort command and terminate frame Aug 16 11:18:25 oozeBot kernel: [188744.510081] ieee80211 phy0: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110 Aug 16 11:18:25 oozeBot kernel: [188744.511323] brcmfmac: brcmf_sdio_hdparse: HW header checksum error Aug 16 11:18:25 oozeBot kernel: [188744.511338] brcmfmac: brcmf_sdio_rxfail: terminate frame Aug 16 11:18:25 oozeBot kernel: [188744.512497] brcmfmac: brcmf_sdio_hdparse: HW header checksum error Aug 16 11:18:25 oozeBot kernel: [188744.512512] brcmfmac: brcmf_sdio_rxfail: terminate frame Aug 16 11:18:25 oozeBot kernel: [188744.512686] brcmfmac: brcmf_sdio_hdparse: HW header checksum error Aug 16 11:18:25 oozeBot kernel: [188744.512714] brcmfmac: brcmf_sdio_rxfail: terminate frame Aug 16 11:18:25 oozeBot kernel: [188744.512891] brcmfmac: brcmf_sdio_hdparse: HW header checksum error Aug 16 11:18:25 oozeBot kernel: [188744.512905] brcmfmac: brcmf_sdio_rxfail: terminate frame Aug 16 11:18:27 oozeBot DuetControlServer[1043]: [info] System time has been changed Aug 16 11:18:27 oozeBot kernel: [188747.069866] brcmfmac: brcmf_sdio_bus_rxctl: resumed on timeout Aug 16 11:18:27 oozeBot kernel: [188747.070224] brcmfmac: brcmf_sdio_readshared: invalid sdpcm_shared address 0x00000000 Aug 16 11:18:27 oozeBot kernel: [188747.070241] brcmfmac: brcmf_sdio_readshared: unable to obtain sdpcm_shared info: rv=-22 (addr=0x0) Aug 16 11:18:27 oozeBot kernel: [188747.070268] ieee80211 phy0: brcmf_cfg80211_get_station: GET STA INFO failed, -110 Aug 16 11:18:27 oozeBot DuetControlServer[1043]: [warn] Controller has been reset Aug 16 11:18:27 oozeBot DuetControlServer[1043]: [info] Aborted job file Aug 16 11:18:27 oozeBot DuetControlServer[1043]: [warn] Trigger: Out-of-order reply: '' Aug 16 11:18:31 oozeBot kernel: [188750.655249] brcmfmac: brcmf_sdio_hdparse: HW header checksum error Aug 16 11:18:31 oozeBot kernel: [188750.655268] brcmfmac: brcmf_sdio_rxfail: terminate frame Aug 16 11:18:34 oozeBot kernel: [188753.630009] brcmfmac: brcmf_sdio_bus_rxctl: resumed on timeout Aug 16 11:18:34 oozeBot kernel: [188753.630360] brcmfmac: brcmf_sdio_readshared: invalid sdpcm_shared address 0xFFB70048 Aug 16 11:18:34 oozeBot kernel: [188753.630377] brcmfmac: brcmf_sdio_readshared: unable to obtain sdpcm_shared info: rv=-22 (addr=0xffb70048) Aug 16 11:18:36 oozeBot DuetControlServer[1043]: [info] System time has been changed Aug 16 11:18:36 oozeBot kernel: [188756.190060] brcmfmac: brcmf_sdio_bus_rxctl: resumed on timeout Aug 16 11:18:36 oozeBot kernel: [188756.190424] brcmfmac: brcmf_sdio_readshared: invalid sdpcm_shared address 0xFFB70048 Aug 16 11:18:36 oozeBot kernel: [188756.190441] brcmfmac: brcmf_sdio_readshared: unable to obtain sdpcm_shared info: rv=-22 (addr=0xffb70048) Aug 16 11:18:36 oozeBot kernel: [188756.190468] ieee80211 phy0: brcmf_cfg80211_get_station: GET STA INFO failed, -110 Aug 16 11:18:41 oozeBot kernel: [188760.659861] brcmfmac: brcmf_sdio_hdparse: HW header checksum error Aug 16 11:18:41 oozeBot kernel: [188760.659879] brcmfmac: brcmf_sdio_rxfail: terminate frame Aug 16 11:18:43 oozeBot kernel: [188762.760085] brcmfmac: brcmf_sdio_bus_rxctl: resumed on timeout Aug 16 11:18:43 oozeBot kernel: [188762.766964] brcmfmac: brcmf_sdiod_ramrw: membytes transfer failed Aug 16 11:18:43 oozeBot kernel: [188762.766974] brcmfmac: brcmf_sdio_readshared: unable to obtain sdpcm_shared info: rv=-84 (addr=0x20000010) Aug 16 11:18:43 oozeBot kernel: [188762.767421] brcmfmac: brcmf_sdio_hdparse: HW header checksum error Aug 16 11:18:43 oozeBot kernel: [188762.767428] brcmfmac: brcmf_sdio_rxfail: terminate frame Aug 16 11:18:43 oozeBot kernel: [188762.775888] ieee80211 phy0: brcmf_cfg80211_get_station: GET STA INFO failed, -52 Aug 16 11:18:43 oozeBot dhcpcd[628]: wlan0: carrier lost
-
It's a bit confusing since brcmfmac is a wifi driver.
-
@oliof agreed. The RPi did not reboot, we just lost connection to it through DWC for ~30 seconds. Once the connection was restored, the print was canceled. I wonder if the change in System time (line 13) had anything to do with it? That's the first thing reported by DuetControlServer.
-
CMD53 is some kind of configuration command for a wifi chip; which is where some error was triggered. This seems to have blocked some subsystem for about 2 seconds, long enough for a realtime system to decide it is in an irrevocable state of de-sync.
Sleuthing a bit, it could be related to an unfixed issue with the wifi chip and/or driver on the Raspberry Pi which has been around since 2018 ...
I'd suggest moving to Ethernet if possible.
-
@oliof I think CMD53 actually related to the SD card interface. It looks to me that there was a timeout waiting for an interrupt from the SD card subsystem.
-
@oozebot said in Assistance reading diagnostics after failure- 3.3:
brcmfmac: mmc_submit_one
Thank you both. Based on what we are finding, we are starting to think we overtaxed the wifi module with several webcam streams going at once.
-
@dc42 I'm confused why the wifi driver does SD card interface management, but that may be the vagaries of integrated electronics. It still seems to be resulting in a failure of the wifi interface twice (
Aug 16 11:18:27 oozeBot kernel: [188747.070268] ieee80211 phy0: brcmf_cfg80211_get_station: GET STA INFO failed, -110
and again at 11:18:43). -
-
on the RPi 3B it seems to be BCM43438 (on RPi4 a BCM4345/6) -- and you're right, they are connected via SDIO interface. I am less confused now.