Reboots/crashes - RRF ≤3.5.0-rc1
-
@dc42 copy, just installed it.
I will post a new M122 report ASAP
👍 -
Here we go:
debuglog:
2023-11-16 03:59:44 [warn] HTTP client 192.168.10.xx login succeeded (session key 1771356456) power up + 00:00:03 [info] Event logging started at level debug power up + 00:00:03 [info] Running: Duet 3 Mini5plus WiFi: 3.5.0-rc.1+ (2023-11-15 20:11:19) power up + 00:00:03 [info] Event logging stopped power up + 00:00:03 [info] Event logging started at level debug power up + 00:00:03 [info] Running: Duet 3 Mini5plus WiFi: 3.5.0-rc.1+ (2023-11-15 20:11:19) power up + 00:00:03 [debug] Done! power up + 00:00:03 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running. power up + 00:00:04 [warn] WiFi module started power up + 00:00:04 [warn] Error: Heater 1 fault: failed to read sensor: notReady power up + 00:00:04 [info] M291: - Event notification - Heater 1 fault: failed to read sensor: notReady power up + 00:00:04 [debug] - Event notification - power up + 00:00:04 [debug] Heater 1 fault: failed to read sensor: notReady power up + 00:00:06 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:06 [warn] WiFi module is idle power up + 00:00:08 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:08 [warn] WiFi module is idle power up + 00:00:11 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:11 [warn] WiFi module is idle power up + 00:00:13 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:13 [warn] WiFi module is idle power up + 00:00:15 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:15 [warn] WiFi module is idle power up + 00:00:18 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:18 [warn] WiFi module is idle power up + 00:00:20 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:20 [warn] WiFi module is idle power up + 00:00:22 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:22 [warn] WiFi module is idle power up + 00:00:25 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:25 [warn] WiFi module is idle power up + 00:00:27 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:27 [warn] WiFi module is idle power up + 00:00:30 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:30 [warn] WiFi module is idle power up + 00:00:32 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:32 [warn] WiFi module is idle power up + 00:00:34 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:34 [warn] WiFi module is idle power up + 00:00:37 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:37 [warn] WiFi module is idle power up + 00:00:41 [warn] WiFi module is connected to access point RV32-IOT2G, IP address 192.168.10.x power up + 00:00:42 [warn] HTTP client 192.168.10.xx login succeeded (session key 1408720838) 2023-11-16 09:15:05 [warn] Date and time set at power up + 00:00:42M122:
=== Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-11-15 20:11:19) running on Duet 3 Mini5plus WiFi (standalone mode) Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ Used output buffers: 1 of 40 (29 max) === RTOS === Static ram: 102844 Dynamic ram: 123820 of which 0 recycled Never used RAM 11896, free system stack 186 words Tasks: NETWORK(1,ready,3212.5%,232) HEAT(3,nWait,4.5%,327) Move(4,nWait,0.5%,352) CanReceiv(6,nWait,9.0%,774) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,1.4%,350) TMC(4,nWait,129.2%,108) MAIN(1,running,1199.1%,670) IDLE(0,ready,45.8%,29) AIN(4,delaying,153.3%,264), total 4755.3% Owned mutexes: WiFi(NETWORK) === Platform === Last reset 06:23:49 ago, cause: software Last software reset at 2023-11-16 09:14, reason: AssertionFailed, Gcodes spinning, available RAM 10964, slot 2 Software reset code 0x4123 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00487000 BFAR 0xe000ed38 SP 0x20011fbc Task NETW Freestk 495 ok Stack: 00000904 000af288 0002ddbf 2002bed4 2002be01 000001a8 2002c100 20031c40 2002c118 2001e888 00000000 a5a5a5a5 00000000 00000000 00000000 00000000 20031c4c 00000800 20035940 2002c100 20018668 2002bf9d 20018668 2001e888 00030037 00000000 00000000 Error status: 0x00 Aux0 errors 0,0,0 MCU revision 3, ADC conversions started 23029641, completed 23029641, timed out 0, errs 0 MCU temperature: min 34.7, current 35.1, max 37.8 Supply voltage: min 23.0, current 24.1, max 26.0, under voltage events: 0, over voltage events: 0, power good: yes Heap OK, handles allocated/used 99/33, heap memory allocated/used/recyclable 2048/716/280, gc cycles 1071 Events: 1 queued, 1 completed Driver 0: standstill, SG min 16, read errors 0, write errors 1, ifcnt 170, reads 32406, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 1: standstill, SG min 222, read errors 0, write errors 1, ifcnt 170, reads 32406, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 2: standstill, SG min 2, read errors 0, write errors 1, ifcnt 127, reads 32406, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 3: standstill, SG min 2, read errors 0, write errors 1, ifcnt 125, reads 32406, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 4: standstill, SG min 2, read errors 0, write errors 1, ifcnt 126, reads 32406, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 5: not present Driver 6: not present Date/time: 2023-11-16 15:38:13 Cache data hit count 4294967295 Slowest loop: 12.92ms; fastest: 0.14ms === Storage === Free file entries: 18 SD card 0 detected, interface speed: 22.5MBytes/sec SD card longest read time 7.2ms, write time 4.1ms, max retries 0 === Move === DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00 no step interrupt scheduled Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0 === DDARing 0 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === DDARing 1 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0 === GCodes === Movement locks held by null, null HTTP is idle in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is doing "G4 S1" in state(s) 0 0, running macro Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 File2 is idle in state(s) 0 Queue2 is idle in state(s) 0 Q0 segments left 0, axes/extruders owned 0x0000803 Code queue 0 is empty Q1 segments left 0, axes/extruders owned 0x0000000 Code queue 1 is empty === Filament sensors === in 0 notIn 0 Extruder 0 sensor: no filament === CAN === Messages queued 207276, received 472154, lost 0, errs 1, boc 0 Longest wait 2ms for reply type 6057, peak Tx sync delay 275, free buffers 26 (min 25), ts 115148/115147/0 Tx timeouts 0,0,0,0,0,0 === Network === Slowest loop: 5.98ms; fastest: 0.00ms Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) HTTP sessions: 1 of 8 === WiFi === Interface state: active Module is connected to access point Failed messages: pending 0, notrdy 0, noresp 0 Firmware version 2.1beta4 MAC address c4:5b:be:ce:91:93 Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42960 WiFi IP address 192.168.10.x Signal strength -52dBm, channel 6, mode 802.11n, reconnections 0 Clock register 00002001 Socket states: 0 0 0 0 0 0 0 0 -
@Exerqtor thanks, that's just what I needed to pinpoint what's going on. A DMA transfer is overwriting the return address on the stack. Unless I can spot the reason, I'll need to add some more debug to the firmware and ask you to run in again to find out why.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 Thats great news! Starting to get the feeling it might be some light breaking through the clouds now
🤣Just put in what ever you might need and i'll give it a go, can't really use the printer in this state anyways lol.
-
@Exerqtor thanks, I've had to work on other things today but I'll come back to this very soon. I'm still puzzled that it's not reported more widely.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 No problem! Yeah it's struck me as odd too
🤷♂️ -
@dc42 New crash:
debug:2023-11-17 21:01:32 [warn] HTTP client 192.168.10.xx login succeeded (session key 3428338341) power up + 00:00:03 [info] Event logging started at level debug power up + 00:00:03 [info] Running: Duet 3 Mini5plus WiFi: 3.5.0-rc.1+ (2023-11-15 20:11:19) power up + 00:00:03 [info] Event logging stopped power up + 00:00:03 [info] Event logging started at level debug power up + 00:00:03 [info] Running: Duet 3 Mini5plus WiFi: 3.5.0-rc.1+ (2023-11-15 20:11:19) power up + 00:00:03 [debug] Done! power up + 00:00:03 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running. power up + 00:00:04 [warn] Error: Heater 1 fault: failed to read sensor: notReady power up + 00:00:04 [info] M291: - Event notification - Heater 1 fault: failed to read sensor: notReady power up + 00:00:04 [debug] - Event notification - power up + 00:00:04 [debug] Heater 1 fault: failed to read sensor: notReady power up + 00:00:04 [warn] WiFi module started power up + 00:00:06 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:06 [warn] WiFi module is idle power up + 00:00:08 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:08 [warn] WiFi module is idle power up + 00:00:11 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:11 [warn] WiFi module is idle power up + 00:00:13 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:13 [warn] WiFi module is idle power up + 00:00:16 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:16 [warn] WiFi module is idle power up + 00:00:18 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:18 [warn] WiFi module is idle power up + 00:00:20 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:20 [warn] WiFi module is idle power up + 00:00:23 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:23 [warn] WiFi module is idle power up + 00:00:25 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:25 [warn] WiFi module is idle power up + 00:00:27 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:27 [warn] WiFi module is idle power up + 00:00:30 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:30 [warn] WiFi module is idle power up + 00:00:32 [warn] Error: WiFi module reported: Failed to load credentials power up + 00:00:32 [warn] WiFi module is idle power up + 00:00:36 [warn] WiFi module is connected to access point RV32-IOT2G, IP address 192.168.10.x power up + 00:00:36 [warn] HTTP client 192.168.10.xx login succeeded (session key 2751057796) 2023-11-18 07:01:07 [warn] Date and time set at power up + 00:00:36M122:
=== Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-11-15 20:11:19) running on Duet 3 Mini5plus WiFi (standalone mode) Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ Used output buffers: 1 of 40 (34 max) === RTOS === Static ram: 102844 Dynamic ram: 123844 of which 0 recycled Never used RAM 11872, free system stack 186 words Tasks: NETWORK(1,ready,16.7%,219) HEAT(3,nWait,0.0%,352) Move(4,nWait,0.0%,358) CanReceiv(6,nWait,0.0%,797) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,108) MAIN(1,running,81.1%,670) IDLE(0,ready,0.6%,29) AIN(4,delaying,0.8%,264), total 100.0% Owned mutexes: === Platform === Last reset 00:22:02 ago, cause: software Last software reset at 2023-11-18 07:00, reason: AssertionFailed, Gcodes spinning, available RAM 11268, slot 1 Software reset code 0x4123 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x20011fbc Task NETW Freestk 495 ok Stack: 00000904 000af288 0002ddbf 2002bed4 2002be01 000001a8 2002c100 200303f8 2002c118 2001e888 00000000 a5a5a5a5 00000000 00000000 00000000 00000000 20030404 00000800 20035940 2002c100 20018668 2002bf9d 20018668 2001e888 00030037 00000000 00000000 Error status: 0x00 Aux0 errors 0,0,0 MCU revision 3, ADC conversions started 1322550, completed 1322550, timed out 0, errs 0 MCU temperature: min 34.8, current 35.1, max 35.3 Supply voltage: min 24.0, current 24.1, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes Heap OK, handles allocated/used 99/33, heap memory allocated/used/recyclable 2048/1856/1420, gc cycles 61 Events: 1 queued, 1 completed Driver 0: standstill, SG min 16, read errors 0, write errors 0, ifcnt 207, reads 965, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 1: standstill, SG min 240, read errors 0, write errors 0, ifcnt 207, reads 965, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 160, reads 965, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 157, reads 966, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 158, reads 965, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 5: not present Driver 6: not present Date/time: 2023-11-18 07:22:32 Cache data hit count 2212461605 Slowest loop: 10.21ms; fastest: 0.13ms === Storage === Free file entries: 18 SD card 0 detected, interface speed: 22.5MBytes/sec SD card longest read time 3.6ms, write time 4.2ms, max retries 0 === Move === DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00 no step interrupt scheduled Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0 === DDARing 0 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === DDARing 1 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0 === GCodes === Movement locks held by null, null HTTP is idle in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is doing "G4 S1" in state(s) 0 0, running macro Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 File2 is idle in state(s) 0 Queue2 is idle in state(s) 0 Q0 segments left 0, axes/extruders owned 0x0000803 Code queue 0 is empty Q1 segments left 0, axes/extruders owned 0x0000000 Code queue 1 is empty === Filament sensors === in 0 notIn 0 Extruder 0 sensor: no filament === CAN === Messages queued 165, received 379, lost 0, errs 0, boc 0 Longest wait 0ms for reply type 0, peak Tx sync delay 3, free buffers 26 (min 26), ts 92/92/0 Tx timeouts 0,0,0,0,0,0 === Network === Slowest loop: 20.33ms; fastest: 0.06ms Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) HTTP sessions: 1 of 8 === WiFi === Interface state: active Module is connected to access point Failed messages: pending 0, notrdy 0, noresp 0 Firmware version 2.1beta4 MAC address c4:5b:be:ce:91:93 Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 43120 WiFi IP address 192.168.10.x Signal strength -53dBm, channel 6, mode 802.11n, reconnections 0 Clock register 00002001 Socket states: 0 0 0 0 0 0 0 0
But just out of curiosity, as I've understood it it's some DMA transfer between the CPU and WiFi-module that's causing the crashes.
Does this mean that if I've got a print I need/want to do I could just:
- Upload print job.
- Turn off the WiFi module with
M552 S-1. - Start the print from PanelDue, and let it complete.
- Once the job is done, turn on networking again from PanelDue with
M552 S1.
Or will the DMA transfers still happen or try to happen, potentially causing a crash even if the WiFi-module is disabled?
-
@Exerqtor yes I think the crash will only occur when the WiFi module is enabled, because it's the code that talks to the WiFi module that is crashing.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 Ok, good to know if I want to run a larger more time consuming print.
-
@Exerqtor please install the firmware at https://www.dropbox.com/scl/fo/nta4k70rxl6n0bsv8q4q0/h?rlkey=1oq6j3jrvoxa6uaofyn9re3nn&dl=0 and post the M122 after it next resets.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 said in Reboots/crashes - RRF 3.5.0-rc1:
@Exerqtor please install the firmware at https://www.dropbox.com/scl/fo/nta4k70rxl6n0bsv8q4q0/h?rlkey=1oq6j3jrvoxa6uaofyn9re3nn&dl=0 and post the M122 after it next resets.
Will do!
-
@Exerqtor I had the same errors with 3.4.5 - 3.4.6 and 3.5 Rc 1 with a duet 2 wifi.
I never had this kind of errors, it started to happen when i used a smartphone as a router. I suspected from the beginning, that the error was caused due to the continuous connect/reconnect. I "solved" (it is a workaround not a fix) using a PC as an Hotspot instead the phone and the crash never happened again. Later i inspected a bit, and with almost every phone or pad i used as an hotspot will cause connection-disconnection-reconnection continously and crashes. -
@ClockWatchdog thanks, that's useful information. We know that the crash happens when something goes wrong during a transaction between the Duet and the WiFi module, but we haven't pinned it down yet.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 new crash!
=== Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-11-20 10:16:40) running on Duet 3 Mini5plus WiFi (standalone mode) Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ Used output buffers: 1 of 40 (40 max) === RTOS === Static ram: 102844 Dynamic ram: 123844 of which 0 recycled Never used RAM 11872, free system stack 186 words Tasks: NETWORK(1,ready,17.4%,234) HEAT(3,nWait,0.0%,352) Move(4,nWait,0.0%,358) CanReceiv(6,nWait,0.0%,774) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,108) MAIN(1,running,80.8%,670) IDLE(0,ready,0.3%,29) AIN(4,delaying,0.8%,264), total 100.0% Owned mutexes: WiFi(NETWORK) === Platform === Last reset 01:06:22 ago, cause: software Last software reset at 2023-11-21 16:39, reason: AssertionFailed, Gcodes spinning, available RAM 10964, slot 0 Software reset code 0x4123 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00446000 BFAR 0xe000ed38 SP 0x20011fbc Task NETW Freestk 495 ok Stack: 00000905 000af288 0002ddb7 2002bed4 2002be01 000001a8 2002c100 20030c10 2002c118 2001e888 a5a5a5a5 a5a5a5a5 00000000 00000000 00000000 00000000 20030c1c 00000800 00000002 2002c100 00000002 2002bf9d 20018668 2001e888 0003002f 00000000 00000000 Error status: 0x04 Aux0 errors 0,0,0 MCU revision 3, ADC conversions started 3983208, completed 3983208, timed out 0, errs 0 MCU temperature: min 34.8, current 35.3, max 38.2 Supply voltage: min 22.9, current 24.1, max 26.1, under voltage events: 0, over voltage events: 0, power good: yes Heap OK, handles allocated/used 99/33, heap memory allocated/used/recyclable 2048/1400/964, gc cycles 185 Events: 1 queued, 1 completed Driver 0: standstill, SG min 16, read errors 0, write errors 1, ifcnt 30, reads 12987, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 1: standstill, SG min 246, read errors 0, write errors 1, ifcnt 30, reads 12987, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 218, reads 12986, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 216, reads 12986, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 216, reads 12987, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 5: not present Driver 6: not present Date/time: 2023-11-21 17:45:13 Cache data hit count 4294967295 Slowest loop: 15.72ms; fastest: 0.14ms === Storage === Free file entries: 18 SD card 0 detected, interface speed: 22.5MBytes/sec SD card longest read time 6.1ms, write time 4.5ms, max retries 0 === Move === DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00 no step interrupt scheduled Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0 === DDARing 0 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === DDARing 1 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0 === GCodes === Movement locks held by null, null HTTP is idle in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is doing "G4 S1" in state(s) 0 0, running macro Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 File2 is idle in state(s) 0 Queue2 is idle in state(s) 0 Q0 segments left 0, axes/extruders owned 0x0000803 Code queue 0 is empty Q1 segments left 0, axes/extruders owned 0x0000000 Code queue 1 is empty === Filament sensors === in 0 notIn 0 Extruder 0 sensor: no filament === CAN === Messages queued 35855, received 81685, lost 0, errs 0, boc 0 Longest wait 2ms for reply type 6053, peak Tx sync delay 264, free buffers 26 (min 25), ts 19914/19913/0 Tx timeouts 0,0,0,0,0,0 === Network === Slowest loop: 6.22ms; fastest: 0.00ms Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) HTTP sessions: 1 of 8 === WiFi === Interface state: active Module is connected to access point Failed messages: pending 0, notrdy 0, noresp 0 Firmware version 2.1beta4 MAC address c4:5b:be:ce:91:93 Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 41332 WiFi IP address 192.168.10.x Signal strength -55dBm, channel 6, mode 802.11n, reconnections 0 Clock register 00002001 Socket states: 0 0 0 0 0 0 0 0 -
@dc42 I wondered, is it possible that the error is not exactly an error? I mean, when this disconnection-reconnection routine starts, is the number of interactions it performs too high and too concentrated? What if we reduced the speed of reconnection attempts and the number of attempts?
-
Hm. Reading through all this what your effort yielded, I wonder if whatever @dc42 might find here could also affect the (much smaller) issues I have on my Duet 3 6HC with wifi module
- dwell of about 5...10s required during boot prior to activating the module, otherwise the the module takes up to 10...15min to connect to the Wifi network
- random disconnects and
- very infrequent spontaneous reboots which I did not post here since I could not find anything useful about them yet.
I blamed the latter two on the web server and the firmware being beta in general - but who knows...?
-
@Exerqtor please try the firmware at https://www.dropbox.com/scl/fo/9hum0skl7rew21go56q7z/h?rlkey=mh1wx7mi72oz0dmk86c7uf4un&dl=0.
@NeoDue if you get any more spontaneous reboots of the Duet, please run M122 and post the output here.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 New binaries installed, saw you did some changes to the WiFi interface in this fork.
Guess that means you're on to something? wishfull thinking🙈
Just a little feedback after running the fw above for a while i've tried running a couple prints with a couple DWC instances open, and so far it haven't crashed at least. I'll try to have the printer printing for as many hours i can today with a bunch of DWC instances open to see if anything happens.
-
@dc42 There she came, new crash:
=== Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-11-24 09:09:58) running on Duet 3 Mini5plus WiFi (standalone mode) Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ Used output buffers: 10 of 40 (40 max) === RTOS === Static ram: 102844 Dynamic ram: 124692 of which 12 recycled Never used RAM 10388, free system stack 138 words Tasks: NETWORK(2,nWait,17.0%,221) HEAT(3,nWait,0.0%,352) Move(4,nWait,0.0%,244) CanReceiv(6,nWait,0.0%,797) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,341) TMC(4,delaying,0.7%,80) MAIN(1,running,80.8%,670) IDLE(0,ready,0.6%,29) AIN(4,delaying,0.8%,264), total 100.0% Owned mutexes: WiFi(NETWORK) === Platform === Last reset 00:08:14 ago, cause: software Last software reset at 2023-11-25 16:29, reason: AssertionFailed, Gcodes spinning, available RAM 5228, slot 0 Software reset code 0x4923 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00487000 BFAR 0xe000ed38 SP 0x20011fbc Task NETW Freestk 495 ok Stack: 00000907 000af288 0002dde9 2002bed4 2002be01 000001ae 2002c100 200303e0 2002c118 2001e888 0d362e30 a5a5a5a5 0a0d312e 00000000 00000000 00000000 200303ec 00000800 20035928 2002c100 20018668 2002bf9d 20018668 2001e888 0003007b 00000000 00000000 Error status: 0x04 Aux0 errors 0,0,0 MCU revision 3, ADC conversions started 494787, completed 494785, timed out 0, errs 0 MCU temperature: min 45.3, current 45.3, max 45.8 Supply voltage: min 24.0, current 24.0, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes Heap OK, handles allocated/used 99/34, heap memory allocated/used/recyclable 2048/520/64, gc cycles 28 Events: 0 queued, 0 completed Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 34, reads 863, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 1: standstill, SG min 2, read errors 0, write errors 0, ifcnt 34, reads 863, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 2: standstill, SG min 2, read errors 0, write errors 0, ifcnt 26, reads 862, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 3: standstill, SG min 2, read errors 0, write errors 0, ifcnt 27, reads 862, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 4: standstill, SG min 2, read errors 0, write errors 0, ifcnt 27, reads 863, writes 0, timeouts 0, DMA errors 0, CC errors 0 Driver 5: not present Driver 6: not present Date/time: 2023-11-25 16:37:43 Cache data hit count 846895387 Slowest loop: 10.63ms; fastest: 0.14ms === Storage === Free file entries: 18 SD card 0 detected, interface speed: 22.5MBytes/sec SD card longest read time 3.6ms, write time 4.0ms, max retries 0 === Move === DMs created 83, segments created 26, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00 no step interrupt scheduled Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0 === DDARing 0 === Scheduled moves 43, completed 43, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === DDARing 1 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0 Heater 1 is on, I-accum = 0.0 === GCodes === Movement locks held by null, null HTTP is idle in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is doing "G4 S1" in state(s) 0 0, running macro Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 File2 is idle in state(s) 0 Queue2 is idle in state(s) 0 Q0 segments left 0, axes/extruders owned 0x0000807 Code queue 0 is empty Q1 segments left 0, axes/extruders owned 0x0000000 Code queue 1 is empty === Filament sensors === in 0 notIn 0 Extruder 0 sensor: no filament === CAN === Messages queued 147, received 333, lost 0, errs 0, boc 0 Longest wait 0ms for reply type 0, peak Tx sync delay 6, free buffers 26 (min 26), ts 82/82/0 Tx timeouts 0,0,0,0,0,0 === Network === Slowest loop: 20.10ms; fastest: 0.07ms Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) HTTP sessions: 2 of 8 === WiFi === Interface state: active Module is connected to access point Failed messages: pending 0, notrdy 0, noresp 0 Firmware version 2.1beta4 MAC address c4:5b:be:ce:91:93 Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42968 WiFi IP address 192.168.10.x Signal strength -49dBm, channel 6, mode 802.11n, reconnections 0 Clock register 00002001 Socket states: 0 0 0 0 0 0 0 0
EDIT 28.11.2023:
Was it any helpfull info in the M122 report @dc42? The printer has been idle since that crash, without any new ones. -
@dc42 New crash:
M122 === Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.5.0-rc.1+ (2023-11-24 09:09:58) running on Duet 3 Mini5plus WiFi (standalone mode) Board ID: XNHXF-HR6KL-K65J0-409N2-K9W1Z-RV2MZ Used output buffers: 1 of 40 (40 max) === RTOS === Static ram: 102844 Dynamic ram: 123356 of which 0 recycled Never used RAM 12360, free system stack 188 words Tasks: NETWORK(1,ready,16.7%,219) HEAT(3,nWait,0.0%,327) Move(4,nWait,0.0%,348) CanReceiv(6,nWait,0.0%,774) CanSender(5,nWait,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait,0.7%,108) MAIN(1,running,81.5%,670) IDLE(0,ready,0.3%,29) AIN(4,delaying,0.8%,264), total 100.0% Owned mutexes: WiFi(NETWORK) === Platform === Last reset 00:29:56 ago, cause: software Last software reset at 2023-11-29 18:54, reason: HardFault imprec, Gcodes spinning, available RAM 10104, slot 1 Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00000400 ICSR 0x00000803 BFAR 0xe000ed38 SP 0x20011fe8 Task NETW Freestk 506 ok Stack: 2002c118 20030c48 200014e8 00000000 20032e7d 0002fef5 0002ffb4 610ff200 00034731 2002bed0 2002bed0 00000004 0002f635 200014e8 e000e000 00000020 0009fc25 00000000 20034fa8 00000000 20034fa8 00000000 df0bd10a e000e000 2001e888 a5a5a5a5 a5a5a5a5 Error status: 0x04 Aux0 errors 0,0,0 MCU revision 3, ADC conversions started 1796796, completed 1796794, timed out 0, errs 0 MCU temperature: min 32.9, current 33.3, max 36.5 Supply voltage: min 22.0, current 24.1, max 27.2, under voltage events: 0, over voltage events: 0, power good: yes Heap OK, handles allocated/used 99/33, heap memory allocated/used/recyclable 2048/456/20, gc cycles 84 Events: 0 queued, 0 completed Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 49, reads 29006, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 1: standstill, SG min 2, read errors 0, write errors 1, ifcnt 49, reads 29006, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 43, reads 29006, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 44, reads 29005, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 44, reads 29006, writes 13, timeouts 0, DMA errors 0, CC errors 0 Driver 5: not present Driver 6: not present Date/time: 2023-11-29 19:24:40 Cache data hit count 3045378646 Slowest loop: 12.84ms; fastest: 0.13ms === Storage === Free file entries: 18 SD card 0 detected, interface speed: 22.5MBytes/sec SD card longest read time 6.7ms, write time 4.5ms, max retries 0 === Move === DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, ebfmin 0.00, ebfmax 0.00 no step interrupt scheduled Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0 === DDARing 0 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === DDARing 1 === Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0 Heater 1 is on, I-accum = 0.0 === GCodes === Movement locks held by null, null HTTP is idle in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is doing "G4 S1" in state(s) 0 0, running macro Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 File2 is idle in state(s) 0 Queue2 is idle in state(s) 0 Q0 segments left 0, axes/extruders owned 0x0000803 Code queue 0 is empty Q1 segments left 0, axes/extruders owned 0x0000000 Code queue 1 is empty === Filament sensors === in 0 notIn 0 Extruder 0 sensor: no filament === CAN === Messages queued 16181, received 36876, lost 0, errs 1, boc 0 Longest wait 2ms for reply type 6053, peak Tx sync delay 357, free buffers 26 (min 25), ts 8984/8983/0 Tx timeouts 0,0,0,0,0,0 === Network === Slowest loop: 6.04ms; fastest: 0.00ms Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) HTTP sessions: 1 of 8 === WiFi === Interface state: active Module is connected to access point Failed messages: pending 0, notrdy 0, noresp 0 Firmware version 2.1beta4 MAC address c4:5b:be:ce:91:93 Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42796 WiFi IP address 192.168.10.x Signal strength -47dBm, channel 6, mode 802.11n, reconnections 0 Clock register 00002001 Socket states: 0 0 0 0 0 0 0 0
