@droftarts said in Duet 2 WiFi network slowdowns and board restarts:

@ProteanReverie It's a long shot, but... it does seem possible that Duets are a bit susceptible to malformed Ethernet packets coming from the network. Some other users have reported similar issues, and the Hardfaults seem to be related to networking, and occasionally oddly setup browsers. @dc42 has often asked what extra languages are installed in these cases, as it seems foreign characters or character sets can trigger the Duet crash.

What made me think of this is when you say you have '3 blink cameras' on your network. Security cameras are a favourite target for hackers, that can then be used to snoop on the local network (strange packets) as well as launch DoS attacks. The Blink camera does seem susceptible: https://www.electronicshub.org/can-blink-cameras-be-hacked/

Running a WireShark scan of the network for a while, and capturing any strange packets, particularly at the point when the Duet crashes, may be useful. It would be great to track this one down!

Ian

No extra languages are set up on any of the PCs or tablets, or any other devices to my knowledge.

I am aware to some extent the security issues present with networked cameras and will keep an eye on that. Nothing that I have seen so far has indicated to me that they have been compromised at any point, but those kinds of things not being able to be noticed is usually a key point so can't say for sure.

I have got wireshark set up, and next time I see issues I will run a capture focused on the IP of one of the machines to share here. I plan to take it through a file upload (which will likely fail) as well as some network interface resets which hopefully cause the restart while doing that capture. If there are other actions I should take during that capture, please let me know.

@dc42 said in Duet 2 WiFi network slowdowns and board restarts:

@ProteanReverie it might be worth setting up a macro that disables wifi using M552 S-1, delays for a few seconds using G4, then restarts wifi using M552 S1. Resetting the connection in this way may be simpler than resetting the Duet or the router.

I actually wrote that exact macro a couple of years ago when having other issues which were later resolved. I still use it from time to time when I see this issue crop up, though often times when the issue is present the interface will be so inaccessible that I cannot activate the macro so it is easier to connect via serial to send commands or just power cycle the machine. Thank you for the recommendation.

Another occurance over the weekend.

M122

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: 65LYX-8Q6KL-K65J0-409N0-NP02Z-ZFK3T
Used output buffers: 14 of 40 (18 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 119916 of which 24 recycled
Never used RAM 15388, free system stack 150 words
Tasks: NETWORK(2,nWait 7,580.1%,218) HEAT(3,nWait 6,0.4%,372) Move(4,nWait 6,0.1%,355) CanReceiv(6,nWait 1,0.6%,808) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.3%,348) TMC(4,nWait 6,34.6%,102) MAIN(1,running,35.1%,677) IDLE(0,ready,0.3%,30) AIN(4,delaying,36.4%,260), total 688.1%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 11:24:14 ago, cause: software
Last software reset at 2024-06-23 21:22, reason: HardFault bfarValid precise, Gcodes spinning, available RAM 15364, slot 0
Software reset code 0x0063 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x00000803 BFAR 0x00000004 SP 0x20011ff0 Task NETW Freestk 483 ok
Stack: 00000160 00000002 200014e8 0000015f ffffffff 000a07ef 0002ff8e 810f0000 0002ff85 00000000 00000000 00000000 2002fdcc 00000800 20035308 2002c5f8 200187fc 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 ffffffff
Error status: 0x00
MCU revision 3, ADC conversions started 41054418, completed 41054418, timed out 0, errs 0
MCU temperature: min 38.2, current 39.6, max 41.0
Supply voltage: min 24.1, current 24.1, max 24.2, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/12/12, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 43, reads 63568, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 43, reads 63568, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 43, reads 63568, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 43, reads 63568, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 43, reads 63568, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-06-24 08:46:40
Cache data hit count 4294967295
Slowest loop: 7.67ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 3.2ms, write time 3.8ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters -1 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000000
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 205271, received 328436, lost 0, errs 1, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 390, free buffers 26 (min 26), ts 205271/205270/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 7.95ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(1) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:d0
Module reset reason: Power up, Vcc 3.42, flash size 2097152, free heap 29568
WiFi IP address 10.1.10.46
Signal strength -59dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 3 0 0 0 0 0 0 0

Relevant log file section

2024-06-21 20:23:42 [warn] HTTP client 10.1.10.9 login succeeded (session key 2318516549)
2024-06-22 12:46:56 [warn] HTTP client 10.1.10.9 login succeeded (session key 2498040479)
2024-06-22 12:51:11 [warn] HTTP client 10.1.10.9 login succeeded (session key 2882910232)
2024-06-23 12:58:37 [warn] HTTP client 10.1.10.9 login succeeded (session key 2274061159)
2024-06-23 13:14:38 [warn] HTTP client 10.1.10.9 login succeeded (session key 4219374896)
2024-06-23 13:14:58 [warn] HTTP client 10.1.10.9 login succeeded (session key 1587684493)
2024-06-23 13:15:18 [warn] HTTP client 10.1.10.9 login succeeded (session key 871240276)
2024-06-23 18:32:45 [warn] HTTP client 10.1.10.9 login succeeded (session key 640519980)
2024-06-23 20:18:14 [warn] HTTP client 10.1.10.9 login succeeded (session key 1301043269)
2024-06-23 20:19:52 [warn] HTTP client 10.1.10.9 login succeeded (session key 3916440988)
2024-06-23 20:20:20 [warn] HTTP client 10.1.10.9 login succeeded (session key 3232781641)
2024-06-23 20:20:40 [warn] HTTP client 10.1.10.9 login succeeded (session key 1725357550)
2024-06-23 20:20:56 [warn] HTTP client 10.1.10.9 login succeeded (session key 2497689280)
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.1 (2024-04-19 14:41:25)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:00 [warn] WiFi module started
power up + 00:00:08 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.46
power up + 00:00:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 3424165787)
2024-06-23 21:22:35 [warn] Date and time set at power up + 00:00:08
2024-06-23 21:54:45 [warn] HTTP client 10.1.10.9 login succeeded (session key 835669197)
2024-06-23 21:54:54 [warn] HTTP client 10.1.10.9 login succeeded (session key 1712060142)
2024-06-23 21:55:04 [warn] HTTP client 10.1.10.9 login succeeded (session key 2620441938)
2024-06-23 21:55:12 [warn] HTTP client 10.1.10.9 login succeeded (session key 2547895376)
2024-06-23 21:55:35 [warn] HTTP client 10.1.10.9 login succeeded (session key 519733317)
2024-06-24 00:17:19 [warn] HTTP client 10.1.10.9 login succeeded (session key 1091016330)
2024-06-24 00:17:48 [warn] HTTP client 10.1.10.9 login succeeded (session key 598095062)
2024-06-24 00:18:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 994352261)
2024-06-24 00:57:15 [warn] HTTP client 10.1.10.9 login succeeded (session key 852304322)
2024-06-24 04:59:31 [warn] HTTP client 10.1.10.9 login succeeded (session key 713145612)
2024-06-24 08:46:40 [debug] === Diagnostics ===

No occurrences through the week last week, but did have another over the weekend.

M122

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.1 (2024-04-19 14:41:25) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: 65LYX-8Q6KL-K65J0-409N0-NP02Z-ZFK3T
Used output buffers: 1 of 40 (18 max)
=== RTOS ===
Static ram: 103232
Dynamic ram: 119940 of which 24 recycled
Never used RAM 15364, free system stack 156 words
Tasks: NETWORK(1,ready,419.7%,218) HEAT(3,nWait 6,0.7%,356) Move(4,nWait 6,0.2%,341) CanReceiv(6,nWait 1,1.0%,808) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.6%,348) TMC(4,nWait 6,57.1%,102) MAIN(1,running,87.1%,677) IDLE(0,ready,0.5%,30) AIN(4,delaying,60.0%,260), total 627.0%
Owned mutexes:
=== Platform ===
Last reset 20:58:29 ago, cause: software
Last software reset at 2024-06-30 11:35, reason: HardFault bfarValid precise, Platform spinning, available RAM 15364, slot 1
Software reset code 0x0060 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x00000803 BFAR 0x00000004 SP 0x20011ff0 Task NETW Freestk 483 ok
Stack: 00000160 00000002 200014e8 0000015f ffffffff 000a07ef 0002ff8e 810f0000 0002ff85 00000000 00000000 00000000 200305e4 00000800 2002c5f8 2002c5f8 00000001 2002c41e 200187fc 2001e9f8 0003011f 00000000 00000000 00000000 200120a0 00000014 20468121
Error status: 0x00
MCU revision 3, ADC conversions started 75510090, completed 75510090, timed out 0, errs 0
MCU temperature: min 38.7, current 39.8, max 41.4
Supply voltage: min 24.0, current 24.1, max 24.2, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/12/12, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 54, reads 42014, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 54, reads 42014, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 54, reads 42014, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 54, reads 42014, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 54, reads 42015, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-07-01 08:34:24
Cache data hit count 4294967295
Slowest loop: 7.01ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 0.7ms, write time 2.6ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters -1 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000000
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 377549, received 604082, lost 0, errs 1, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 297, free buffers 26 (min 26), ts 377549/377548/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 7.38ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:d0
Module reset reason: Power up, Vcc 3.39, flash size 2097152, free heap 42784
WiFi IP address 10.1.10.46
Signal strength -60dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

Log.txt

2024-06-26 13:58:26 [warn] HTTP client 10.1.10.9 login succeeded (session key 3446445976)
2024-06-27 07:44:43 [warn] HTTP client 10.1.10.9 login succeeded (session key 2598975707)
2024-06-27 11:34:42 [warn] HTTP client 10.1.10.9 login succeeded (session key 3419397635)
2024-06-27 12:00:55 [warn] HTTP client 10.1.10.9 login succeeded (session key 3802106080)
2024-06-27 14:17:11 [warn] HTTP client 10.1.10.9 login succeeded (session key 3855690080)
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.1 (2024-04-19 14:41:25)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:00 [warn] WiFi module started
power up + 00:00:07 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.46
power up + 00:00:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 526990551)
2024-06-30 11:36:03 [warn] Date and time set at power up + 00:00:08
2024-06-30 13:26:46 [warn] HTTP client 10.1.10.9 login succeeded (session key 2188381128)
2024-06-30 13:26:51 [warn] HTTP client 10.1.10.9 login succeeded (session key 1375067276)
2024-06-30 13:26:54 [warn] HTTP client 10.1.10.9 login succeeded (session key 587158328)
2024-06-30 13:27:19 [warn] HTTP client 10.1.10.9 login succeeded (session key 3511153781)
2024-06-30 14:25:24 [warn] HTTP client 10.1.10.9 login succeeded (session key 2762896906)
2024-07-01 01:18:55 [warn] HTTP client 10.1.10.9 login succeeded (session key 96132253)
2024-07-01 01:20:58 [warn] HTTP client 10.1.10.9 login succeeded (session key 1951105233)
2024-07-01 08:34:24 [debug] === Diagnostics ===

Wanted to report in that following the changing of the SD cards I mentioned around 7/30/24 my benchtop board has not seen a single restart and currently has over 1000 hours of uptime. The other board which is running in a machine has not seen any issues since my last reported crash in this thread on 8/1/24, which was a crash that happened after the SD cards were changed. That machine has seen various restarts and downtime since then, due to other factors.

It would appear that my previous post had exactly the effect I was hoping for - the board that is in a machine crashed shortly after I posted, for the first time in over a month. Machine was idle in between prints when this occurred.

M122:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (36 max)
Error in macro line 17 while starting up: Driver 1.0 does not exist
Driver 1.1 does not exist
Driver 1.3 does not exist
Driver 1.4 does not exist
Driver 1.2 does not exist
=== RTOS ===
Static ram: 103368
Dynamic ram: 122336 of which 0 recycled
Never used RAM 12856, free system stack 202 words
Tasks: NETWORK(1,ready,13.8%,203) HEAT(3,nWait 6,0.0%,350) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,796) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,348) TMC(4,nWait 6,0.8%,101) MAIN(1,running,84.4%,665) IDLE(0,ready,0.0%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:43:24 ago, cause: software
Last software reset at 2024-09-11 12:29, reason: HardFault bfarValid precise, Gcodes spinning, available RAM 12024, slot 2
Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x00487803 BFAR 0x0000001c SP 0x20012008 Task NETW Freestk 482 ok
Stack: 2002c658 200317f0 200014e8 00000000 200331a7 0003015d 00030070 610f0000 00030149 00000000 00000000 00000000 200317fc 00000800 20035d08 2002c640 2001882c 2002c496 2001882c 2001ea80 000302e3 00000000 00000000 00000000 200120b8 00000014 4738ebf6
Error status: 0x00
MCU revision 3, ADC conversions started 2604593, completed 2604593, timed out 0, errs 0
MCU temperature: min 38.6, current 39.1, max 44.4
Supply voltage: min 23.8, current 23.9, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/140/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 1, ifcnt 32, reads 5983, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 32, reads 5983, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 5983, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 1, ifcnt 32, reads 5982, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 5983, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-09-11 13:08:49
Cache data hit count 4294967295
Slowest loop: 21.48ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 1.8ms, write time 113.6ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 2 is on, I-accum = 0.0
Heater 4 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 24753400
Extruder 0 sensor: no filament
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 23463, received 52125, lost 0, errs 1, boc 0
Longest wait 1ms for reply type 6018, peak Tx sync delay 253, free buffers 26 (min 25), ts 13023/13022/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 118.64ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:b7
Module reset reason: Power up, Vcc 3.40, flash size 2097152, free heap 42764
WiFi IP address 10.1.10.21
Signal strength -57dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

Log:

2024-09-11 12:15:04 [warn] Finished printing file 0:/gcodes/PLA-2.5Atom-spiral_40base30degree_x_100tall_Cone.gcode, print time was 0h 33m
2024-09-11 12:15:44 [warn] HTTP client 10.1.10.161 login succeeded (session key 359056603)
2024-09-11 12:16:06 [warn] HTTP client 10.1.10.161 login succeeded (session key 3366296215)
2024-09-11 12:16:23 [warn] HTTP client 10.1.10.161 login succeeded (session key 3162514298)
2024-09-11 12:16:35 [warn] HTTP client 10.1.10.161 login succeeded (session key 3095629251)
2024-09-11 12:16:50 [warn] HTTP client 10.1.10.161 login succeeded (session key 3273569751)
2024-09-11 12:17:44 [warn] HTTP client 10.1.10.161 login succeeded (session key 3524598834)
2024-09-11 12:18:17 [warn] HTTP client 10.1.10.161 login succeeded (session key 2236099279)
2024-09-11 12:18:50 [warn] HTTP client 10.1.10.161 login succeeded (session key 2038503451)
2024-09-11 12:19:45 [warn] HTTP client 10.1.10.161 login succeeded (session key 3308195184)
2024-09-11 12:20:35 [warn] HTTP client 10.1.10.161 login succeeded (session key 946720536)
2024-09-11 12:21:28 [warn] HTTP client 10.1.10.161 login succeeded (session key 1138736952)
2024-09-11 12:21:50 [warn] HTTP client 10.1.10.161 login succeeded (session key 3108662693)
2024-09-11 12:22:43 [warn] HTTP client 10.1.10.161 login succeeded (session key 2976044894)
2024-09-11 12:23:05 [warn] HTTP client 10.1.10.161 login succeeded (session key 1080405183)
2024-09-11 12:23:17 [warn] HTTP client 10.1.10.161 login succeeded (session key 1254727083)
2024-09-11 12:23:39 [warn] HTTP client 10.1.10.161 login succeeded (session key 614652800)
2024-09-11 12:24:10 [warn] HTTP client 10.1.10.161 login succeeded (session key 2640235784)
2024-09-11 12:24:57 [warn] HTTP client 10.1.10.161 login succeeded (session key 3181799873)
2024-09-11 12:25:53 [warn] HTTP client 10.1.10.161 login succeeded (session key 1776770804)
2024-09-11 12:26:45 [warn] HTTP client 10.1.10.161 login succeeded (session key 2156320815)
2024-09-11 12:27:18 [warn] HTTP client 10.1.10.161 login succeeded (session key 2647272730)
2024-09-11 12:27:30 [warn] HTTP client 10.1.10.161 login succeeded (session key 1370953698)
2024-09-11 12:27:57 [warn] HTTP client 10.1.10.161 login succeeded (session key 831992087)
2024-09-11 12:28:36 [warn] HTTP client 10.1.10.161 login succeeded (session key 1995411029)
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.2 (2024-06-11 17:14:16)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:01 [warn] WiFi module started
power up + 00:00:11 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.21
power up + 00:00:14 [warn] HTTP client 10.1.10.9 login succeeded (session key 2317065672)
2024-09-11 12:25:39 [warn] Date and time set at power up + 00:00:14
2024-09-11 12:26:07 [warn] HTTP client 10.1.10.161 login succeeded (session key 2214435877)
2024-09-11 12:27:17 [warn] HTTP client 10.1.10.161 login succeeded (session key 2319523673)
2024-09-11 12:27:17 [warn] HTTP client 10.1.10.161 login succeeded (session key 4186603624)
2024-09-11 12:27:17 [warn] HTTP client 10.1.10.161 attempted login but no more sessions available
2024-09-11 12:27:17 [warn] HTTP client 10.1.10.161 attempted login but no more sessions available
2024-09-11 12:27:17 [warn] HTTP client 10.1.10.161 attempted login but no more sessions available
2024-09-11 13:08:49 [debug] === Diagnostics ===

Wanted to report in that following the changing of the SD cards I mentioned around 7/30/24 my benchtop board has not seen a single restart and currently has over 1000 hours of uptime. The other board which is running in a machine has not seen any issues since my last reported crash in this thread on 8/1/24, which was a crash that happened after the SD cards were changed. That machine has seen various restarts and downtime since then, due to other factors.

Had another case today. This time it only appeared to only be an issue one Duet system, but it seemed like the usual issue from this thread. The slowdown/inaccessibility issue persisted through multiple power-cycles, multiple uses of [M552 S-1] followed by [M552 S1], and switching to host mode with [M552 S2] and then going back to client mode. Everything seemed to work just fine in host mode, as usual. There were no resets during this.

This Duet 2 was running on firmware version 3.6.0A2+3. A Duet 3 also running 3.6.0A2+3 did not appear to be affected. Additionally, two Duet 2s running 3.5.2 did not seem to exhibit symptoms and DWC stayed connected the whole time I was monitoring them while the one with the issue was having its problem.

I ran a packet capture, the Duet is 10.1.10.38. Before long this devolved into only showing requests from my PC (10.1.10.9) to find the Duet, while the Duet seemed completely unreachable. I stopped the capture in case the connection would not be recovered to prevent a needlessly long file. That is the first packet capture here. Then, in case it did reconnect I started a new capture, which did see a point where the Duet was able to re-establish communications, though it continued to disconnect from DWC shortly after any time it did connect.

8-9-24-A3-D2-idle-disconnect.pcapng
8-9-24-A3-D2-idle-reconnect.pcapng

I was also able to get M122 data via serial console, which I have pulled to share here from the log file. The first M122 was run during the period of time where the packet capture showed the board as completely unreachable, right around where the capture was stopped and then restarted. The second M122 was run shortly after it reconnected, while still being too unstable to get the M122 through DWC.

Disconnected:

2024-08-09 12:09:57 [debug] WiFi module is connected to access point SD3D-24, IP address 10.1.10.38
ok
2024-08-09 12:24:13 [debug] === Diagnostics ===
2024-08-09 12:24:13 [debug] RepRapFirmware for Duet 2 WiFi/Ethernet version 3.6.0-alpha.2+3 (2024-07-17 20:03:21) running on Duet WiFi 1.02 or later + DueX5v0.11
2024-08-09 12:24:13 [debug] Board ID: 0JD2M-9F8TA-GJ4TN-6J1DD-3S46S-17SU6
2024-08-09 12:24:13 [debug] Used output buffers: 1 of 26 (24 max)
2024-08-09 12:24:13 [debug] === RTOS ===
2024-08-09 12:24:13 [debug] Static ram: 23368
2024-08-09 12:24:13 [debug] Dynamic ram: 70144 of which 12 recycled
2024-08-09 12:24:13 [debug] Never used RAM 24192, free system stack 134 words
2024-08-09 12:24:13 [debug] Tasks:
2024-08-09 12:24:13 [debug]  NETWORK(1,ready,64.1%,217)
2024-08-09 12:24:13 [debug]  ACCEL(6,nWait 5,0.0%,249)
2024-08-09 12:24:13 [debug]  HEAT(3,nWait 5,0.1%,328)
2024-08-09 12:24:13 [debug]  Move(4,nWait 5,0.0%,274)
2024-08-09 12:24:13 [debug]  DUEX(5,nWait 5,0.0%,23)
2024-08-09 12:24:13 [debug]  MAIN(1,running,35.8%,501)
2024-08-09 12:24:13 [debug]  IDLE(0,ready,0.0%,29)
2024-08-09 12:24:13 [debug] , total 100.0%
Owned mutexes:
2024-08-09 12:24:13 [debug]  WiFi(NETWORK)
2024-08-09 12:24:13 [debug]  USB(MAIN)
2024-08-09 12:24:13 [debug] === Platform ===
2024-08-09 12:24:13 [debug] Last reset 01:18:37 ago, cause: power up
2024-08-09 12:24:13 [debug] Last software reset at 2024-08-06 15:49, reason: HeatTaskStuck, Gcodes spinning, available RAM 26972, slot 0
2024-08-09 12:24:13 [debug] Software reset code 0x4143 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0041f80f BFAR 0xe000ed38 SP 0x20004818 Task ACCE Freestk 4294954872 ok
2024-08-09 12:24:13 [debug] Stack: 00000000 200041bc 10000000 e000e000 006dc7e3 0045d535 0045dc80 61000000 0045dc69 20005370 2000d448 00011d4a 2000d478 200041a8 200041b0 20004c78 0042f4a5 2000d448 a5a5a5a5 a5a5a5a5 20004c78 a5a5a5a5 0042f54b a5a5a5a5 0045c459 a5a5a5a5 a5a5a5a5
2024-08-09 12:24:13 [debug] Error status: 0x00
2024-08-09 12:24:13 [debug] MCU temperature: min 32.3, current 33.0, max 34.3
2024-08-09 12:24:13 [debug] Supply voltage: min 24.1, current 24.3, max 24.6, under voltage events: 0, over voltage events: 0, power good: yes
2024-08-09 12:24:13 [debug] Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/12/12, gc cycles 0
2024-08-09 12:24:13 [debug] Events: 0 queued, 0 completed
2024-08-09 12:24:13 [debug] Date/time: 
2024-08-09 12:24:13 [debug] 2024-08-09 12:24:13
2024-08-09 12:24:13 [debug] Slowest loop: 1005.96ms; fastest: 0.21ms
2024-08-09 12:24:13 [debug] I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
2024-08-09 12:24:13 [debug] === Storage ===
Free file entries: 9
2024-08-09 12:24:13 [debug] SD card 0 detected, interface speed: 20.0MBytes/sec
2024-08-09 12:24:13 [debug] SD card longest read time 104.0ms, write time 101.0ms, max retries 0
2024-08-09 12:24:13 [debug] === Move ===
Segments created 15, maxWait 1915146ms, bed comp in use: none, height map offset 0.000, hiccups added 0 (0.00ms), max steps late 0, ebfmin 0.00, ebfmax 0.00, mcet 0.025
2024-08-09 12:24:13 [debug] Pos req/act/dcf: 80000.00/79999/1.00 -4.00/-3/-1.00 6400.00/6400/0.00 88056.00/88056/0.00
2024-08-09 12:24:13 [debug] no step interrupt scheduled
2024-08-09 12:24:13 [debug] === DDARing 0 ===
Scheduled moves 4, completed 4, LaErrors 0, Underruns [0, 0, 0]
2024-08-09 12:24:13 [debug] Driver 0: standstill, SG min 0
2024-08-09 12:24:13 [debug] Driver 1: standstill, SG min 5
2024-08-09 12:24:13 [debug] Driver 2: standstill, SG min n/a
2024-08-09 12:24:13 [debug] Driver 3: standstill, SG min n/a
2024-08-09 12:24:13 [debug] Driver 4: standstill, SG min n/a
2024-08-09 12:24:13 [debug] Driver 5: standstill, SG min n/a
2024-08-09 12:24:13 [debug] Driver 6: standstill, SG min n/a
2024-08-09 12:24:13 [debug] Driver 7: standstill, SG min n/a
2024-08-09 12:24:13 [debug] Driver 8: standstill, SG min n/a
2024-08-09 12:24:13 [debug] Driver 9: standstill, SG min n/a
2024-08-09 12:24:13 [debug] Driver 10: 
2024-08-09 12:24:13 [debug] Driver 11: 
2024-08-09 12:24:13 [debug] === Heat ===
2024-08-09 12:24:13 [debug] Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
2024-08-09 12:24:13 [debug] Heater 2 is on, I-accum = 0.0
2024-08-09 12:24:13 [debug] === GCodes ===
2024-08-09 12:24:13 [debug] Movement locks held by null
2024-08-09 12:24:13 [debug] HTTP is idle in state(s) 0
2024-08-09 12:24:13 [debug] Telnet is idle in state(s) 0
2024-08-09 12:24:13 [debug] File is idle in state(s) 0
2024-08-09 12:24:13 [debug] USB is ready with "m122" in state(s) 0
2024-08-09 12:24:13 [debug] Aux is idle in state(s) 0
2024-08-09 12:24:13 [debug] Trigger is idle in state(s) 0
2024-08-09 12:24:13 [debug] Queue is idle in state(s) 0
2024-08-09 12:24:13 [debug] LCD is idle in state(s) 0
2024-08-09 12:24:13 [debug] Daemon is idle in state(s) 0
2024-08-09 12:24:13 [debug] Autopause is idle in state(s) 0
2024-08-09 12:24:13 [debug] Q0 segments left 0
2024-08-09 12:24:13 [debug] Code queue 0 is empty
2024-08-09 12:24:13 [debug] === Filament sensors ===
check 0 clear 14997920
2024-08-09 12:24:13 [debug] Extruder 0 sensor: ok
2024-08-09 12:24:13 [debug] Extruder 1 sensor: ok
2024-08-09 12:24:13 [debug] === DueX ===
Read count 1, 0.01 reads/min
2024-08-09 12:24:13 [debug] === Network ===
2024-08-09 12:24:13 [debug] Slowest loop: 302.78ms; fastest: 0.00ms
2024-08-09 12:24:13 [debug] Responder states:
2024-08-09 12:24:13 [debug]  HTTP(0)
2024-08-09 12:24:13 [debug]  HTTP(0)
2024-08-09 12:24:13 [debug]  HTTP(0)
2024-08-09 12:24:13 [debug]  FTP(0)
2024-08-09 12:24:13 [debug]  Telnet(0)
2024-08-09 12:24:13 [debug] HTTP sessions: 0 of 8
2024-08-09 12:24:13 [debug] === WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 16435, noresp 41
2024-08-09 12:24:13 [debug] Failed to get WiFi status
2024-08-09 12:24:13 [debug] Socket states:
2024-08-09 12:24:13 [debug]  6
2024-08-09 12:24:13 [debug]  6
2024-08-09 12:24:13 [debug]  0
2024-08-09 12:24:13 [debug]  4
2024-08-09 12:24:13 [debug]  0
2024-08-09 12:24:13 [debug]  0
2024-08-09 12:24:13 [debug]  0
2024-08-09 12:24:13 [debug]  0
2024-08-09 12:24:13 [debug] ok

Post re-connect, unstable:

2024-08-09 12:49:30 [debug] === Diagnostics ===
2024-08-09 12:49:30 [debug] RepRapFirmware for Duet 2 WiFi/Ethernet version 3.6.0-alpha.2+3 (2024-07-17 20:03:21) running on Duet WiFi 1.02 or later + DueX5v0.11
2024-08-09 12:49:30 [debug] Board ID: 0JD2M-9F8TA-GJ4TN-6J1DD-3S46S-17SU6
2024-08-09 12:49:30 [debug] Used output buffers: 1 of 26 (24 max)
2024-08-09 12:49:30 [debug] === RTOS ===
2024-08-09 12:49:30 [debug] Static ram: 23368
2024-08-09 12:49:30 [debug] Dynamic ram: 70144 of which 12 recycled
2024-08-09 12:49:30 [debug] Never used RAM 24192, free system stack 134 words
2024-08-09 12:49:30 [debug] Tasks:
2024-08-09 12:49:30 [debug]  NETWORK(1,ready,48.4%,217)
2024-08-09 12:49:30 [debug]  ACCEL(6,nWait 5,0.0%,249)
2024-08-09 12:49:30 [debug]  HEAT(3,nWait 5,0.1%,328)
2024-08-09 12:49:30 [debug]  Move(4,nWait 5,0.0%,274)
2024-08-09 12:49:30 [debug]  DUEX(5,nWait 5,0.0%,23)
2024-08-09 12:49:30 [debug]  MAIN(1,running,51.5%,501)
2024-08-09 12:49:30 [debug]  IDLE(0,ready,0.0%,29)
2024-08-09 12:49:30 [debug] , total 100.0%
Owned mutexes:
2024-08-09 12:49:30 [debug]  WiFi(NETWORK)
2024-08-09 12:49:30 [debug] === Platform ===
2024-08-09 12:49:30 [debug] Last reset 01:43:54 ago, cause: power up
2024-08-09 12:49:30 [debug] Last software reset at 2024-08-06 15:49, reason: HeatTaskStuck, Gcodes spinning, available RAM 26972, slot 0
2024-08-09 12:49:30 [debug] Software reset code 0x4143 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0041f80f BFAR 0xe000ed38 SP 0x20004818 Task ACCE Freestk 4294954872 ok
2024-08-09 12:49:30 [debug] Stack: 00000000 200041bc 10000000 e000e000 006dc7e3 0045d535 0045dc80 61000000 0045dc69 20005370 2000d448 00011d4a 2000d478 200041a8 200041b0 20004c78 0042f4a5 2000d448 a5a5a5a5 a5a5a5a5 20004c78 a5a5a5a5 0042f54b a5a5a5a5 0045c459 a5a5a5a5 a5a5a5a5
2024-08-09 12:49:30 [debug] Error status: 0x00
2024-08-09 12:49:30 [debug] MCU temperature: min 32.4, current 33.3, max 33.8
2024-08-09 12:49:30 [debug] Supply voltage: min 24.1, current 24.3, max 24.6, under voltage events: 0, over voltage events: 0, power good: yes
2024-08-09 12:49:30 [debug] Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/12/12, gc cycles 0
2024-08-09 12:49:30 [debug] Events: 0 queued, 0 completed
2024-08-09 12:49:30 [debug] Date/time: 
2024-08-09 12:49:30 [debug] 2024-08-09 12:49:30
2024-08-09 12:49:30 [debug] Slowest loop: 673.42ms; fastest: 0.21ms
2024-08-09 12:49:30 [debug] I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
2024-08-09 12:49:30 [debug] === Storage ===
Free file entries: 9
2024-08-09 12:49:30 [debug] SD card 0 detected, interface speed: 20.0MBytes/sec
2024-08-09 12:49:30 [debug] SD card longest read time 104.0ms, write time 100.8ms, max retries 0
2024-08-09 12:49:30 [debug] === Move ===
Segments created 15, maxWait 0ms, bed comp in use: none, height map offset 0.000, hiccups added 0 (0.00ms), max steps late 0, ebfmin 0.00, ebfmax 0.00, mcet 0.000
2024-08-09 12:49:30 [debug] Pos req/act/dcf: 79999.00/79999/1.00 -3.00/-3/-1.00 6400.00/6400/0.00 88056.00/88056/0.00
2024-08-09 12:49:30 [debug] no step interrupt scheduled
2024-08-09 12:49:30 [debug] === DDARing 0 ===
Scheduled moves 4, completed 4, LaErrors 0, Underruns [0, 0, 0]
2024-08-09 12:49:30 [debug] Driver 0: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 1: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 2: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 3: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 4: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 5: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 6: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 7: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 8: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 9: standstill, SG min n/a
2024-08-09 12:49:30 [debug] Driver 10: 
2024-08-09 12:49:30 [debug] Driver 11: 
2024-08-09 12:49:30 [debug] === Heat ===
2024-08-09 12:49:30 [debug] Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
2024-08-09 12:49:30 [debug] Heater 2 is on, I-accum = 0.0
2024-08-09 12:49:30 [debug] === GCodes ===
2024-08-09 12:49:30 [debug] Movement locks held by null
2024-08-09 12:49:30 [debug] HTTP is idle in state(s) 0
2024-08-09 12:49:30 [debug] Telnet is idle in state(s) 0
2024-08-09 12:49:30 [debug] File is idle in state(s) 0
2024-08-09 12:49:30 [debug] USB is idle in state(s) 0
2024-08-09 12:49:30 [debug] Aux is idle in state(s) 0
2024-08-09 12:49:30 [debug] Trigger is idle in state(s) 0
2024-08-09 12:49:30 [debug] Queue is idle in state(s) 0
2024-08-09 12:49:30 [debug] LCD is idle in state(s) 0
2024-08-09 12:49:30 [debug] Daemon is idle in state(s) 0
2024-08-09 12:49:30 [debug] Autopause is idle in state(s) 0
2024-08-09 12:49:30 [debug] Q0 segments left 0
2024-08-09 12:49:30 [debug] Code queue 0 is empty
2024-08-09 12:49:30 [debug] === Filament sensors ===
check 0 clear 21956288
2024-08-09 12:49:30 [debug] Extruder 0 sensor: ok
2024-08-09 12:49:30 [debug] Extruder 1 sensor: ok
2024-08-09 12:49:30 [debug] === DueX ===
Read count 0, 0.00 reads/min
2024-08-09 12:49:30 [debug] === Network ===
2024-08-09 12:49:30 [debug] Slowest loop: 302.74ms; fastest: 0.08ms
2024-08-09 12:49:30 [debug] Responder states:
2024-08-09 12:49:30 [debug]  HTTP(0)
2024-08-09 12:49:30 [debug]  HTTP(0)
2024-08-09 12:49:30 [debug]  HTTP(0)
2024-08-09 12:49:30 [debug]  FTP(0)
2024-08-09 12:49:30 [debug]  Telnet(0)
2024-08-09 12:49:30 [debug] HTTP sessions: 1 of 8
2024-08-09 12:49:30 [debug] === WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 22587, noresp 91
2024-08-09 12:49:30 [debug] Firmware version 2.1.0
2024-08-09 12:49:30 [debug] MAC address 48:55:19:09:31:80
2024-08-09 12:49:30 [debug] Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 23616
2024-08-09 12:49:30 [debug] WiFi IP address 10.1.10.38
2024-08-09 12:49:30 [debug] Signal strength -59dBm, channel 1, mode 802.11n, reconnections 0
2024-08-09 12:49:30 [debug] Clock register 00002002
2024-08-09 12:49:30 [debug] Socket states:
2024-08-09 12:49:30 [debug]  4
2024-08-09 12:49:30 [debug]  0
2024-08-09 12:49:30 [debug]  4
2024-08-09 12:49:30 [debug]  4
2024-08-09 12:49:30 [debug]  0
2024-08-09 12:49:30 [debug]  0
2024-08-09 12:49:30 [debug]  0
2024-08-09 12:49:30 [debug]  0

Since then, I had the Duet in host mode for about a half an hour to get some other things done, and then switched back to client mode. The issue was no longer present after switching back that time.

@ProteanReverie said in RepRapFirmware 3.6.0-alpha.2 for Duet main boards available:

These tests were done with a Hemera Revo 0.4HF nozzle and PETG material. I am also starting tests with a 1.0 nozzle but do not have that data yet.

Here is data from my 1.0 nozzle tests.

Duet 2

accel test
3.5.2 = Never used RAM 11736, free system stack 112 words
3.6A2+3 = Never used RAM 22616, free system stack 116 words

benchy
3.5.2 = Never used RAM 11664, free system stack 122 words
3.6A2+3 = Never used RAM 18992, free system stack 116 words

ghost text
3.5.2 = Never used RAM 11808, free system stack 114 words
3.6A2+3 = Never used RAM 22112, free system stack 114 words

---

Duet 3

accel test
3.5.2 = Never used RAM 12068, free system stack 130 words
3.6A2+3 = Never used RAM 40076, free system stack 130 words

benchy
3.5.2 = Never used RAM 11972, free system stack 128 words
3.6A2+3 = Never used RAM 36236, free system stack 130 words

ghost text
3.5.2 = Never used RAM 12092, free system stack 128 words
3.6A2+3 = Never used RAM 39788, free system stack 130 words

---

additionally, here is data from the duet 3 using a 0.4 nozzle for the accel test model, since that data wasn't ready for my prior post
3.5.2 = Never used RAM 11876, free system stack 128 words
3.6A2+3 = Never used RAM 39884, free system stack 130 words

---

I have still not encountered any noticeable issues on either the Duet 2 Wi-Fi or Duet 3 Mini 5+ setups when using 3.6A2+3.

@DonStauffer said in RepRapFirmware 3.6.0-alpha.2 for Duet main boards available:

@ProteanReverie Sorry I'm not up to speed on the latest features, but is there some particular feature that has to be enabled or configured to get that improvement in the ringing, or is it just improvements to the RFF code?

The difference between my pictures is only an update to the RFF code, but the difference in the code pertains to how input shaping is handled. So, if you are using input shaping the update should improve things. If not, you would likely need to configure input shaping as well as applying the update. You can learn more about Input Shaping for Duet here https://docs.duet3d.com/en/User_manual/Tuning/Input_shaping

I have begun testing on two machines that are nearly identical, but one with a Duet 2 controlling it and one with a Duet 3 controlling it. I am very excited about the results I am seeing in print quality improvement. I have not yet encountered any noticeable issues with 3.6A2+3. Prints have run largely unattended, so if there was anything unusual that happened mid-print I could have missed it. Results off of the Duet 2 setup and Duet 3 setup look the same for the tests I have run.

Gcode was identical across both machines and both firmware versions. Input shaping is configured as ZVDD F15 on both machines. Pressure advance is enabled with a value of 0.04 on both machines.

These tests were done with a Hemera Revo 0.4HF nozzle and PETG material. I am also starting tests with a 1.0 nozzle but do not have that data yet.

---

Duet 2 setup:
Duet 2 Wi-Fi external antenna
Duex 5

Ghost text:
3.5.2 = Never used RAM 11712, free system stack 112 words
3.6A2+3 = Never used RAM 21704, free system stack 120 words

Benchy:
3.5.2 = Never used RAM 11544, free system stack 110 words
3.6A2+3 = Never used RAM 18512, free system stack 116 words

Accel test:
3.5.2 = Never used RAM 11568, free system stack 114 words
3.6A2+3 = Never used RAM 21656, free system stack 122 words

---

Duet 3 setup:
Duet 3 Mini 5+ WiFi main board in standalone mode
Duet 3 Mini 5+ CAN expansion board

Ghost text:
3.5.2 = Never used RAM 12068, free system stack 128 words
3.6A2+3 = Never used RAM 39644, free system stack 130 words

Benchy:
3.5.2 = Never used RAM 11900, free system stack 122 words
3.6A2+3 = Never used RAM 37052, free system stack 130 words

---

Here is an example of the difference in ringing/ghosting from the Ghost Text model. 3.5.2 is on bottom, 3.6A2+3 on top.

Please let me know if there is any other information I can provide that would be beneficial.

Another crash just now. This is on the board installed in a machine. Once again happened shortly after sending and receiving an M122 via DWC after being on overnight. Once again, the M122 prior to restart does not appear in the log file.

I also just noticed on this that the M122 response prior to restart ends part-way through the WiFi section. I double checked my previous posts where M122 resulted in a restart, and that appears to be the case on those as well. It appears when this happens it is just after the "failed messages" line and just before the wifi firmware version line.

M122 after restart:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (18 max)
Error in macro line 17 while starting up: Driver 1.0 does not exist
Driver 1.1 does not exist
Driver 1.3 does not exist
Driver 1.4 does not exist
Driver 1.2 does not exist
=== RTOS ===
Static ram: 103368
Dynamic ram: 122124 of which 0 recycled
Never used RAM 13068, free system stack 202 words
Tasks: NETWORK(1,ready,11.3%,217) HEAT(3,nWait 6,0.0%,364) Move(4,nWait 6,0.0%,355) CanReceiv(6,nWait 1,0.1%,798) CanSender(5,nWait 7,0.0%,336) CanClock(7,delaying,0.0%,350) TMC(4,nWait 6,0.8%,101) MAIN(1,running,85.7%,665) IDLE(0,ready,1.2%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:00:21 ago, cause: software
Last software reset at 2024-08-01 08:11, reason: StuckInSpinLoop, none spinning, available RAM 12068, slot 2
Software reset code 0x4093 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0000080f BFAR 0xe000ed38 SP 0x200043f0 Task NETW Freestk 4294953692 ok
Stack: 00000000 2000305c 10000000 e000e000 200043fd 000a1821 000a0efa 61010000 000a0ee3 20003058 ffffffff 00000000 03552189 20002878 2002c424 ffffffff 20011800 00000009 000a0f97 20004498 2002c420 200114f8 0009e1df 2002c420 0002dfab 2002c424 a5a5a5a5
Error status: 0x00
MCU revision 3, ADC conversions started 21418, completed 21418, timed out 0, errs 0
MCU temperature: min 38.4, current 39.1, max 39.2
Supply voltage: min 23.8, current 23.9, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/140/0, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 2, read errors 0, write errors 1, ifcnt 26, reads 1102, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 1: standstill, SG min 0, read errors 0, write errors 1, ifcnt 26, reads 1102, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 1101, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 2, read errors 0, write errors 1, ifcnt 24, reads 1101, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 1, ifcnt 21, reads 1102, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-08-01 08:12:11
Cache data hit count 41086803
Slowest loop: 5.66ms; fastest: 0.16ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 0.5ms, write time 2.6ms, max retries 0
=== Move ===
DMs created 83, segments created 0, maxWait 0ms, bed compensation in use: none, height map offset 0.000, max steps late 0, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 0.00
no step interrupt scheduled
Moves shaped first try 0, on retry 0, too short 0, wrong shape 0, maybepossible 0
=== DDARing 0 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 0 clear 206056
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 210, received 456, lost 0, errs 1, boc 0
Longest wait 1ms for reply type 6018, peak Tx sync delay 21, free buffers 26 (min 25), ts 107/106/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 8.77ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0
Firmware version 2.1.0
MAC address e8:68:e7:e1:4e:b7
Module reset reason: Power up, Vcc 3.38, flash size 2097152, free heap 42944
WiFi IP address 10.1.10.21
Signal strength -65dBm, channel 11, mode 802.11n, reconnections 0
Clock register 00002001
Socket states: 0 0 0 0 0 0 0 0

M122 prior to restart:

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW
Used output buffers: 1 of 40 (34 max)
=== RTOS ===
Static ram: 103368
Dynamic ram: 122500 of which 0 recycled
Never used RAM 12068, free system stack 128 words
Tasks: NETWORK(2,nWait 7,13.4%,203) HEAT(3,nWait 6,0.0%,325) Move(4,nWait 6,0.3%,241) CanReceiv(6,nWait 1,0.1%,773) CanSender(5,nWait 7,0.0%,327) CanClock(7,delaying,0.0%,348) TMC(4,delaying,0.9%,67) MAIN(1,running,84.5%,665) IDLE(0,ready,0.0%,29) AIN(4,delaying,0.9%,259), total 100.0%
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 15:31:50 ago, cause: power up
Last software reset at 2024-07-29 08:04, reason: StuckInSpinLoop, none spinning, available RAM 12396, slot 1
Software reset code 0x4093 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0044f80f BFAR 0xe000ed38 SP 0x200043f0 Task NETW Freestk 4294953692 ok
Stack: 00000000 2000305c 10000000 e000e000 200043fd 000a1821 000a0efa 61010000 000a0ee3 20003058 ffffffff 00000000 0deb5022 20002878 2002c424 ffffffff 20011800 00000009 000a0f97 20004498 2002c420 200114f8 0009e1df 2002c420 0002dfab 2002c424 a5a5a5a5
Error status: 0x00
MCU revision 3, ADC conversions started 55911215, completed 55911214, timed out 0, errs 0
MCU temperature: min 38.4, current 39.2, max 56.2
Supply voltage: min 23.7, current 23.9, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/72/72, gc cycles 0
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, read errors 0, write errors 0, ifcnt 15, reads 58948, writes 15, timeouts 1, DMA errors 0, CC errors 0, failedOp 0x01
Driver 1: standstill, SG min 0, read errors 0, write errors 0, ifcnt 15, reads 58949, writes 15, timeouts 0, DMA errors 0, CC errors 0
Driver 2: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 58953, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 3: standstill, SG min 0, read errors 0, write errors 0, ifcnt 13, reads 58950, writes 13, timeouts 0, DMA errors 0, CC errors 0
Driver 4: standstill, SG min 0, read errors 0, write errors 0, ifcnt 10, reads 58954, writes 10, timeouts 0, DMA errors 0, CC errors 0
Driver 5: not present
Driver 6: not present
Date/time: 2024-08-01 08:11:29
Cache data hit count 4294967295
Slowest loop: 111.27ms; fastest: 0.15ms
=== Storage ===
Free file entries: 19
SD card 0 detected, interface speed: 22.5MBytes/sec
SD card longest read time 10.2ms, write time 16.5ms, max retries 0
=== Move ===
DMs created 83, segments created 26, maxWait 71928ms, bed compensation in use: mesh, height map offset 0.068, max steps late 1, min interval 0, bad calcs 0, ebfmin 0.00, ebfmax 1.00
no step interrupt scheduled
Moves shaped first try 771, on retry 2134, too short 16058, wrong shape 44923, maybepossible 2196
=== DDARing 0 ===
Scheduled moves 69464, completed 69464, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 0 is on, I-accum = 0.4
Heater 1 is on, I-accum = 0.3
Heater 2 is on, I-accum = 0.0
=== GCodes ===
Movement locks held by null, null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 0, axes/extruders owned 0x0000803
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== Filament sensors ===
check 60938590 clear 466545122
Extruder 0 sensor: ok
Extruder 1 sensor: no filament
=== CAN ===
Messages queued 534665, received 1118265, lost 0, errs 0, boc 0
Longest wait 1ms for reply type 6018, peak Tx sync delay 337, free buffers 26 (min 25), ts 279554/279553/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 15.51ms; fastest: 0.00ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)
HTTP sessions: 1 of 8
=== WiFi ===
Interface state: active
Module is connected to access point 
Failed messages: pending 0, notrdy 0, noresp 0

Log file:

2024-07-31 18:09:14 [warn] HTTP client 10.1.10.161 login succeeded (session key 2966924725)
2024-07-31 18:37:06 [debug] Done printing file
2024-07-31 18:37:06 [debug] Done printing file
2024-07-31 18:37:06 [warn] Finished printing file 0:/gcodes/3.6A2_testing/PETG-0.4_Ghosting_test.gcode, print time was 1h 57m
2024-07-31 23:08:38 [warn] HTTP client 10.1.10.9 login succeeded (session key 1279128385)
2024-07-31 23:08:40 [warn] HTTP client 10.1.10.9 login succeeded (session key 270777459)
2024-07-31 23:09:09 [warn] HTTP client 10.1.10.9 login succeeded (session key 2746186916)
power up + 00:00:00 [info] Event logging started at level debug
power up + 00:00:00 [info] Running: Duet 3 Mini5plus WiFi: 3.5.2 (2024-06-11 17:14:16)
power up + 00:00:00 [debug] Done!
power up + 00:00:00 [debug] RepRapFirmware for Duet 3 Mini 5+ is up and running.
power up + 00:00:01 [warn] WiFi module started
power up + 00:00:07 [warn] WiFi module is connected to access point SD3D-24, IP address 10.1.10.21
power up + 00:00:08 [warn] HTTP client 10.1.10.9 login succeeded (session key 562008642)
2024-08-01 08:11:59 [warn] Date and time set at power up + 00:00:08
2024-08-01 08:12:11 [debug] === Diagnostics ===
2024-08-01 08:12:11 [debug] RepRapFirmware for Duet 3 Mini 5+ version 3.5.2 (2024-06-11 17:14:16) running on Duet 3 Mini5plus WiFi (standalone mode)
2024-08-01 08:12:11 [debug] Board ID: T69HG-KN6KL-K65J0-409NJ-JYW1Z-RZQJW

@Phaedrux said in Duet 3 mini5+ nightly restarts 3.5.2:

The opposite of a "quick" format. It zeros the entire drive area rather than just the file table. If there are bad cells they would get written to and potentially corrected or remapped.

Ok, that is what I had assumed, just never heard it with that term. Thank you for clarifying.

Just for clarity of data, when I formatted yesterday using windows format I did not have "quick format" selected.

I have completed some additional testing. Following an official SD tool formatting, both the main board and the bench board SD cards worked just fine and were able to pass 20 tests without issue. However, the expansion board card still has an issue even after the official SD tool formatting, and initially failed on the second test. I also tested that card in a Duet 2 Wi-Fi running 3.5.1, which passed 20 tests without issue. I then tested it in the bench board where it failed on the first test.

I have replaced the faulty card with the new 16GB card from yesterday and re-built those boards into my machine to resume my testing. The bench board is also set up to idle away to see if it randomly restarts again but could also be used for other testing if any more data would be of use.

@Phaedrux said in Duet 3 mini5+ nightly restarts 3.5.2:

Interesting. Are you able to do a surface format of one of the potentially bad cards and try the test again?

I am unfamiliar with the terminology "surface format", and a quick google isn't clearing that up. What I have done is used the official SD card formatting tool linked to here https://docs.duet3d.com/en/User_manual/RepRapFirmware/SD_card . I used the option of "Overwrite format" without "CHS format size adjustment" checked. This did merge everything into one partition thus resulting in one partition of ~8GB as opposed to how it arrived or how windows formatting left it, which had 2 partitions and a large chunk of unallocated space, with only a 2GB partition visible. This appears to have produced a promising result as it has survived 20 consecutive trials without error. I will proceed with adding files and testing, then trying with the other SD cards and boards and testing those, should no issues arise.