Reset during print Part 2 - Duet 3.1.1 / 3.2beta3 - Duet3+SBC

Via

I have now tried a fresh install of latest duetpi with same results.

From DCS log.

Nov 15 15:55:50 duet3 DuetControlServer[9198]: [warn] Lost connection to Duet (Timeout while waiting for transfer ready pin)
Nov 15 15:55:50 duet3 DuetControlServer[9198]: [warn] Bad header checksum (expected 0x9393, got 0x79bc)
Nov 15 15:55:51 duet3 DuetControlServer[9198]: [warn] Note: RepRapFirmware didn't receive valid data either (code 0x00000000)
Nov 15 15:55:51 duet3 DuetControlServer[9198]: [info] Connection to Duet established
Nov 15 15:55:51 duet3 DuetControlServer[9198]: [warn] Controller has been reset
Nov 15 15:55:51 duet3 DuetControlServer[9198]: [info] Starting macro file config.g on channel Trigger
Nov 15 15:55:51 duet3 DuetControlServer[9198]: [error] M550: Machine name must consist of the same letters and digits as configured by the Linux hostname
Nov 15 15:55:51 duet3 DuetControlServer[9198]: [info] Aborted job file
Nov 15 15:55:51 duet3 DuetControlServer[9198]: [warn] M307: Heater 0 appears to be over-powered. If left on at full power, its temperature is predicted to reach 365C
Nov 15 15:55:52 duet3 DuetControlServer[9198]: [info] Starting macro file config-override.g on channel Trigger
Nov 15 15:55:52 duet3 DuetControlServer[9198]: [info] Finished macro file config-override.g
Nov 15 15:55:52 duet3 DuetControlServer[9198]: [warn] M307: Heater 0 appears to be over-powered. If left on at full power, its temperature is predicted to reach 226C
Nov 15 15:55:52 duet3 DuetControlServer[9198]: [info] Finished macro file config.g

M122

=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.2-beta3 running on Duet 3 MB6HC v0.6 or 1.0 (SBC mode)
Board ID: 08DJM-956L2-G43S4-6J9FA-3S86T-1B5LD
Used output buffers: 3 of 40 (21 max)
=== RTOS ===
Static ram: 122124
Dynamic ram: 139224 of which 24 recycled
Never used RAM 130820, free system stack 200 words
Tasks: Linux(blocked,101) HEAT(blocked,296) CanReceiv(blocked,948) CanSender(blocked,371) CanClock(blocked,352) TMC(blocked,54) MAIN(running,1097) IDLE(ready,19)
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 00:06:45 ago, cause: watchdog
Last software reset at 2020-11-15 15:01, reason: User, Move spinning, available RAM 130780, slot 2
Software reset code 0x0004 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0xffffffff Task Linu
Error status: 0x00
MCU temperature: min 31.4, current 31.5, max 32.3
Supply voltage: min 13.0, current 13.0, max 13.1, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.0, max 12.1, under voltage events: 0
Driver 0: position 0, standstill, reads 63565, writes 0 timeouts 0, SG min/max not available
Driver 1: position 0, standstill, reads 63566, writes 0 timeouts 0, SG min/max not available
Driver 2: position 0, standstill, reads 63565, writes 0 timeouts 0, SG min/max not available
Driver 3: position 0, standstill, reads 63565, writes 0 timeouts 0, SG min/max not available
Driver 4: position 0, standstill, reads 63565, writes 0 timeouts 0, SG min/max not available
Driver 5: position 0, standstill, reads 63565, writes 0 timeouts 0, SG min/max not available
Date/time: 2020-11-15 16:02:36
Slowest loop: 145.54ms; fastest: 0.24ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 375, MinFreeDm: 375, MaxWait: 0ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages sent 1493, send timeouts 1493, longest wait 0ms for type 0, free CAN buffers 47
=== SBC interface ===
State: 0, failed transfers: 0
Last transfer: 22ms ago
RX/TX seq numbers: 54313/13286
SPI underruns 0, overruns 0
Number of disconnects: 0, IAP RAM available 0x20aa0
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.2.0-beta3
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 32.82

chrishamm

You do not have RRF 3.2.0-b3.2 installed. Please run M997 again or upload the MB6HC binary on the System page and confirm the update prompt there.

Via

@chrishamm apologies, didn't notice 3.2beta3.2 update until I posted this.

I have now updated to this and now for something different... Print got a lot further than previously (Layer 21) and then just froze, this time board did not reboot.

Full syslog at the time

Nov 15 17:42:57 duet3 DuetWebServer[523]: #033[40m#033[1m#033[33mwarn#033[39m#033[22m#033[49m: Microsoft.AspNetCore.Server.Kestrel[22]
Nov 15 17:42:59 duet3 DuetWebServer[523]:       Heartbeat took longer than "00:00:01" at "11/15/2020 17:42:00 +00:00". This could be caused by thread pool starvation.
Nov 15 17:43:53 duet3 DuetControlServer[434]: [info] System time has been changed
Nov 15 17:44:48 duet3 DuetWebServer[523]: #033[40m#033[1m#033[33mwarn#033[39m#033[22m#033[49m: Microsoft.AspNetCore.Server.Kestrel[22]
Nov 15 17:44:57 duet3 DuetWebServer[523]:       Heartbeat took longer than "00:00:01" at "11/15/2020 17:44:34 +00:00". This could be caused by thread pool starvation.
Nov 15 17:45:32 duet3 DuetWebServer[523]: #033[40m#033[1m#033[33mwarn#033[39m#033[22m#033[49m: Microsoft.AspNetCore.Server.Kestrel[22]
Nov 15 17:45:41 duet3 DuetWebServer[523]:       Heartbeat took longer than "00:00:01" at "11/15/2020 17:45:16 +00:00". This could be caused by thread pool starvation.

Screen attached to pi has frozen and web connection sits on "Connecting".

Left while writing this post (to save filament waste I am using a modified version now with temps set to 0 so it goes through the motions without actually printing) and after a while it suddenly restarted printing.

Looking at logs (extrusion warning because of zeroed temps) it did last print move at 17:42:55 and then two at 17:45:53 and 17:45:58 (did think I heard it move but when I went to check display was still frozen so thought I imagined it) and then started reprinting fully (and still continuing now 10 mins later) at 17:51:01

The DCS section has nothing (other than the restarting transfer shortly before) - tool not driven warnings removed from all logs for clarity.

Nov 15 17:41:42 duet3 DuetControlServer[434]: [warn] Restarting transfer because a bad data response was received (0x00000005)
Nov 15 17:41:51 duet3 DuetControlServer[434]: [warn] Restarting transfer because a bad header response was received (0x0003015f)
Nov 15 17:42:54 duet3 DuetControlServer[434]: [info] System time has been changed
Nov 15 17:43:53 duet3 DuetControlServer[434]: [info] System time has been changed
Nov 15 17:45:51 duet3 DuetControlServer[434]: [info] System time has been changed
Nov 15 17:45:52 duet3 DuetControlServer[434]: [info] System time has been changed
Nov 15 17:46:46 duet3 DuetControlServer[434]: [info] System time has been changed
Nov 15 17:48:03 duet3 DuetControlServer[434]: [info] System time has been changed
Nov 15 17:50:52 duet3 DuetControlServer[434]: [info] System time has been changed
Nov 15 17:50:52 duet3 DuetControlServer[434]: [info] System time has been changed

EDIT: M122 taken shortly after it restarted printing...

=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.2-beta3.2 running on Duet 3 MB6HC v0.6 or 1.0 (SBC mode)
Board ID: 08DJM-956L2-G43S4-6J9FA-3S86T-1B5LD
Used output buffers: 1 of 40 (12 max)
=== RTOS ===
Static ram: 122236
Dynamic ram: 139232 of which 64 recycled
Never used RAM 130660, free system stack 126 words
Tasks: Linux(ready,67) HEAT(blocked,296) CanReceiv(blocked,948) CanSender(blocked,352) CanClock(blocked,352) TMC(blocked,20) MAIN(running,1073) IDLE(ready,19)
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 01:44:39 ago, cause: software
Last software reset at 2020-11-15 16:13, reason: User, GCodes spinning, available RAM 130700, slot 0
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0xffffffff Task Linu
Error status: 0x00
MCU temperature: min 30.7, current 32.5, max 33.3
Supply voltage: min 12.9, current 13.0, max 13.1, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 11.9, current 12.0, max 12.1, under voltage events: 0
Driver 0: position 27914, ok, reads 37426, writes 19 timeouts 0, SG min/max 0/280
Driver 1: position 6439, ok, reads 37426, writes 19 timeouts 0, SG min/max 0/545
Driver 2: position 1480, standstill, reads 37438, writes 7 timeouts 0, SG min/max 0/317
Driver 3: position 0, standstill, reads 37439, writes 7 timeouts 0, SG min/max 0/332
Driver 4: position 0, standstill, reads 37427, writes 19 timeouts 0, SG min/max not available
Driver 5: position 0, standstill, reads 37446, writes 0 timeouts 0, SG min/max not available
Date/time: 2020-11-15 17:57:40
Slowest loop: 243.63ms; fastest: 0.09ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 372, MinFreeDm: 339, MaxWait: 249818ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 18341, completed moves 18327, StepErrors 0, LaErrors 0, Underruns [0, 0, 28], CDDA state 3
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 1
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File* is doing "G0 F7200 X82.716003 Y80.648003" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue* is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages sent 24971, send timeouts 24971, longest wait 0ms for type 0, free CAN buffers 47
=== SBC interface ===
State: 0, failed transfers: 2
Last transfer: 23ms ago
RX/TX seq numbers: 52632/53918
SPI underruns 1, overruns 1
Number of disconnects: 8, IAP RAM available 0x20a30
Buffer RX/TX: 1168/1704-0
=== Duet Control Server ===
Duet Control Server v3.2.0-beta3
File:
Buffered code: G0 X152.632 Y162.681
Buffered code: G0 X152.632 Y162.84
Buffered code: G1 F4800 X174.699 Y80.486 E2.1268
Buffered code: G0 F7200 X174.463 Y80.487
Buffered code: G1 F4800 X214.597 Y120.621 E1.41584
Buffered code: G0 F7200 X214.599 Y120.621
Buffered code: G1 F4800 X221.858 Y118.676 E0.18746
Buffered code: G0 F7200 X221.858 Y117.7
Buffered code: G1 F4800 X184.646 Y80.487 E1.31277
Buffered code: G1 X184.54 Y80.381
Buffered code: G0 F7200 X184.54 Y80.648
Buffered code: G0 X185.88 Y80.648
Buffered code: G0 X185.88 Y80.485
Buffered code: G1 F4800 X163.813 Y162.84 E2.12682
Buffered code: G1 X163.775 Y162.984
Buffered code: G0 F7200 X163.775 Y162.681
Buffered code: G0 X165.175 Y162.681
Buffered code: G0 X165.175 Y162.84
Buffered code: G1 F4800 X82.822 Y80.487 E2.90522
Buffered code: G1 X82.716 Y80.381
Buffered code: G0 F7200 X82.716 Y80.648
Buffered code: G0 X81.524 Y80.648
Buffered code: G0 X81.524 Y80.486
Buffered code: G1 F4800 X59.457 Y162.84 E2.1268
Buffered code: G0 F7200 X59.457 Y162.681
Buffered code: G0 X58.26 Y162.681
Buffered code: G0 X58.26 Y162.84
Buffered code: G1 F4800 X58.139 Y162.719 E0.00427
Buffered code: G0 F7200 X58.139 Y162.544
Buffered code: G1 F4800 X213.707 Y120.86 E4.01755
Buffered code: G1 X213.851 Y120.822
Buffered code: G0 F7200 X213.78 Y120.656
==> 1496 bytes
Code buffer space: 2392
Configured SPI speed: 8000000 Hz
Full transfers per second: 29.52
File /opt/dsf/sd/gcodes/[PLA] CFFFP_Top - zero temps.gcode is selected, processing

Via

Sorry for double posting but an update...

Print is still continuing (almost complete now judging by movements) however rpi gui is totally frozen still (mouse pointer not even moving).

Remote web panel working fine however print status shows as 100% complete on layer 21 (where it froze) status box at top shows busy and is updating correctly though with tool position (have also reloaded page).

dwc - frozen print.png

chrishamm

If DCS reports many "system time has changed" messages then its internal update task stalled longer than 5 seconds every time. I suspect the underlying problem may be related to the DWS error messages, too, but TBH I have never observed those messages yet.

I noticed bad SD card performance may cause lags and if yours is damaged, those kinds of problems could be expected. I suggest you rewrite the latest DuetPi version to your board, update everything, and then change once again to the unstable branch. Please use the official Raspberry Pi Imager this time and let it verify the written content. When you start this tool, you can select custom third-party images at the bottom of the distro selection and then select the corresponding DuetPi img file.

If the same problem persists, maybe leave htop running in a console in the background after a fresh start and check if a process is taking an excessive amount of CPU or RAM.

dhusolo

Your issues seem similar to mine. I was getting a few different errors

Warning: Tool 0 was not driven because its heater temperatures were not high enough or it has a heater fault

Connection interrupted, attempting to reconnect...
DCS has been stopped

Warning: Lost connection to Duet (Board is not available (no data response))

11/6/2020, 5:21:16 PM	Cancelled printing file 0:/gcodes/a1.gcode, print time was 0h 19m
11/6/2020, 5:21:15 PM	Warning: Controller has been reset

Removing the pi fixed the issue so I'm waiting until the communication issues are resolved before going back to the SPC
https://forum.duet3d.com/topic/19563/duet-3-sbc-dcs-has-stopped/

Via

@chrishamm that was with the fresh install on a new (Sandisk Ultra) memory card. But I have redone to double check as previously did use Rufus to write. I have used RPi imager this time and verified. All updates done and rebooted, M997 done to confirm all updates OK and shown running 3.2beta3.2.

And this time back to reset ~1hr into print.

Kept a casual eye on htop during print, chromium did sit fairly high on CPU usage averaging ~55% but spiking as high as 98% this is without any plugins installed (did have Visualizer on previous) and only window open on gui is standard DWC.

Nothing telling on syslog (added M302 P1 to get rid of tool not driven warnings)

Nov 16 11:21:35 duet3 systemd[1]: Started Session c5 of user pi.
Nov 16 11:54:33 duet3 DuetControlServer[8548]: [warn] Lost connection to Duet (Board is not available (no header))
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [info] Connection to Duet established
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [warn] Controller has been reset
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [info] Starting macro file config.g on channel Trigger
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [warn] File: Out-of-order reply: 'G0/G1: insufficient axes homed'
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [error] G0/G1: insufficient axes homed
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [warn] File: Out-of-order reply: 'G0/G1: insufficient axes homed'
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [error] G0/G1: insufficient axes homed
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [warn] File: Out-of-order reply: 'G0/G1: insufficient axes homed'
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [error] G0/G1: insufficient axes homed
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [warn] File: Out-of-order reply: 'G0/G1: insufficient axes homed'
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [error] G0/G1: insufficient axes homed
Nov 16 11:54:54 duet3 DuetControlServer[8548]: [info] Aborted job file
Nov 16 11:54:55 duet3 DuetControlServer[8548]: [warn] M307: Heater 0 appears to be over-powered. If left on at full power, its temperature is predicted to reach 365C
Nov 16 11:54:55 duet3 DuetControlServer[8548]: [info] Starting macro file config-override.g on channel Trigger
Nov 16 11:54:55 duet3 DuetControlServer[8548]: [info] Finished macro file config-override.g
Nov 16 11:54:55 duet3 DuetControlServer[8548]: [warn] M307: Heater 0 appears to be over-powered. If left on at full power, its temperature is predicted to reach 226C
Nov 16 11:54:55 duet3 DuetControlServer[8548]: [info] Event logging stopped
Nov 16 11:54:55 duet3 DuetControlServer[8548]: [info] Event logging to /opt/dsf/sd/sys/eventlog.txt started
Nov 16 11:54:55 duet3 DuetControlServer[8548]: [info] Finished macro file config.g

M122

=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.2-beta3.2 running on Duet 3 MB6HC v0.6 or 1.0 (SBC mode)
Board ID: 08DJM-956L2-G43S4-6J9FA-3S86T-1B5LD
Used output buffers: 1 of 40 (10 max)
=== RTOS ===
Static ram: 122236
Dynamic ram: 139232 of which 24 recycled
Never used RAM 130700, free system stack 200 words
Tasks: Linux(ready,81) HEAT(blocked,297) CanReceiv(blocked,948) CanSender(blocked,371) CanClock(blocked,354) TMC(blocked,54) MAIN(running,1217) IDLE(ready,19)
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 00:02:40 ago, cause: watchdog
Last software reset at 2020-11-15 23:19, reason: User, GCodes spinning, available RAM 130660, slot 2
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0xffffffff Task Linu
Error status: 0x00
MCU temperature: min 35.0, current 36.8, max 36.9
Supply voltage: min 13.0, current 13.0, max 13.1, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.0, max 12.1, under voltage events: 0
Driver 0: position 0, standstill, reads 890, writes 14 timeouts 0, SG min/max 0/0
Driver 1: position 0, standstill, reads 891, writes 14 timeouts 0, SG min/max 0/0
Driver 2: position 0, standstill, reads 891, writes 14 timeouts 0, SG min/max 0/0
Driver 3: position 0, standstill, reads 892, writes 14 timeouts 0, SG min/max 0/0
Driver 4: position 0, standstill, reads 893, writes 14 timeouts 0, SG min/max 0/0
Driver 5: position 0, standstill, reads 897, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2020-11-16 11:57:34
Slowest loop: 0.92ms; fastest: 0.10ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 375, MinFreeDm: 375, MaxWait: 0ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File* is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages sent 640, send timeouts 640, longest wait 0ms for type 0, free CAN buffers 47
=== SBC interface ===
State: 0, failed transfers: 0
Last transfer: 20ms ago
RX/TX seq numbers: 61520/5305
SPI underruns 0, overruns 0
Number of disconnects: 0, IAP RAM available 0x20a30
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.2.0-beta3
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 2.79

Via

@dhusolo Just had a look through your thread, looks like that will be the way to go for now which is a shame as whole design of this printer is around the SBC and display design. Did you have any issues going over to standalone? I did try to do that a few days back but having problems getting board to start in standalone mode, most likely not got SD card right as my board only had the old duetpi image (but will do seperate thread on that later if I still have issues).

I am just redoing print now with debug mode as advised in that thread to see if it gives any further insight.

chrishamm

@Via Can you please share your G-code file? The other thread is not related to this issue and that one is likely fixed in the latest beta. Your M122 excerpt reveals that the Duet had an unexpected watchdog reset and it would really help if we manage to reproduce this.

Via

@chrishamm original and zero temps version:
[PLA] CFFFP_Top - ReRender.gcode
[PLA] CFFFP_Top - ReRender - zero temps.gcode

Print using debug just stopped, attached is everything in scrollback buffer. Failure part below.

failed print debug log.txt

[debug] Waiting for finish of G1 F4800 X221.859 Y142.854 E0.51614
[debug] File: Sent G1 F4800 X215.386 Y162.84 E2.88069, remaining space 184, needed 56
[debug] File: Sent G0 F7200 X215.386 Y162.681, remaining space 136, needed 48
[debug] File: Sent G0 X216.503 Y162.681, remaining space 96, needed 40
[debug] File: Sent G0 X216.503 Y162.84, remaining space 56, needed 40
[debug] File: Sent G1 F4800 X221.859 Y142.854 E0.51614, remaining space 0, needed 56
[warn] Bad header checksum (expected 0x0000, got 0x1890)
[warn] Bad header checksum (expected 0x0000, got 0x1890)
[warn] Bad header checksum (expected 0x0000, got 0x1890)
[warn] Restarting transfer because the number of maximum retries has been exceeded
[debug] Cancelled G1 F4800 X221.859 Y142.854 E0.51614
[debug] Cancelled G0 X216.503 Y162.84
[debug] Cancelled G0 X216.503 Y162.681
[warn] Controller has been reset
[debug] Cancelled G0 F7200 X215.386 Y162.681
[debug] Cancelled G1 F4800 X215.386 Y162.84 E2.88069
[debug] Cancelled G0 X112.148 Y162.84
[debug] Cancelled G0 X112.148 Y162.681
[debug] Cancelled G0 F7200 X221.858 Y133.072
[debug] Cancelled G1 F4800 X141.964 Y162.84 E2.10885
[debug] Cancelled G1 F4800 X110.759 Y162.841 E2.86914
[debug] Cancelled G0 F7200 X215.504 Y127.32
[debug] Cancelled G1 X141.926 Y162.984
[debug] Cancelled G1 F4800 X214.831 Y127.353 E0.91645
[debug] Cancelled G0 F7200 X205.323 Y162.84
[debug] Cancelled G0 F7200 X141.926 Y162.681
[debug] Cancelled G1 F4800 X205.204 Y162.84 E2.88069
[debug] Cancelled G1 F4800 X123.417 Y81.182 E1.68581
[warn] File: Out-of-order reply: ''
[warn] File: Out-of-order reply: ''
[debug] Cancelled G0 X58.3 Y98.673
[debug] Cancelled G0 X144.11 Y162.84
[debug] Cancelled G0 F7200 X58.3 Y100.269
[debug] Cancelled G1 X58.101 Y100.269
[debug] Cancelled G1 F4800 X62.453 Y81.182 E2.88069
[debug] Cancelled G1 F4800 X58.139 Y100.125 E0.48921
[debug] Cancelled G0 X144.11 Y162.681
[debug] Cancelled G0 F7200 X63.215 Y81.182
[debug] Cancelled G0 F7200 X133.729 Y81.182
[debug] Cancelled G0 X58.139 Y98.673
[debug] Cancelled G1 F4800 X134.028 Y81.181 E2.10885
[debug] Cancelled G0 F7200 X110.615 Y162.681
[debug] Cancelled G1 X110.615 Y162.879
[debug] Cancelled G0 F7200 X123.547 Y81.182
[debug] Cancelled G1 F4800 X221.859 Y133.675 E0.22419
[info] Aborted job file
[info] Cancelled printing file 0:/gcodes/[PLA] CFFFP_Top - ReRender - zero temps.gcode, print time was 1h 18m
[debug] Requesting update of key job, seq 47 -> 48
[debug] Updated key job

dhusolo

@Via I flashed 3.1.1 to the board with Bossa and made a new SD card config. It didn't take too long

Via

I've put Duet3 into standalone now, retried print using 3.1.1 and print failed again. M122 shows Memory protection fault at time of failure.

=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.1.1 running on Duet 3 MB6HC v0.6 or 1.0 (standalone mode)
Board ID: 08DJM-956L2-G43S4-6J9FA-3S86T-1B5LD
Used output buffers: 1 of 40 (11 max)
=== RTOS ===
Static ram: 154604
Dynamic ram: 162852 of which 44 recycled
Exception stack ram used: 272
Never used ram: 75444
Tasks: NETWORK(ready,364) ETHERNET(blocked,436) HEAT(blocked,1200) CanReceiv(suspended,3820) CanSender(suspended,1488) CanClock(blocked,1452) TMC(blocked,204) MAIN(running,4472) IDLE(ready,76)
Owned mutexes:
=== Platform ===
Last reset 00:00:47 ago, cause: software
Last software reset at 2020-11-18 11:53, reason: Memory protection fault, spinning module GCodes, available RAM 75108 bytes (slot 0)
Software reset code 0x4163 HFSR 0x00000000 CFSR 0x00000001 ICSR 0x04427804 BFAR 0x00000000 SP 0x204175c4 Task MAIN
Stack: 0000000a 0000000a 40070000 20417668 204176b9 00000015 00000000 0000000d 0046207b 00000052 20417630 
Error status: 0
MCU temperature: min 37.2, current 37.4, max 38.0
Supply voltage: min 13.0, current 13.0, max 13.1, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.1, max 12.1, under voltage events: 0
Driver 0: standstill, reads 38262, writes 14 timeouts 0, SG min/max 0/0
Driver 1: standstill, reads 38263, writes 14 timeouts 0, SG min/max 0/0
Driver 2: standstill, reads 38263, writes 14 timeouts 0, SG min/max 0/0
Driver 3: standstill, reads 38264, writes 14 timeouts 0, SG min/max 0/0
Driver 4: standstill, reads 38264, writes 14 timeouts 0, SG min/max 0/0
Driver 5: standstill, reads 38268, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2020-11-18 11:54:07
Slowest loop: 6.14ms; fastest: 0.14ms
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 25.0MBytes/sec
SD card longest read time 0.8ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 375, MinFreeDm: 375, MaxWait: 0ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 9.70ms; fastest: 0.03ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- Ethernet -
State: active
Error counts: 0 0 0 0 0
Socket states: 5 2 2 2 2 0 0 0
=== CAN ===
Messages sent 190, longest wait 0ms for type 0
=== Linux interface ===
State: 0, failed transfers: 0
Last transfer: 47750ms ago
RX/TX seq numbers: 0/1
SPI underruns 0, overruns 0
Number of disconnects: 0
Buffer RX/TX: 0/0-0

chrishamm

I've got more details about this problem, it looks like lxpanel (a program for the desktop running in the background) has a memory leak and which, at some point, causes problems with DSF. I hope this will be fixed by Raspbian soon.

@Via Please try out 3.2.0-b3.2 in standalone mode, the memory protection fault should be fixed in that build.

Via

@chrishamm Thanks for the update.

Would using the non-gui version of duetpi and forgoing connected screen work around this issue for the short term keeping SBC control?

Have updated standalone to 3.2.0-b3.2 and running test again now.

chrishamm

@Via Yes, I think so. My printer has the GUI-less variant installed (DuetPi lite), so that probably explains why I couldn't observe this problem before.

Via

@chrishamm No luck again on standalone with 3.2.0-beta3.2, this time "AssertionFailed"

I will have another try later with duetpi lite.

=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.2-beta3.2 running on Duet 3 MB6HC v0.6 or 1.0 (standalone mode)
Board ID: 08DJM-956L2-G43S4-6J9FA-3S86T-1B5LD
Used output buffers: 1 of 40 (11 max)
=== RTOS ===
Static ram: 122236
Dynamic ram: 168580 of which 376 recycled
Never used RAM 101000, free system stack 180 words
Tasks: NETWORK(ready,165) ETHERNET(blocked,109) HEAT(blocked,293) CanReceiv(blocked,948) CanSender(blocked,371) CanClock(blocked,358) TMC(blocked,54) MAIN(running,1111) IDLE(ready,19)
Owned mutexes:
=== Platform ===
Last reset 00:00:28 ago, cause: software
Last software reset at 2020-11-19 11:11, reason: AssertionFailed, GCodes spinning, available RAM 101000, slot 0
Software reset code 0x4123 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00427000 BFAR 0x00000000 SP 0x2040fd54 Task MAIN
Stack: 00000599 004886b0 00468ce5 00000000 ffffffff 20427700 2040e2a0 20427ab0 ffffffff 00000000 56a3ba63 a5a5a5a5 00468d9b 2040fda4 00000000 20423aa8 004664d5 20427aa0 00440cc3 00000000 20427aa4 20408001 2040fdb4 00000101 00469700 0046972e 61000000
Error status: 0x00
MCU temperature: min 37.1, current 37.8, max 38.0
Supply voltage: min 13.0, current 13.0, max 13.1, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.1, max 12.1, under voltage events: 0
Driver 0: position 0, standstill, reads 9148, writes 14 timeouts 0, SG min/max 0/0
Driver 1: position 0, standstill, reads 9149, writes 14 timeouts 0, SG min/max 0/0
Driver 2: position 0, standstill, reads 9151, writes 14 timeouts 0, SG min/max 0/0
Driver 3: position 0, standstill, reads 9152, writes 14 timeouts 0, SG min/max 0/0
Driver 4: position 0, standstill, reads 9153, writes 14 timeouts 0, SG min/max 0/0
Driver 5: position 0, standstill, reads 9157, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2020-11-19 11:12:08
Slowest loop: 6.68ms; fastest: 0.21ms
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 25.0MBytes/sec
SD card longest read time 1.2ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 375, MinFreeDm: 375, MaxWait: 0ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 11.39ms; fastest: 0.03ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- Ethernet -
State: active
Error counts: 0 0 0 0 0
Socket states: 5 2 2 2 2 0 0 0
=== CAN ===
Messages sent 113, send timeouts 113, longest wait 0ms for type 0, free CAN buffers 47

dc42

@Via , I will look into this later today.

Via

@dc42 Many thanks.

Is it worth trying with DuetPi Lite? Was going to try earlier but work got in the way

dc42

@Via said in Reset during print Part 2 - Duet 3.1.1 / 3.2beta3 - Duet3+SBC:

@dc42 Many thanks.

Is it worth trying with DuetPi Lite? Was going to try earlier but work got in the way

I doubt it. The assertion failure looks like it was caused either by memory corruption, or possibly by a power brownout. I guess a hardware problem is also a possibility.

Can you set up a macro that runs the file in simulation mode within a loop, to see if you can provoke the failure that way?

Via

@dc42 I've ran the file as a normal simulation which completes fine.

Trying to do a macro to loop it but my gcode knowledge is terrible and keep freezing the board can you point me in right direction?

Was thinking this (I know never ending loops are bad in themselves but was just going to let it run until I reset it) but looks like it is just buffering all the M24s in one go how would I go about waiting for it to finish one before starting the next?

M37 S1
M23 "[PLA] CFFFP_Top - ReRender - zero temps.gcode"
while true
	M24