Duet3 Stuck in spin loop

[projectR3D]

We have been testing a new unit with Duet3 with SBC (Pi3b+), we have run into a couple of errors. The first would be a full system hang after a print cancel and then hitting reprint. Since a print has been canceled the temp is still high but it will never start printing. It will sit at 0.0% and nothing will respond, if you hit a button in the web interface it will just spin. The second more common (only about 5% of the time) issue is a disconnect and restart of the system upon an "Upload and Print" process. So far its mostly been with S3D files.

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.1.1 running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956L2-G43S8-6J1FJ-3S46K-9S26D
Used output buffers: 1 of 40 (11 max)
=== RTOS ===
Static ram: 154604
Dynamic ram: 162900 of which 64 recycled
Exception stack ram used: 528
Never used ram: 75120
Tasks: NETWORK(ready,1980) HEAT(blocked,1188) CanReceiv(suspended,3820) CanSender(suspended,1432) CanClock(blocked,1436) TMC(blocked,204) MAIN(running,5016) IDLE(ready,76)
Owned mutexes:
=== Platform ===
Last reset 00:05:11 ago, cause: software
Last software reset at 2020-09-15 02:15, reason: Stuck in spin loop, spinning module GCodes, available RAM 74976 bytes (slot 1)
Software reset code 0x4083 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0444a80f BFAR 0x00000000 SP 0x20417b14 Task MAIN
Stack: 00000034 00427e86 010f0000 00000000 40a25c00 42595364 bf8f81ee 42c80001 3d9096e1 00000000 3e1096e1 
Error status: 0
MCU temperature: min 52.8, current 52.8, max 54.4
Supply voltage: min 24.2, current 24.2, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.3, current 12.4, max 12.4, under voltage events: 0
Driver 0: standstill, reads 10631, writes 17 timeouts 0, SG min/max 0/158
Driver 1: standstill, reads 10632, writes 17 timeouts 0, SG min/max 0/184
Driver 2: standstill, reads 10633, writes 17 timeouts 0, SG min/max 0/145
Driver 3: standstill, reads 10633, writes 17 timeouts 0, SG min/max 0/146
Driver 4: standstill, reads 10634, writes 17 timeouts 0, SG min/max 0/141
Driver 5: standstill, reads 10637, writes 14 timeouts 0, SG min/max 0/0
Date/time: 2020-09-15 02:20:50
Slowest loop: 5.38ms; fastest: 0.15ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 375, MinFreeDm: 373, MaxWait: 270570ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 13, completed moves: 13, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is ready with "M122" in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon* is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 1.10ms; fastest: 0.01ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions
HTTP sessions: 0 of 8
- Ethernet -
State: disabled
Error counts: 0 0 0 0 0
Socket states: 0 0 0 0 0 0 0 0
=== CAN ===
Messages sent 1247, longest wait 0ms for type 0
=== Linux interface ===
State: 0, failed transfers: 0
Last transfer: 17ms ago
RX/TX seq numbers: 44715/10150
SPI underruns 0, overruns 0
Number of disconnects: 0
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.1.1
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 32.14
 

G21						;Work in millimetres
G90						;Send absolute coordinates...
M83						;...but relative extruder moves
 
; Axis and motor configuration
M569 P0 S1					;Drive 0 X / Rear
M569 P1 S1					;Drive 1 Y / Front
M569 P2 S1					;Drive 2 Z Front Left
M569 P3 S1					;Drive 3 Z Rear Left
M569 P4 S1					;Drive 4 Z Right
M569 P5 S0					;Drive 5 Extruder
M667 S1						;corexy mode
 
M584 X0 Y1 Z2:3:4 E5			; Map X to drive 0 Y to drive 1, Z to drives 2, 3, 4, and E to drive 5
;Leadscrew locations
M671 X-38:-38:340  Y62.5:247.5:139 S7.5		;X -38, -38, 340
 
M350 X16 Y16 Z16 E16 I1				;set 16x microstepping for axes with interpolation
M906 X1400 Y1400 Z1200 E1500 I60			;Set motor currents (mA)
M201 X2000 Y2000 Z100 E1500			;Accelerations (mm/s^2)
M203 X24000 Y24000 Z900 E3600			;Maximum speeds (mm/min) was Z1800 tr8*4
M566 X1000 Y1000 Z100 E1500			;Maximum jerk speeds mm/minute E60 Z60
M208 X310 Y300 Z435				;set axis maxima and high homing switch positions (adjust to suit your machine)
M208 X0 Y0 Z-2 S1				;set axis minima and low homing switch positions (adjust to make X=0 and Y=0 the edges of the bed)
;M92 X200 Y200 Z1600 E819.5			;steps/mm
M92 X200 Y200 Z800 E830			;steps/mm 0.9 z
 
 
; End Stops
M574 X1 S1! P"io3.in"				;Map the X endstop to io3.in
M574 Y2 S1! P"io1.in"				;May the Y endstop to io1.in
 
; Thermistors
M308 S0 P"temp0" Y"thermistor" A"bed_heat" T100000 B4240 H0 L0 		;Bed thermistor - duet 3, connected to temp0
 
M308 S1 P"temp1" Y"thermistor" A"e0_heat" T100000 B4725 C7.06e-8 H0 L0	;duet3 e3d
;M308 S1 P"temp1" Y"thermistor" A"e0_heat" T500000 B4723 C1.196220e-7	;duet3 slice
 
;Define Heaters
M950 H0 C"out0" T0				;Bed heater is on out0
M950 H1 C"out1" T1				;Hotend heater is on out1
 
M307 H0 A90.0 C700.0 D10.0 S1.00 B1 		;Keenovo duet 3 configuration
M307 H1 A548.8 C298.6 D4.6 S1.00 B0 		;Hotend  duet 3 configuration
 
M570 S360					;Hot end may be a little slow to heat up so allow it 180 seconds
M143 S285					;Set max hotend temperature			
 
; Fans
M950 F0 C"out4"					;Hotend fan on "out4" connector
M106 P0 S255 H1 T50 				;enable thermostatic mode for hotend fan
M950 F1 C"out5"					;Layer fan on "out5" connector
M106 P1 S0 					;Layer Fan
M950 F2 C"out6"					;Layer fan on "out6" connector
M106 P2 S0 					;Layer Fan
 
; Tool definitions
M563 P0 D0 H1 F1:2                        	;Define tool 0
G10 P0 S0 R0                        		;Set tool 0 operating and standby temperatures
 
;BLTouch
;M558 P9 C"io7.in" H5 R1 F50 T6000 A5 S0.02 B1	;define the bltouch input on io8.in
;M950 S0 C"!io7.out"				;define the bltouch servo on io8.out"
M558 P5 C"!io7.in" I0 H10 F180 T3600 ; set Z probe type to bltouch and the dive height + speeds
G31 X-10 Y66 Z1.9 P25
 
 
 
M557 X66:250 Y66:250 S184
 
M501
M140 H0
T0						;select first hot end 

Any help would be greatly appreciated, if you have any questions please let me know

[Phaedrux]

What do you have in your pause.g and cancel.g files?

Can you switch to standalone mode to test if it behaves the same way?

[projectR3D]

Pause.g

; pause.g
; called when a print from SD card is paused
;
; generated by RepRapFirmware Configuration Tool v2.1.8 on Sun Feb 02 2020 20:26:36 GMT-0600 (Central Standard Time)
M83            ; relative extruder moves
G1 E-10 F3600  ; retract 10mm of filament
G91            ; relative positioning
G1 Z5 F360     ; lift Z by 5mm
G90            ; absolute positioning
G1 X0 Y0 F6000 ; go to X=0 Y=0

Stop.g

; stop.g
; called when M0 (Stop) is run (e.g. when a print from SD card is cancelled)
;
; generated by RepRapFirmware Configuration Tool v2.1.8 on Sun Feb 02 2020 20:26:37 GMT-0600 (Central Standard Time)

[Phaedrux]

Try creating a cancel.g with some sort of commands in it. Turn off heaters and rehome x and y or something like that.

[projectR3D]

Ill give that a try, it only happens with S3D files and only on the duet3. The most common error is the upload and print then getting stuck in the loop, do you have any experience with that?

[Phaedrux]

I don't use S3D and haven;t heard of anything like that.

How are you uploading the files from S3D? Curl script I assume? or from DWC?

[projectR3D]

Upload with DWC. It just happened again

m122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.1.1 running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956L2-G43S8-6J1FJ-3S46K-9S26D
Used output buffers: 1 of 40 (11 max)
=== RTOS ===
Static ram: 154604
Dynamic ram: 162900 of which 64 recycled
Exception stack ram used: 516
Never used ram: 75132
Tasks: NETWORK(ready,1972) HEAT(blocked,1188) CanReceiv(suspended,3820) CanSender(suspended,1420) CanClock(blocked,1436) TMC(blocked,68) MAIN(running,4928) IDLE(ready,76)
Owned mutexes:
=== Platform ===
Last reset 00:11:33 ago, cause: software
Last software reset at 2020-09-15 05:17, reason: Memory protection fault, spinning module Move, available RAM 75040 bytes (slot 0)
Software reset code 0x4164 HFSR 0x00000000 CFSR 0x00000082 ICSR 0x0444a804 BFAR 0xe4aa804a SP 0x20417ba4 Task MAIN
Stack: 004256fd 00423352 010f0000 00000000 ffffffff 00000000 00000000 00000000 00000000 2042d338 00463e7b 
Error status: 0
MCU temperature: min 51.5, current 51.8, max 52.9
Supply voltage: min 23.8, current 24.2, max 24.5, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.3, current 12.3, max 12.4, under voltage events: 0
Driver 0: standstill, reads 41247, writes 19 timeouts 0, SG min/max 0/181
Driver 1: standstill, reads 41248, writes 19 timeouts 0, SG min/max 0/181
Driver 2: standstill, reads 41248, writes 19 timeouts 0, SG min/max 0/145
Driver 3: standstill, reads 41249, writes 19 timeouts 0, SG min/max 0/147
Driver 4: standstill, reads 41249, writes 19 timeouts 0, SG min/max 0/140
Driver 5: standstill, reads 41255, writes 14 timeouts 0, SG min/max 0/0
Date/time: 2020-09-15 05:28:41
Slowest loop: 5.52ms; fastest: 0.15ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 375, MinFreeDm: 373, MaxWait: 620834ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 13, completed moves: 13, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is ready with "M122" in state(s) 0
Telnet is idle in state(s) 0
File* is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon* is idle in state(s) 0 0, running macro
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 0.98ms; fastest: 0.01ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions
HTTP sessions: 0 of 8
- Ethernet -
State: disabled
Error counts: 0 0 0 0 0
Socket states: 0 0 0 0 0 0 0 0
=== CAN ===
Messages sent 2778, longest wait 0ms for type 0
=== Linux interface ===
State: 0, failed transfers: 0
Last transfer: 20ms ago
RX/TX seq numbers: 36345/22606
SPI underruns 0, overruns 0
Number of disconnects: 0
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.1.1
Daemon:
Finishing macro daemon.g, started by system
> Next stack level
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 32.53
 

[Phaedrux]

@projectR3D said in Duet3 Stuck in spin loop:

reason: Memory protection fault, spinning module Move,

Thanks.

You may also be able to get more information by enabling debugging.
https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Monitoring_optional

[projectR3D]

@Phaedrux Debugging has now been setup, would I see the additional info in M122 response?

[projectR3D]

Response from debug

pi@duet3:~ $ sudo /opt/dsf/bin/DuetControlServer -1 debug
Duet Control Server v3.1.1
Written by Christian Hammacher for Duet3D
Licensed under the terms of the GNU Public License Version 3
 
[info] Settings loaded
[info] Environment initialized
[info] Connection to Duet established
[info] IPC socket created at /var/run/dsf/dcs.sock
 

Response from sudo /opt/dsf/bin/DuetWebServer

pi@duet3:~ $ sudo /opt/dsf/bin/DuetWebServer
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://[::]:80
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /home/pi
info: DuetWebServer.Services.ModelObserver[0]
      Connections to DuetControlServer established
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 GET http://192.168.1.123/machine
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
      Executing endpoint 'DuetWebServer.Controllers.WebSocketController.Get (DuetWebServer)'
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
      Route matched with {action = "Get", controller = "WebSocket"}. Executing controller action with signature System.Threading.Tasks.Task Get() on controller DuetWebServer.Controllers.WebSocketController (DuetWebServer).
info: DuetWebServer.Controllers.WebSocketController[0]
      WebSocket connected from ::ffff:192.168.1.114:59415
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 GET http://192.168.1.123/machine/directory/0:%2Fmacros application/json
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
      Executing endpoint 'DuetWebServer.Controllers.MachineController.GetFileList (DuetWebServer)'
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
      Route matched with {action = "GetFileList", controller = "Machine"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] GetFileList(System.String) on controller DuetWebServer.Controllers.MachineController (DuetWebServer).
info: Microsoft.AspNetCore.Mvc.Infrastructure.ContentResultExecutor[1]
      Executing ContentResult with HTTP Response ContentType of application/json
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
      Executed action DuetWebServer.Controllers.MachineController.GetFileList (DuetWebServer) in 195.8381ms
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
      Executed endpoint 'DuetWebServer.Controllers.MachineController.GetFileList (DuetWebServer)'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished in 300.9561ms 200 application/json