Duet 2 Wifi rebooting/crashing during print

bwnance

@Phaedrux
Another crash from this morning (berd air still connected)

The gcode file is too large to attach here -- is there a preferred hosting site for me to upload the file?

M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later
Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP
Used output buffers: 3 of 24 (11 max)
=== RTOS ===
Static ram: 27980
Dynamic ram: 93236 of which 60 recycled
Exception stack ram used: 480
Never used ram: 9316
Tasks: NETWORK(ready,368) HEAT(blocked,1224) MAIN(running,1848) IDLE(ready,80)
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:12:58 ago, cause: software
Last software reset at 2020-11-05 11:55, reason: Hard fault, spinning module PrintMonitor, available RAM 8996 bytes (slot 1)
Software reset code 0x4069 HFSR 0x40000000 CFSR 0x00020000 ICSR 0x0441f803 BFAR 0xe000ed38 SP 0x20005414 Task MAIN
Stack: 0042eaf1 20003d50 60030000 00000001 00000009 00000014 0044eaab 0044eaae 21030000 3f000000 41492f1a 
Error status: 0
MCU temperature: min 24.1, current 28.4, max 28.7
Supply voltage: min 23.9, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: standstill, SG min/max 0/158
Driver 1: standstill, SG min/max 5/150
Driver 2: standstill, SG min/max 0/166
Driver 3: standstill, SG min/max not available
Driver 4: standstill, SG min/max not available
Date/time: 2020-11-05 12:08:02
Cache data hit count 1269324432
Slowest loop: 5.60ms; fastest: 0.14ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 1.2ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 169, MinFreeDm: 163, MaxWait: 19944ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 5, completed moves: 5, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 15.91ms; fastest: 0.00ms
Responder states: HTTP(2) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.23
WiFi MAC address dc:4f:22:6e:6c:ca
WiFi Vcc 3.43, reset reason Unknown
WiFi flash size 4194304, free heap 21840
WiFi IP address 192.168.86.26
WiFi signal strength -53dBm, reconnections 0, sleep mode modem
Socket states: 4 0 0 0 0 0 0 0

Phaedrux

@bwnance said in Duet 2 Wifi rebooting/crashing during print:

is there a preferred hosting site for me to upload the file?

You may be able to zip it and change the file extension to .txt and upload it here if it's within 4Mb (i think) otherwise dropbox or google drive.

bwnance

@Phaedrux
Good call! Attached is the zipped gcode file, renamed to .txt:

vibrationDampener2.gcode.txt

Phaedrux

What's in the macro here:

M98 P"0:/macros/start.g"

bwnance

@Phaedrux
I've got a little bit of a macro tree:
start.g:

G28
M98 P"0:/macros/brushclean.g"
M98 P"0:/macros/bottomhome.g"
G29 S1 ; load height map

brushclean.g:

G0 Z25 F12000
M564 S0 ; disable limits
G0 X0 Y0 Z12 F3000 ; middle of bed, z=5
G0 X0 Y-130 ; middle of brush
G0 Z8 ; drop nozzle slightly
G0 X15 ; 
G0 Y-128 X-15
G0 Y-125 X15
G0 Y-125 X0
G0 Z12
G0 X0 Y0
M564 S1 ; enable limits again

bottomhome.g:

M106 P0 S0
G0 Z25 F12000
G30 
G0 Z15 F12000

bwnance

@Phaedrux @dc42 Hey y'all, just bumping this again -- here's another crash report from today, along with the offending gcode:
tree_new.txt

=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later
Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP
Used output buffers: 3 of 24 (11 max)
=== RTOS ===
Static ram: 27980
Dynamic ram: 93236 of which 60 recycled
Exception stack ram used: 304
Never used ram: 9492
Tasks: NETWORK(ready,368) HEAT(blocked,1224) MAIN(running,1880) IDLE(ready,80)
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:29:46 ago, cause: software
Last software reset at 2020-12-14 23:54, reason: Hard fault, spinning module GCodes, available RAM 8940 bytes (slot 2)
Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x04427003 BFAR 0xa5a5a5a5 SP 0x2001ffbc Task MAIN
Stack: 0040f451 0040f452 0100001f 200052f0 200052ef 20008c2c 20003d50 a5a5a5a5 ffffffed 00000000 00f00000 
Error status: 0
MCU temperature: min 23.4, current 28.8, max 29.5
Supply voltage: min 23.9, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: standstill, SG min/max not available
Driver 1: standstill, SG min/max not available
Driver 2: standstill, SG min/max not available
Driver 3: standstill, SG min/max not available
Driver 4: standstill, SG min/max not available
Date/time: 2020-12-15 00:24:45
Cache data hit count 2932587391
Slowest loop: 3.44ms; fastest: 0.14ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 0.9ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 169, MinFreeDm: 169, MaxWait: 0ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 15.73ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.23
WiFi MAC address dc:4f:22:6e:6c:ca
WiFi Vcc 3.43, reset reason Unknown
WiFi flash size 4194304, free heap 21056
WiFi IP address 192.168.86.34
WiFi signal strength -57dBm, reconnections 0, sleep mode modem
Socket states: 0 0 0 0 0 0 0 0

bwnance

and another one just now, the more the merrier, i guess

=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later
Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP
Used output buffers: 3 of 24 (11 max)
=== RTOS ===
Static ram: 27980
Dynamic ram: 93236 of which 60 recycled
Exception stack ram used: 472
Never used ram: 9324
Tasks: NETWORK(ready,408) HEAT(blocked,1224) MAIN(running,1840) IDLE(ready,80)
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:00:37 ago, cause: software
Last software reset at 2020-12-15 02:33, reason: Hard fault, spinning module none, available RAM 8892 bytes (slot 3)
Software reset code 0x4071 HFSR 0x40000000 CFSR 0x00008200 ICSR 0x0441f003 BFAR 0x91200200 SP 0x2001ff8c Task NETW
Stack: 00454e15 00454e20 6100000e 40090000 0041d011 00000000 20002860 10000000 2000b98c 00000000 200034f0 
Error status: 0
MCU temperature: min 24.2, current 24.8, max 25.0
Supply voltage: min 23.9, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: ok, SG min/max not available
Driver 1: ok, SG min/max not available
Driver 2: standstill, SG min/max not available
Driver 3: standstill, SG min/max not available
Driver 4: standstill, SG min/max not available
Date/time: 2020-12-15 02:34:35
Cache data hit count 58687792
Slowest loop: 3.66ms; fastest: 0.14ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 8
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 1.0ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 166, MinFreeDm: 166, MaxWait: 18624ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 1, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: 3
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by HTTP
HTTP is idle in state(s) 1 0 8, running macro
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 15.75ms; fastest: 0.00ms
Responder states: HTTP(2) HTTP(4) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.23
WiFi MAC address dc:4f:22:6e:6c:ca
WiFi Vcc 3.43, reset reason Unknown
WiFi flash size 4194304, free heap 21408
WiFi IP address 192.168.86.34
WiFi signal strength -55dBm, reconnections 0, sleep mode modem
Socket states: 4 2 0 0 0 0 0 0

markz

Since no one has jumped in here, my usual opinion when I see something like this is that (assuming the board is ok) it's either a power supply issue or a voltage noise issue.

How much power does your heated bed use? Is it enough to impact the power supply?

How is the 24V to the pump connected? Could noise on the 24V line be caused by the pump pwm?

bwnance

@markz
Resistance on my heated bed (measured at connector on board) is 3.3ohms, it's a 24v bed, so that would be a ~175watt bed, correct? I'm using a Mean Well 24v 350w PSU and my hotend is 50w, so i should have plenty of extra room, wattage wise, right?

24v to the pump is over a fan header on the board (AFAIK newer duets have the flyback diode on all fan inputs internally), I'm not sure how to diagnose if i have noise on the 24v line.

markz

@bwnance thanks.

The pump has two inputs (kind of) power and control. I know the control line goes to the fan pins but where do you pull off the 24V supply for the pump - is that from the power supply or the duet board somewhere? Are you getting the 24v from the fan header?

bwnance

@markz I'm not using "EZ-Switch" board that was packaged with the berd-air, so both lines are hooked up to the fan header -- as best as i could tell before installing, this was fine, since the fan headers have flybacks wired into them

Phaedrux

Can you try 3.2 Beta 4? It's believed that a fix for this crash may already be present there.

https://github.com/Duet3D/RepRapFirmware/releases

bwnance

@Phaedrux just upgraded, will do a few test prints! We shall see...

Phaedrux

@bwnance Just realized there is an odd ordering problem on that page that doesn't list the most recent at the top.

https://github.com/Duet3D/RepRapFirmware/releases/tag/3.2beta4.1

Beta 4.1 can be found here.

bwnance

@Phaedrux ah, ty! gonna let this print finish on beta 4, then will update to 4.1

bwnance

@Phaedrux was out of town for the holidays, just started a 4 hour print and the printer rebooted ~40mins in

m122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.2-beta4 running on Duet WiFi 1.02 or later
Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP
Used output buffers: 3 of 24 (12 max)
=== RTOS ===
Static ram: 24108
Dynamic ram: 99836 of which 60 recycled
Never used RAM 6044, free system stack 106 words
Tasks: NETWORK(ready,183) HEAT(blocked,308) MAIN(running,451) IDLE(ready,20)
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:01:49 ago, cause: software
Last software reset at 2021-01-07 01:30, reason: AssertionFailed, GCodes spinning, available RAM 5836, slot 1
Software reset code 0x4123 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0041f000 BFAR 0xe000ed38 SP 0x200044ac Task MAIN
Stack: 00000c55 00472af0 00452a9d 2000ae20 00000000 200094c8 00000000 00000000 00451edb 00000000 ffffffff 2000ae20 200034cc 200094c8 ffffffff 00000000 20004524 a5a5a5a5 00451f1b 20004514 200094b8 20008380 0044f543 20008380 0042ee47 2001e448 200094bc
Error status: 0x00
MCU temperature: min 25.0, current 25.2, max 26.1
Supply voltage: min 23.9, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: position 63150, standstill, SG min/max 0/138
Driver 1: position 63150, standstill, SG min/max 0/133
Driver 2: position 63150, standstill, SG min/max 0/156
Driver 3: position 0, standstill, SG min/max not available
Driver 4: position 0, standstill, SG min/max not available
Driver 5: position 0
Driver 6: position 0
Driver 7: position 0
Driver 8: position 0
Driver 9: position 0
Driver 10: position 0
Driver 11: position 0
Date/time: 2021-01-07 01:32:29
Cache data hit count 170332161
Slowest loop: 3.51ms; fastest: 0.17ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 1.1ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 169, MinFreeDm: 163, MaxWait: 22002ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 5, completed moves 5, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 15.70ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.25beta0
WiFi MAC address dc:4f:22:6e:6c:ca
WiFi Vcc 3.43, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 19552
WiFi IP address 192.168.86.34
WiFi signal strength -57dBm, mode 802.11n, reconnections 0, sleep mode modem
Clock register 00002002
Socket states: 0 0 0 0 0 0 0 0

I see that 4.2 was just released so i'll update to that and try again.

bwnance

@Phaedrux no dice, another crash. @dc42 any ideas?

M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.2 running on Duet WiFi 1.02 or later
Board ID: 08DJM-9178L-L2MSD-6J1FA-3S46N-KSFYP
Used output buffers: 3 of 24 (12 max)
=== RTOS ===
Static ram: 23460
Dynamic ram: 72744 of which 60 recycled
Never used RAM 15816, free system stack 99 words
Tasks: NETWORK(ready,157) HEAT(blocked,307) MAIN(running,450) IDLE(ready,19)
Owned mutexes:
=== Platform ===
Last reset 00:01:12 ago, cause: software
Last software reset at 2021-01-07 19:24, reason: HardFault invState, GCodes spinning, available RAM 15608, slot 1
Software reset code 0x4063 HFSR 0x40000000 CFSR 0x00020000 ICSR 0x0041f003 BFAR 0xe000ed38 SP 0x2001fe90 Task MAIN Freestk 29308 ok
Stack: 00000000 00000001 400e1400 00001000 3ed0bedf 0044adf3 2001ff28 400f0227 00000160 00000100 2001db10 fffffff1 000001ab 00000000 00a69e38 0000012b 2001c8b8 38000000 005a0073 00000001 28bb5437 fffffffb 29767437 2000770e 2001ff28 01000001 2001ff28
Error status: 0x00
Aux0 errors 0,0,0
MCU temperature: min 24.9, current 25.4, max 26.0
Supply voltage: min 24.0, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: position 63150, standstill, SG min/max 0/147
Driver 1: position 63150, standstill, SG min/max 0/131
Driver 2: position 63150, standstill, SG min/max 0/157
Driver 3: position 0, standstill, SG min/max not available
Driver 4: position 0, standstill, SG min/max not available
Driver 5: position 0
Driver 6: position 0
Driver 7: position 0
Driver 8: position 0
Driver 9: position 0
Driver 10: position 0
Driver 11: position 0
Date/time: 2021-01-07 19:25:21
Cache data hit count 87576278
Slowest loop: 4.81ms; fastest: 0.20ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 0.9ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, maxWait 11729ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 5, completed moves 5, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 15.95ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.25
WiFi MAC address dc:4f:22:6e:6c:ca
WiFi Vcc 3.43, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 19568
WiFi IP address 192.168.86.34
WiFi signal strength -56dBm, mode 802.11n, reconnections 0, sleep mode modem
Clock register 00002002
Socket states: 0 0 0 0 0 0 0 0

Veti

can you post a picture of your wiring on the duet?

dc42

@bwnance, your M122 software reset data doesn't make any sense, so I think it must have been caused by a transient or possible power brownout. Can you run a few more prints to see if any future crashes are similar? You can do air prints to avoid wasting filament.

The assertion failure you had with 3.2beta4 is different, that's a known bug that was fixed in 3.2beta4.1.

bwnance

@dc42 Will do dry-runs throughout today. It's weird, sometimes it completes a print just fine, sometimes it fails -- but all with the same file.

@Veti