Duet rebooting mid print

[vrghost]

Hello,

I have hit an odd issue of duet rebooting while printing. This is first three-material print I am attempting on the e3d Toolchanger and all three print attempts I've made so far have failed at the approximately same spot. So I am wondering if the controller is overheating. I believe that I am using the latest and greatest 3.3 firmware build.

Here is the M112 output:

10/09/2021, 16:14:52    M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.3 (2021-06-15 21:44:54) running on Duet WiFi 1.02 or later + DueX5
Board ID: 0JD0M-9P6B2-NJ4S8-6JKD4-3S86K-9A76L
Used output buffers: 3 of 24 (19 max)
=== RTOS ===
Static ram: 23876
Dynamic ram: 78152 of which 12 recycled
Never used RAM 9520, free system stack 184 words
Tasks: NETWORK(ready,14.7%,231) HEAT(delaying,0.0%,330) Move(notifyWait,0.1%,319) DUEX(notifyWait,0.0%,24) MAIN(running,85.1%,441) IDLE(ready,0.1%,29), total 100.0%
Owned mutexes: WiFi(NETWORK) HTTP(MAIN)
=== Platform ===
Last reset 00:12:54 ago, cause: software
Last software reset at 2021-09-10 16:01, reason: OutOfMemory, GCodes spinning, available RAM 660, slot 1
Software reset code 0x41c3 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0041f000 BFAR 0xe000ed38 SP 0x200024f0 Task MAIN Freestk 877 ok
Stack: 2001998c 0040f871 00000063 0040fa4f 00000008 0000031c 2001a70a 0040633f 20019744 00000005 20019744 00406455 20019744 2000259c 20019744 00000005 20002594 004064a3 00000000 2000259c 20019e54 20001488 20002618 20019744 00000000 004064c7 20019744
Error status: 0x00
Aux0 errors 0,0,0
Step timer max interval 0
MCU temperature: min 38.7, current 39.1, max 45.1
Supply voltage: min 24.1, current 24.3, max 24.4, under voltage events: 0, over voltage events: 0, power good: yes
Heap OK, handles allocated/used 99/4, heap memory allocated/used/recyclable 2048/94/0, gc cycles 0
Driver 0: position 0, standstill, SG min/max not available
Driver 1: position 0, standstill, SG min/max not available
Driver 2: position 0, standstill, SG min/max not available
Driver 3: position 0, standstill, SG min/max not available
Driver 4: position 0, standstill, SG min/max not available
Driver 5: position 0, standstill, SG min/max not available
Driver 6: position 0, standstill, SG min/max not available
Driver 7: position 0, standstill, SG min/max not available
Driver 8: position 0, standstill, SG min/max not available
Driver 9: position 0, standstill, SG min/max not available
Driver 10: position 0
Driver 11: position 0
Date/time: 2021-09-10 16:14:50
Cache data hit count 4294967295
Slowest loop: 4.60ms; fastest: 0.17ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 3.4ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, maxWait 0ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is ready with "M122 " in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== DueX ===
Read count 1, 0.08 reads/min
=== Network ===
Slowest loop: 16.01ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.23
WiFi MAC address e0:98:06:22:d0:49
WiFi Vcc 3.36, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 23856
WiFi IP address 192.168.242.45
WiFi signal strength -54dBm, mode none, reconnections 0, sleep mode modem
Clock register ffffffff
Socket states: 0 0 0 0 0 0 0 0
 

And the config.g can be found here (amongst any and all other scripts I'm using on the machine).

I wasn't able to find any "Software reset code" tables or alike. Any suggestions/help are greatly appreciated.

[vrghost]

Ugh, I've just noticed the

@vrghost said in Duet rebooting mid print:

reason: OutOfMemory

message.

I guess I've got little bit too carried away with the scripting. Is there any way I can release a variable or get more RAM?

[dc42]

@vrghost I am surprised that you are running out of memory. Please share the file that you are trying to print.

[vrghost]

@dc42
Here is the file
Tool-Changer Cover Long.gcode

And here is the event log from the last print I've attempted (with the gcode whereabouts)
eventlog.txt

[Phaedrux]

Can you also share your config.g and any other macros that get called along the way?

[mrenz999]

@vrghost I had a similar issue. Did not reboot just stopped printing twice now. Once 3 mins in the second time 6 mins in. I checked on printer both times to find the heaters off, bed and tool, and it stopped. No errors reported just not printing.

[vrghost]

@phaedrux
Of course I can.

https://github.com/VRGhost/tc-rrf/blob/main/dist/ilo-tc/sys/config.g

All other scripts can be found in that repo too.

[gloomyandy]

@mrenz999 If your board is not resetting it is likely that you have a different problem, so you will probably get better help if you start a new thread and describe exactly what happens along with posting the output from running m122 along with your config.g file.

[mrenz999]

@gloomyandy Will do. My heaters just turned off mid print. So no board reset.

[dc42]

@mrenz999 said in Duet rebooting mid print:

@vrghost I had a similar issue. Did not reboot just stopped printing twice now. Once 3 mins in the second time 6 mins in. I checked on printer both times to find the heaters off, bed and tool, and it stopped. No errors reported just not printing.

Did you run M122 and check the last reset time, to see if it was caused by a power glitch?

[vrghost]

Hello,

In my attempts to make a minimal code to replicate the issue, I've stumbled on another bit of unexpected behavior.

If one declares a local variable on the top level of a "job" (a file that is put into the "gcodes" directory on the duet), that local variable is NOT released upon completion of the job. So, a second run of the same gcode is aborted with the

M32 "0:/gcodes/min_bug.gcode"
File 0:/gcodes/min_bug.gcode selected for printing
Error: in GCode file line 2 column 16: meta command: variable 'outer_idx' already exists
Cancelled printing file 0:/gcodes/min_bug.gcode, print time was 0h 0m

message.

To replicate just run this gcode file as a print job twice.
min_var_bug.gcode

[vrghost]

Also, is "abort" command preventing GC of local vars?

The file below works as expected on the first run, but the execution starts with "outer_idx" set to 3 when running the job second time.

min_abort_bug.gcode

[dc42]

@vrghost said in Duet rebooting mid print:

If one declares a local variable on the top level of a "job" (a file that is put into the "gcodes" directory on the duet), that local variable is NOT released upon completion of the job.

That one has already been reported, and fixed in 3.4beta3.

[dc42]

@vrghost I've run that macro several times, using M98 from DWC, and it produces the same output each time. So I think this one is also fixed already.

Did you manage to create a file that causes all the memory to be used up?