First Duet Wifi crash after 3 years of regular use
Just had my first crash of the Duet Wifi at the end of a 23hr print. Looks like it was at the last few layers. I have my board now I think 3 years or so and used it regularly.
It happend when I tried to connect to the DWC and it paused and rebooted.
It's strange - what could have caused that?
Is there a way to continue the print where it left off?
Please see the M122 log:
24.6.2020, 20:40:20 M122 === Diagnostics === RepRapFirmware for Duet 2 WiFi/Ethernet version 2.05 running on Duet WiFi 1.02 or later Board ID: 08DDM-9FAM2-LW4S8-6JTDA-3S06P-KMZBW Used output buffers: 1 of 24 (8 max) === RTOS === Static ram: 25712 Dynamic ram: 93176 of which 380 recycled Exception stack ram used: 272 Never used ram: 11532 Tasks: NETWORK(ready,764) HEAT(blocked,1232) MAIN(running,3752) IDLE(ready,160) Owned mutexes: === Platform === Last reset 00:01:43 ago, cause: software Last software reset at 2020-06-24 20:37, reason: Stuck in spin loop, spinning module GCodes, available RAM 11144 bytes (slot 0) Software reset code 0x4043 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0441f80f BFAR 0xe000ed38 SP 0x20001efc Task 0x5754454e Stack: 00446a37 0044a4ba 21000000 00000000 40ddbd6b 00000000 00000000 3331bb4c 40000000 3f317200 b5ddea0e 388aa908 42292c6c 43dbd000 40480f1e 40a00000 488d652b 00000000 40a7cabc 20000010 00000004 00000000 00000004 Error status: 0 Free file entries: 10 SD card 0 detected, interface speed: 20.0MBytes/sec SD card longest block write time: 0.0ms, max retries 0 MCU temperature: min 31.9, current 32.1, max 34.9 Supply voltage: min 24.3, current 24.4, max 24.7, under voltage events: 0, over voltage events: 0, power good: yes Driver 0: standstill, SG min/max not available Driver 1: standstill, SG min/max not available Driver 2: standstill, SG min/max not available Driver 3: standstill, SG min/max not available Driver 4: standstill, SG min/max not available Date/time: 2020-06-24 20:40:19 Cache data hit count 310671732 Slowest loop: 5.45ms; fastest: 0.06ms I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0 === Move === Hiccups: 0, FreeDm: 160, MinFreeDm: 160, MaxWait: 0ms Bed compensation in use: none, comp offset 0.000 === DDARing === Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0 === Heat === Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 === GCodes === Segments left: 0 Stack records: 2 allocated, 0 in use Movement lock held by null http is idle in state(s) 0 telnet is idle in state(s) 0 file is idle in state(s) 0 serial is idle in state(s) 0 aux is idle in state(s) 0 daemon is idle in state(s) 0 queue is idle in state(s) 0 autopause is idle in state(s) 0 Code queue is empty. === Network === Slowest loop: 15.73ms; fastest: 0.00ms Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0) HTTP sessions: 1 of 8 - WiFi - Network state is running WiFi module is connected to access point Failed messages: pending 0, notready 0, noresp 0 WiFi firmware version 1.23 WiFi MAC address 5c:cf:7f:ef:79:0a WiFi Vcc 3.38, reset reason Turned on by main processor WiFi flash size 4194304, free heap 24528 WiFi IP address 192.168.8.123 WiFi signal strength -45dBm, reconnections 0, sleep mode modem Socket states: 0 0 0 0 0 0 0 0
That's unfortunate. If the print is still attached to the bed you can recover still recover it.
Measure the height of the print on the bed if possible so you can get an accurate idea of what layer it was on. Then go to your slicer and either load your saved project or reslice the file with the same settings. Then find the layer that matches what you have already printed. Then use the slicer tools to drop the model down into the bed or cut the height of the model (varies by slicer) so that you only have the top portion of the model left. Change your print settings to have 0 bottom layers. Slice the file and save it.
Now you'll need to modify the gcode file to remove any homing commands or movement commands that would put the print head into the printed part. Get things up to temp and move the Z axis to give yourself some clearance. Home X and Y and send a G92 Z100 command to fake the z axis position to mark it as homed to allow you to move it. Then jog the X Y and Z axis so that you're just touching the top of the print. Now send G92 Z0 to set the top of the print as if it were actually the bed.
Before you start the print prime the nozzle and use the speed factor override to slow things down so you can make adjustments on the fly if need be. Now when you start the modified gcode file it should move over to the print and start printing. Use baby stepping as needed to get the layer right. There will probably be a visible line on that layer but if everything else goes alright you can still recover it to a usable state. Once it's looking ok return to normal speed factor.
As for the crash, I'm not sure what the exact cause could be. @dc42 might be able to tell more from the error code.
I do notice that you are on 2.05. There was a bug fix release for 2.05.1 that you should update to. Or if you're ready to move to RRF3 3.1.1 is available and 3.2 is around the corner.
That's great. Thanks @Phaedrux. The rest of the print is back on now and it's looking perfect, it mightn't leave any visible line.
I took the gcode in question and previewed it in S3D and visually matched it with the failed print. Then I went on and deleted all previous lines in a code editor and checked the result, once happy I did the steps Phaedrux outlined but sent G92 Z238.2 at the end instead, the actual height of the failed print. And it worked flawlessly. Luckily it failed at a solid infill layer, so any damage will be covered up.
@dc42 Any idea why the Duet Wifi crashed?
JoergS5 last edited by JoergS5
@jw88 here https://forum.duet3d.com/topic/15752/stuck-in-spin-loop/4 it was spinning error also, reason was a defective sd card. It mentions a test program, which you could use to verify the card.
@jw88 I’d guess it was the SD card too. See https://duet3d.dozuki.com/Wiki/SD_Card#Section_Troubleshooting_SD_Card_issues
Thanks @JoergS5 I followed the steps mentioned in that post, formatted the sd card with the official tool and checked it with H2testw. Here the result:
So, it's looking ok.
Thanks @droftarts I looked through the wiki and @dc42 mentions problems with some of the pins on the duet wifi board. I'm wondering if that's the cause. Here is a picture of the duet wifi and it looks like the first pin could be off a by a bit.
It seems strange that it would play up after 3 years of regular use.
I would say in the meantime until DC42 can take a look, update to 2.05.1 and report back if you get another reset. So far this has been the only one, right?