How to get DWC to Display what line of Gcode it is on?
-
Context: My Duet 3 6HC keeps locking up on the gcode I'm creating. There is no obvious problem I can see. I've also used some gcode simulators and visualizers and there doesn't seem to be an obvious problem. The files are all around 1,500KB file but the problem is near the beginning. Somewhere between line 1100 and line 1400.
Specific Question: Is there a way to get DWC to display which line of gcode it is processing so that I can narrow the scope of the problem to the specific part of the file? The board is still responsive when it freezes, for example I can hit the emergency stop button in DWC and it responds.
-
@Tinman In DWC 3.5 the G-Code Viewer will display the line that RRF has processed while printing as well as providing the option to play a simulation of the gcode. This includes being able to use arrow keys or mouse clicks to move through the gcode. The thing to know is that it may be a couple lines ahead based on what has been processed and queued by RRF during a print but it will get you close. There is also a CNC mode in the viewer to visualize CNC gcode.
Today you can use the Object Model plugin Job -> File Position to get the position that it has processed to in the file.
-
@Tinman Try running M122 and post the output here. That should either show the gcode it is currently processing or may provide clues as to what has happened (if the firmware crashed for instance). If you can't get the M122 output via the DWC console, try connecting via USB and run the M122 in a terminal.
-
Additionally, you can edit a copy of your gcode file and just cut it off near where you suspect the problem is happening. If the failure still happens, cut off some more. If it doesn't, go back to the original file and cut off less.
This sounds like it might take forever, but not if you do a binary search. Start with big chunks (maybe 1000 lines of code). When you've found which 1000 lines contain the hang, use a 500 line chunk, then a 250 line chunk, then a 125 line chunk, then 60 lines, then 30, then 15, then 8 then 4 then 2, and finally 1.
-
@Sindarius Thanks for your reply. Just turned on the G-code viewer plugin. It looks like it will help a lot with finding the problem. Many thanks!
-
I was able to run M122 where it stopped. Any thoughts on this?
4/9/2023, 3:29:57 PM M122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.5beta2 (2023-02-08 17:43:01) running on Duet 3 MB6HC v1.01 (standalone mode)
Board ID: 08DJM-956L2-G43S4-6J9FA-3SJ6J-1B6QG
Used output buffers: 1 of 40 (37 max)
=== RTOS ===
Static ram: 154344
Dynamic ram: 119692 of which 0 recycled
Never used RAM 68364, free system stack 154 words
Tasks: NETWORK(ready,33.2%,194) ETHERNET(notifyWait,0.2%,401) HEAT(notifyWait,0.0%,355) Move(notifyWait,0.2%,239) CanReceiv(notifyWait,0.0%,943) CanSender(notifyWait,0.0%,335) CanClock(delaying,0.0%,334) TMC(notifyWait,8.3%,56) MAIN(running,58.1%,446) IDLE(ready,0.0%,30), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:33:12 ago, cause: power up
Last software reset details not available
Error status: 0x00
Aux0 errors 0,0,0
Step timer max interval 220
MCU temperature: min 34.9, current 35.1, max 35.6
Supply voltage: min 23.7, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.0, max 12.1, under voltage events: 0
Heap OK, handles allocated/used 99/1, heap memory allocated/used/recyclable 2048/1860/1832, gc cycles 2
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, mspos 968, reads 59035, writes 11 timeouts 0
Driver 1: standstill, SG min 0, mspos 824, reads 59034, writes 11 timeouts 0
Driver 2: standstill, SG min 0, mspos 536, reads 59034, writes 11 timeouts 0
Driver 3: standstill, SG min n/a, mspos 8, reads 59045, writes 0 timeouts 0
Driver 4: standstill, SG min n/a, mspos 8, reads 59045, writes 0 timeouts 0
Driver 5: standstill, SG min n/a, mspos 8, reads 59045, writes 0 timeouts 0
Date/time: 2023-04-09 15:29:56
Slowest loop: 241.56ms; fastest: 0.04ms
=== Storage ===
Free file entries: 18
SD card 0 detected, interface speed: 25.0MBytes/sec
SD card longest read time 1.2ms, write time 13.3ms, max retries 0
=== Move ===
DMs created 125, segments created 9, maxWait 1342009ms, bed compensation in use: none, comp offset 0.000
next step interrupt due in 285840 ticks, disabled
=== DDARing 0 ===
Scheduled moves 1160, completed 1155, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 16], CDDA state 3
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, File2
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is doing "G1 X-6.962 Y2" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is doing "M0" in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 1, axes/extruders owned 0x0000007
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 6962, received 0, lost 0, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 0, free buffers 50 (min 50), ts 6962/0/0
Tx timeouts 0,0,6962,0,0,0 last cancelled message type 30 dest 127
=== Network ===
Slowest loop: 227.78ms; fastest: 0.03ms
Responder states: MQTT(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
HTTP sessions: 2 of 8
= Ethernet =
State: active
Error counts: 0 0 0 0 0 0
Socket states: 5 2 2 2 2 0 0 0
=== Multicast handler ===
Responder is inactive, messages received 0, responses 0 -
@Tinman Doesn't look like the firmware has crashed, but for some reason it seems to be stuck trying to process this line:
File is doing "G1 X-6.962 Y2"Given this line:
Scheduled moves 1160, completed 1155, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 16], CDDA state 3The problem may be a little bit before the above line, as it looks like for some reason the firmware has not been able to complete all of the scheduled moves. Not really sure why. It would probably help if you can reduce things down to as simple as possible gcode file.
-
@gloomyandy Thanks for looking at this with me. Just ran the file again. It failed in a similar manner but not on exactly the same line.
4/9/2023, 4:07:37 PM M122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.5beta2 (2023-02-08 17:43:01) running on Duet 3 MB6HC v1.01 (standalone mode)
Board ID: 08DJM-956L2-G43S4-6J9FA-3SJ6J-1B6QG
Used output buffers: 5 of 40 (36 max)
=== RTOS ===
Static ram: 154344
Dynamic ram: 119692 of which 0 recycled
Never used RAM 68364, free system stack 141 words
Tasks: NETWORK(ready,34.2%,194) ETHERNET(notifyWait,0.2%,423) HEAT(notifyWait,0.0%,357) Move(notifyWait,0.3%,239) CanReceiv(notifyWait,0.0%,943) CanSender(notifyWait,0.0%,335) CanClock(delaying,0.0%,334) TMC(notifyWait,8.6%,56) MAIN(running,56.7%,446) IDLE(ready,0.0%,30), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:13:10 ago, cause: power up
Last software reset details not available
Error status: 0x00
Aux0 errors 0,0,0
Step timer max interval 220
MCU temperature: min 28.3, current 35.4, max 35.7
Supply voltage: min 23.7, current 24.1, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.0, max 12.2, under voltage events: 0
Heap OK, handles allocated/used 99/1, heap memory allocated/used/recyclable 2048/528/500, gc cycles 3
Events: 0 queued, 0 completed
Driver 0: standstill, SG min 0, mspos 376, reads 43089, writes 29 timeouts 0
Driver 1: standstill, SG min 0, mspos 984, reads 43092, writes 29 timeouts 0
Driver 2: standstill, SG min 0, mspos 472, reads 43092, writes 29 timeouts 0
Driver 3: standstill, SG min n/a, mspos 8, reads 43110, writes 11 timeouts 0
Driver 4: standstill, SG min n/a, mspos 8, reads 43110, writes 11 timeouts 0
Driver 5: standstill, SG min n/a, mspos 8, reads 43110, writes 11 timeouts 0
Date/time: 2023-04-09 16:07:37
Slowest loop: 242.52ms; fastest: 0.04ms
=== Storage ===
Free file entries: 18
SD card 0 detected, interface speed: 25.0MBytes/sec
SD card longest read time 0.7ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 125, segments created 9, maxWait 112639ms, bed compensation in use: none, comp offset 0.000
next step interrupt due in 266136 ticks, disabled
=== DDARing 0 ===
Scheduled moves 1130, completed 1127, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 12], CDDA state 3
=== DDARing 1 ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
=== GCodes ===
Movement locks held by null, File2
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is doing "G1 X5.984 Y2.15" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is doing "M0" in state(s) 0
Queue2 is idle in state(s) 0
Q0 segments left 1, axes/extruders owned 0x0000007
Code queue 0 is empty
Q1 segments left 0, axes/extruders owned 0x0000000
Code queue 1 is empty
=== CAN ===
Messages queued 3951, received 0, lost 0, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 0, free buffers 50 (min 50), ts 3951/0/0
Tx timeouts 0,0,3950,0,0,0 last cancelled message type 30 dest 127
=== Network ===
Slowest loop: 226.08ms; fastest: 0.03ms
Responder states: MQTT(0) HTTP(0) HTTP(2) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
HTTP sessions: 2 of 8
= Ethernet =
State: active
Error counts: 0 0 0 1 0 0
Socket states: 5 3 2 2 2 0 0 0
=== Multicast handler ===
Responder is inactive, messages received 0, responses 0
4/9/2023, 4:06:00 PM Stopped at height 47.713 mm
Tool Offset c is Z47.713
Final Tool Offset is Z47.70417
This macro sees global var and the difference between Workpiece Surface and Sexbolt is: -10.2158318m
4/9/2023, 4:05:56 PM Stopped at height 47.693 mm
Tool Offset b is Z47.693
4/9/2023, 4:05:51 PM Stopped at height 47.708 mm
Tool Offset a is Z47.708
4/9/2023, 4:05:48 PM Stopped at height 47.684 mm
4/9/2023, 4:05:21 PM M32 "0:/gcodes/Left Brake Threaded Side v3.nc"
File 0:/gcodes/Left Brake Threaded Side v3.nc selected for printing
4/9/2023, 4:02:42 PM Stopped at height 47.732 mm
Tool Offset c is Z47.732
Final Tool Offset is Z47.72500
This macro sees global var and the difference between Workpiece Surface and Sexbolt is: -10.2158318m
4/9/2023, 4:02:38 PM Stopped at height 47.728 mm
Tool Offset b is Z47.728
4/9/2023, 4:02:34 PM Stopped at height 47.715 mm
Tool Offset a is Z47.715
4/9/2023, 4:02:31 PM Stopped at height 47.702 mm -
@gloomyandy Something I noticed is it is milling at Z-4 and then suddenly the machine freezes and simultaneously DWC displays Z0.886 which is not accurate- it's still at Z-4 position.
-
@Tinman As I said I think the next step is to try and get a small reproducible case that DC42 can take a look at. Ideally you want a very simple config.g file and a short gcode file.
-
@gloomyandy will do
-
-
@gloomyandy The problem goes away if I separate the toolpath steps. I've been working hard to enable toolchanges etc so that I can put in a workpiece, run the file and get a finished part. Having to run 8 separate files to mill a part is not ideal.
-
@gloomyandy You may find it interesting that I can take a file that consistently fails at around 2% of completion, delete the last 90% of the file so that the part where it fails is still intact, re-run the file and it sails right past the place it failed before.
Are there issues related to file size when doing CNC files?
-
@Tinman I see you are running firmware 3.5beta2. Please upgrade to 3.5beta3 and see whether the issue still occurs.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 Thanks for looking into this. I have just upgraded to beta 3 and still having this problem. I'd be happy to send you a file to look at.
-
@Tinman Until then I need to break CNC jobs into 4 to 8 multiple parts to get the machine to run them without needing to do a hard reset. Problem with a hard reset is that it results in the loss of WCS which is almost always probed on a surface that has now been milled away.
-
@Tinman I think the issue may be related to the multiple motion systems in RRF 3.5 and the code in your stop.g file. Does the problem go away if you remove the M0 command from the end of the job file? What commands do you have in stop.g?
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 I opened this topic because I wanted to be able to run a full job with multiple toolchanges from a single gcode file. Perhaps I will start a separate thread about recovering WCS from an e-stop.
For now though, would you take a look at one of my multi toolchange files that fails on beta 2 and beta 3?
-
@Tinman as I said in my previous post, I think the issue is related to the contents of your stop.g file. So I need to see the contents of that file.
I can think of a way to save and restore the WCS offsets, so please start a separate thread on that topic.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com
