[3.6.0-rc.1] Recovery from failed tool changes

Serbitar

Hi

Not sure this is specific to 3.6.0-rc1 but that's the version I am currently running.

I am trying to set up a auto filament changing system like a box turtle.
In my testing occasionally the filament fails to properly swap for whatever reason.
I have switches at several points along the filament path so detecting a failed swap is easily possible.

The swaps are being done in tool change macros tfree and tpost. When an unexpected state is found, currently the macros call

T-1 P0
M25

This correctly deselects the tool and pauses the printer.
However when trying to recover the state manually in DWC by selecting a tool with T3 or just resuming the print with the resume button on the status page the UI just spins forever.

Hardware is a Duet3 mini in SBC mode with a MB6HC 1.0 as fd-can expansion (yeah I know this sounds backwards but its how the machine evolved).

I couldn't find any examples of aborting a tool change and allowing manual recovery. Am I doing something silly or is there something that needs improving in the firmware?

Thanks

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.6.0-rc.1 (2025-02-28 14:59:42) running on Duet 3 Mini5plus WiFi (SBC mode)
Board ID: W4ZW4-V296U-D65J0-40KM0-LR03Z-H6928
Used output buffers: 1 of 40 (19 max)
=== RTOS ===
Static ram: 94764
Dynamic ram: 89564 of which 228 recycled
Never used RAM 44864, free system stack 132 words
Tasks: SBC(2,nWait 7,2.2%,700) HEAT(3,nWait 6,0.0%,325) Move(4,nWait 6,0.1%,247) TMC(4,nWait 6,1.5%,65) CanReceiv(6,nWait 1,0.0%,814) CanSender(5,nWait 7,0.0%,335) CanClock(7,delaying,0.0%,348) MAIN(1,running,94.9%,683) IDLE(0,ready,0.4%,29) USBD(3,blocked,0.0%,147) AIN(4,delaying,0.8%,261), total 100.0%
Owned mutexes: Aux(MAIN)
=== Platform ===
Last reset 00:40:28 ago, cause: software
Last software reset at 2025-03-20 21:01, reason: User, Platform spinning, available RAM 56288, slot 2
Software reset code 0x2000 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task SBC Freestk 0 n/a
=== Storage ===
Free file entries: 20
SD card 0 not detected, interface speed: 0.0MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
Segments created 502, maxWait 113293ms, bed comp in use: none, height map offset 0.000, hiccups added 0/0 (0.00/0.00ms), max steps late 0, ebfmin 0.00, ebfmax 0.00
Pos req/act/dcf: 1217.00/1216/1.00 -11501.00/-11500/-1.00 -1215.00/-1216/1.00
No step interrupt scheduled
Driver 0: standstill, SG min 0, r/w errs 0/2, ifcnt 19, reads/writes 23470/34, timeouts 0, DMA errs 0, CC errs 0
Driver 1: standstill, SG min 0, r/w errs 0/2, ifcnt 19, reads/writes 23470/34, timeouts 0, DMA errs 0, CC errs 0
Driver 2: standstill, SG min 0, r/w errs 0/2, ifcnt 21, reads/writes 23466/36, timeouts 0, DMA errs 0, CC errs 0
Driver 3: standstill, SG min 0, r/w errs 0/2, ifcnt 15, reads/writes 23474/28, timeouts 0, DMA errs 0, CC errs 0
Driver 4: standstill, SG min 0, r/w errs 0/2, ifcnt 11, reads/writes 23484/20, timeouts 0, DMA errs 0, CC errs 0
Driver 5: standstill, SG min 0, r/w errs 0/2, ifcnt 11, reads/writes 23482/20, timeouts 0, DMA errs 0, CC errs 0
Driver 6: standstill, SG min 0, r/w errs 0/2, ifcnt 11, reads/writes 23482/20, timeouts 0, DMA errs 0, CC errs 0
=== Heat ===
Bed heaters 0 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 0 is on, I-accum = 0.2
=== GCodes ===
Movement locks held by File, null
HTTP* is doing "T1 P0" in state(s) 0
Telnet is idle in state(s) 0
File* is idle in state(s) 0 0 0 7, running macro
USB is idle in state(s) 0
Aux* is doing "M122" in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon* is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
File2 is idle in state(s) 0
Queue2 is idle in state(s) 0
=== CAN ===
Messages queued 25380, received 19462, lost 0, ignored 0, errs 1, boc 0
Longest wait 1ms for reply type 6042, peak Tx sync delay 375, free buffers 26 (min 25), ts 11509/11508/0
Tx timeouts 0,0,0,0,0,0
=== SBC interface ===
Transfer state: 5, failed transfers: 0, checksum errors: 0
RX/TX seq numbers: 27248/27248
SPI underruns 0, overruns 0
State: 5, disconnects: 0, timeouts: 0 total, 0 by SBC, IAP RAM available 0x14878
Buffer RX/TX: 0/0-0, open files: 0
=== Duet Control Server ===
Duet Control Server version 3.6.0-rc.1 (2025-03-03 12:11:46, 32-bit)
HTTP+ProcessInternally:
> Executing M24
Aux+Executed:
> Executing M122
File 0:/gcodes/2color_treefrog_small_hollow_0.2mm_PLA__3h32m.gcode is selected, paused
HTTP:
Buffered code: T1 P0
Buffered codes: 32 bytes total
> Number of flush requests: 1
Code buffer space: 4096
Configured SPI speed: 8000000Hz, TfrRdy pin glitches: 0
Full transfers per second: 38.30, max time between full transfers: 78.0ms, max pin wait times: 58.1ms/3.4ms
Codes per second: 2.40
Maximum length of RX/TX data transfers: 6624/1052

M122 b100
Diagnostics for board 100:
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.6.0-rc.1 (2025-02-28 15:00:13) running on Duet 3 MB6HC v1.0 or earlier (expansion mode)
Board ID: 08DJM-956L2-G43S8-6J9D8-3S86J-9B36F
Used output buffers: 0 of 40 (1 max)
=== RTOS ===
Static ram: 137420
Dynamic ram: 125272 of which 0 recycled
Never used RAM 81820, free system stack 148 words
Tasks: NETWORK(1,ready,11.1%,545) HEAT(3,nWait 6,0.0%,343) Move(4,nWait 6,0.0%,333) TMC(4,nWait 6,3.0%,343) CanReceiv(6,running,0.0%,594) CanSender(5,nWait 7,0.0%,334) MAIN(1,ready,85.8%,500) IDLE(0,ready,0.0%,29) USBD(3,blocked,0.0%,149), total 100.0%
Owned mutexes:
=== Platform ===
Last reset 00:41:02 ago, cause: software
Last software reset at 2025-03-20 21:01, reason: User, none spinning, available RAM 84148, slot 2
Software reset code 0x0013 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0x00000000 Task MAIN Freestk 0 n/a
=== Storage ===
Free file entries: 20
SD card 0 detected, interface speed: 25.0MBytes/sec
SD card longest read time 0.4ms, write time 0.0ms, max retries 0
=== Move ===
Segments created 100, maxWait 0ms, bed comp in use: none, height map offset 0.000, hiccups added 0/0 (0.00/0.00ms), max steps late 0, ebfmin 0.00, ebfmax 0.00
Pos req/act/dcf: 0.00/0/0.00 42401.83/42402/-0.13 0.00/0/0.00 0.00/0/0.00 0.00/0/0.00 0.00/0/0.00
Peak sync jitter -12/1, peak Rx sync delay 186, resyncs 0/0
Next step interrupt due in 119 ticks, disabled
Driver 0: standstill, SG min 0, mspos 8, reads 61369, writes 154 timeouts 13
Driver 1: standstill, SG min 0, mspos 552, reads 61359, writes 164 timeouts 13
Driver 2: standstill, SG min 0, mspos 8, reads 61362, writes 161 timeouts 13
Driver 3: standstill, SG min 0, mspos 8, reads 61362, writes 161 timeouts 13
Driver 4: standstill, SG min 0, mspos 8, reads 61376, writes 154 timeouts 13
Driver 5: standstill, SG min 0, mspos 8, reads 61369, writes 161 timeouts 13
Phase step loop runtime (us): min=0, max=60, frequency (Hz): min=0, max=11718
=== Heat ===
Bed heaters -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamber heaters -1 -1 -1 -1 -1 -1 -1 -1, ordering errs 0
=== CAN ===
Messages queued 19783, received 25696, lost 0, ignored 0, errs 842, boc 0
Longest wait 0ms for reply type 0, peak Tx sync delay 0, free buffers 50 (min 50), ts 1/0/0
Tx timeouts 0,0,0,0,0,0
Motion dup 0, oos 0/0/0/0

jay_s_uk

@Serbitar just an FYI, i'm running an AFC with the AFC-lite board and RRF.
my repo is here but its a little bit outdated https://github.com/jaysuk/BoxTurtle_RRF

Serbitar

@jay_s_uk Oh thats a nice approach, thanks!

jay_s_uk

@Serbitar I don't use any sensors on the tool change. I support both using the turtleneck to load and a measured length.
I still have work to do on when something goes wrong

jay_s_uk

@Serbitar I've just updated my repo to include all my latest changes.
Thanks for the PR!
I'll look at adding instructions to the TeamGloomy wiki for flashing the AFC board

Serbitar

@jay_s_uk No worries! Do you have a thread or somewhere else to discuss the code?

My question regarding recovery from failed changes is still not resolved.
I changed the error code to

M25
abort

And this still results in the firmware getting stuck when attempting manual recovery.

jay_s_uk

@Serbitar not yet. I need to do an announcement post on here. Just need to write the pages for the AFC first.

Looks like it's definitely getting stuck running resume.g
Do you have a resume.g file?
Can you also post your pahse and toolchange files se we can see what's going on?

gloomyandy

@Serbitar I suspect that pausing from a tool change macro is going to cause problems. I'm not even sure what the correct resumption point should be if you do that. Having said that I've also no idea what the correct way is to handle a failed tool change. I tend to treat them in effect as a fatal error on my e3d toolchanger and just abort everything, but I can see how on printers that may have "expected" fails during a change this is not really a good solution.