Extrusion not synced with motion during tpre/tpost on CAN boards

brendon

Wow, something very interesting just happened. I was rearranging the priming/wiping lines, trying to find a correlation between my code and the 'extra filament coming out of the nozzle' and I made a weird discovery.

Let's say this is my t0_prime script:

; Prime
G92 E0.000
G1 E8 F240
G92 E0.000

; WIPE
G91
G1 Y-60 E-0.5 F24000;
G90

If I use this priming macro, the extruder will purge/wipe at the same time, resulting in filament being dragged onto the print bed. If I simply adjust the following line:

G1 E8 F240

and set it to

G1 E8 F150

everything works fine. This issue only occurs on T0, during the tool change. The extruder behaves as normal at all other times. I'm thoroughly confused.

brendon

Now I am even more confused. Priming with

G1 E12 F250;

causes filament to continue extruding. BUT priming with

G1 E11 F250;

works as expected.

droftarts

@brendon marking this for the attention of @dc42, as I don’t know why it would be doing this. My only suggestion is to put an M400 between the two moves.

Ian

brendon

I had M400 commands in my script, but they had no effect. I removed them to reduce the number of lines required to repro my issue.

dc42

@brendon, please can you test whether you still observe these behaviours when you run the Duet in standalone mode. Also, does it go away if you replace the line M98 P"/macros/priming/t0_prime" by the commands in that file. I have a feeling that it may be connected with M98 commands not being synchronised properly.

brendon

Hi @dc42 thanks for the reply. I believe I had tried that on 3.01rc5 and had the same issue. I just moved the priming code into tpostN.g (removed m98 calls) on version 3.01rc6 and things seem to be behaving as expected. I'm about to run a test print to see how things go.

brendon

Calling Tn from DWC/PanelDue properly swaps tools and primes now. However, attempting to run a simple test print, after a layer with T0, the whole system locked up right as T1 was called. DWC is unresponsive the Emergency Stop is ineffective in halting/rebooting the system.

I experienced a similar lockup during the first layer of a single tool print (T1 only for the entire part) I tried Saturday (also on 3.01rc6).

edit: after a power-cycle the machine seems to be running the 4-tool test print fine. Are there logs I can look at if it freezes again?

dc42

@brendon said in Extrusion not synced with motion during tpre/tpost on CAN boards:

Calling Tn from DWC/PanelDue properly swaps tools and primes now. However, attempting to run a simple test print, after a layer with T0, the whole system locked up right as T1 was called. DWC is unresponsive the Emergency Stop is ineffective in halting/rebooting the system.

I experienced a similar lockup during the first layer of a single tool print (T1 only for the entire part) I tried Saturday (also on 3.01rc6).

edit: after a power-cycle the machine seems to be running the 4-tool test print fine. Are there logs I can look at if it freezes again?

If the freeze was caused by a reset, then the M122 software reset data will show that.

Was it running in standalone mode that caused the problem to go away, or was it moving the prime commands into the tpost file?

brendon

Removing the macro lines causes it to behave 'more normally'. The next 4-tool test print eventually locked up while printing, this time 'in the middle of the purge'.

I can't easily access to the duet board/raspberry pi, so swapping back and forth to standalone mode isn't an easy task.

brendon

Interestingly, I left this 'freeze' sit there to see what happened (curious if the heaters were being safely managed, etc). After enough time, the hot-end fans all stopped spinning, indicating that the heaters all disengaged and the temperatures dropped low enough for the fans to disable. Rebooting the system showed that all the hot-end temperatures had in fact dropped due to the heaters disengaging.

dc42

I need the M122 data to see whether there was a reset

brendon

I was unable to run M122 until after I had power-cycled.

m122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.01-RC6 running on Duet 3 MB6HC v0.6 or 1.0
Board ID: 08DJM-956L2-G43S4-6JTD6-3SJ6N-KA6YG
Used output buffers: 1 of 40 (13 max)
=== RTOS ===
Static ram: 154084
Dynamic ram: 162604 of which 20 recycled
Exception stack ram used: 288
Never used ram: 76220
Tasks: ETHERNET(blocked,804) NETWORK(ready,2076) HEAT(blocked,1200) CanReceiv(suspended,3452) CanSender(suspended,1488) CanClock(blocked,1436) TMC(blocked,216) MAIN(running,4776) IDLE(ready,76)
Owned mutexes:
=== Platform ===
Last reset 00:05:33 ago, cause: software
Last software reset at 2020-04-06 17:52, reason: User, spinning module LinuxInterface, available RAM 75972 bytes (slot 2)
Software reset code 0x0010 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x04435000 BFAR 0x00000000 SP 0xffffffff Task 0x4e49414d
Error status: 0
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest block write time: 0.0ms, max retries 0
MCU temperature: min 28.2, current 28.7, max 29.0
Supply voltage: min 23.9, current 24.0, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.1, current 12.2, max 12.2, under voltage events: 0
Driver 0: standstill, reads 10087, writes 14 timeouts 0, SG min/max 0/0
Driver 1: standstill, reads 10088, writes 14 timeouts 0, SG min/max 0/0
Driver 2: standstill, reads 10088, writes 14 timeouts 0, SG min/max 0/0
Driver 3: standstill, reads 10089, writes 14 timeouts 0, SG min/max 0/0
Driver 4: standstill, reads 10092, writes 11 timeouts 0, SG min/max 0/0
Driver 5: standstill, reads 10093, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2020-04-06 17:58:32
Slowest loop: 6.54ms; fastest: 0.21ms
=== Move ===
Hiccups: 0(0), FreeDm: 375, MinFreeDm: 375, MaxWait: 0ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is ready with "M122" in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon* is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 0.69ms; fastest: 0.01ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
HTTP sessions: 0 of 8
- Ethernet -
State: establishingLink
Error counts: 0 0 0 0 0
Socket states: 0 0 0 0 0 0 0 0
=== CAN ===
Messages sent 1367, longest wait 3ms for type 6012
=== Linux interface ===
State: 0, failed transfers: 0
Last transfer: 19ms ago
RX/TX seq numbers: 15528/10694
SPI underruns 0, overruns 0
Number of disconnects: 0
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v1.3.1.0
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 15.82

brendon

After the power-cycle, I attempted the same gcode again. This time the system locked up while waiting for the hot-ends to come to temperature (After g28/g29). It appears to still be in the 'last g29 location' (ie, waiting for m109 command to complete). I have attached the first few lines of the affected gcode for completeness.

G90
M82
M106 S0
M140 S60
M104 S205 T0
M104 S205 T1
M104 S205 T2
M104 S205 T3
M190 S60
G28 ; home all axes
G29
M109 S205 T0
M109 S205 T1
M109 S205 T2
M109 S205 T3

edit: during this test the machine 'maintained' the hot-end/bed temperatures throughout the freeze. They were not 'higher than I was expecting after the power-cycle', but they had retained their set temperatures.

brendon

Actually, it looks it should be reasonably easy to access the sd card on the Duet3, I should be able to convert it over to standalone mode this afternoon. I'll report back.

brendon

Rolling back to stand-alone mode was very easy. I've got the same gcode running now, we haven't made it 'the furthest', yet, but it appears to be running smoothly.

It's worth noting that transition time between tools seems faster. With the SBC attached, there was a momentary 'pause' between tfreeN.g and tpreN.g. The transitions are now immediate in stand-alone mode.

edit: we have made it further than any other attempt. I'll let it keep running as I'm benchmarking 'time per cm with minimal printing' (ie tool-change overhead).

chrishamm

@brendon Sorry to hear you're having problems with the CAN board while connected over the SBC. I'd like to figure out why it is happening, but I will need two separate debug logs for this. Can you please connect your Duet 3 over USB to a computer, open a serial terminal, send M111 P3 S1 and try to reproduce this bug once with the SBC connected and once without it? I hope that will allow me to spot differences in the execution of G/M/T-codes and to track down the underlying bug.