loss of step external driver duet 1XD

dc42

@tom33 I would like to investigate that Hard Fault reset first. It might be recent, or a very old one. So please do the following:

Send:

M122 B10 P1005

This will cause the 1XD board to reset.

If you then send M122 B10 it should display the time of the software reset and also flag it as "deliberate HardFault".
Reset the main board so that it reconfigures the 1XD board.
Run your test again
Run M122 and M122 B10 again, and post the results here.

tom33

@dc42 said in loss of step external driver duet 1XD:

M122

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.3 (2021-06-15 21:45:47) running on Duet 3 MB6HC v1.01 or later (standalone mode)
Board ID: 08DJM-9P63L-DJ3T0-6J9DL-3S06S-K81H9
Used output buffers: 3 of 40 (18 max)
=== RTOS ===
Static ram: 150904
Dynamic ram: 93128 of which 320 recycled
Never used RAM 106984, free system stack 155 words
Tasks: NETWORK(ready,27.1%,224) ETHERNET(notifyWait,0.1%,126) HEAT(delaying,0.0%,325) Move(notifyWait,0.1%,265) CanReceiv(notifyWait,0.0%,822) CanSender(notifyWait,0.0%,362) CanClock(delaying,0.0%,339) TMC(notifyWait,8.0%,93) MAIN(running,64.5%,1114) IDLE(ready,0.1%,29), total 100.0%
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 00:02:28 ago, cause: reset button
Last software reset at 2021-06-19 22:06, reason: User, GCodes spinning, available RAM 110148, slot 2
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0x00000000 Task MAIN Freestk 0 n/a
Error status: 0x00
Step timer max interval 129
MCU temperature: min 30.1, current 31.4, max 31.6
Supply voltage: min 24.0, current 24.0, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.2, current 12.3, max 12.3, under voltage events: 0
Heap OK, handles allocated/used 99/0, heap memory allocated/used/recyclable 2048/8/8, gc cycles 0
Driver 0: position 145600, standstill, reads 37581, writes 16 timeouts 0, SG min/max 0/0
Driver 1: position 166400, standstill, reads 37583, writes 14 timeouts 0, SG min/max 0/0
Driver 2: position 0, standstill, reads 37583, writes 14 timeouts 0, SG min/max 0/0
Driver 3: position 0, standstill, reads 37584, writes 14 timeouts 0, SG min/max 0/0
Driver 4: position 0, standstill, reads 37584, writes 14 timeouts 0, SG min/max 0/0
Driver 5: position 0, standstill, reads 37587, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2021-06-22 08:34:07
Slowest loop: 131.64ms; fastest: 0.04ms
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 25.0MBytes/sec
SD card longest read time 0.8ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 125, maxWait 16399ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 1750, completed moves 1750, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 1749, 5], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = 3 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is ready with "M122 " in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages queued 3106, received 23, lost 0, longest wait 2ms for reply type 6036, peak Tx sync delay 6, free buffers 49 (min 47), ts 745/744/0
Tx timeouts 0,0,0,0,0,0
=== Network ===
Slowest loop: 143.38ms; fastest: 0.02ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions
HTTP sessions: 1 of 8

Ethernet -
State: active
Error counts: 0 0 0 0 0
Socket states: 5 2 2 2 2 0 0 0

M122 B10
Diagnostics for board 10:
Duet EXP1XD firmware version 3.3 (2021-06-15 16:12:29)
Bootloader ID: SAMC21 bootloader version 2.0 (2020-10-15b1)
Never used RAM 5504, free system stack 2743 words
Tasks: Move(notifyWait,0.3%,111) HEAT(delaying,0.0%,117) CanAsync(notifyWait,0.0%,60) CanRecv(notifyWait,0.0%,75) CanClock(notifyWait,0.0%,64) MAIN(running,95.9%,412) IDLE(ready,0.0%,41) AIN(delaying,3.7%,142), total 100.0%
Last reset 00:04:17 ago, cause: software
Last software reset at 2021-06-19 21:27, reason: deliberate HardFault, available RAM 5504, slot 1
Software reset code 0x8060 ICSR 0x00000003 SP 0x20002b40 Task MAIN Freestk 822 ok
Stack: 20000001 00000000 0001e818 00000001 00000000 0000f6f3 00010810 01000000 a5a5a5a5 a5a5a5a5 00000000 00000000 20000d04 00000444 41000000 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 20003980 a5a5a5a5 a5a5a5a5 a5a5a5a5 0000feb5 a5a5a5a5 000158e5 a5a5a5a5
Driver 0: position -174720, 320.0 steps/mm, steps req 889600 done 569607
Moves scheduled 1317, completed 1317, in progress 0, hiccups 0, step errors 0, maxPrep 577, maxOverdue 5238786, maxInc 143999, mcErrs 0, gcmErrs 0
Peak sync jitter 0/5, peak Rx sync delay 217, resyncs 0/1, no step interrupt scheduled
VIN: 24.3V
MCU temperature: min 19.5C, current 20.0C, max 20.1C
Ticks since heat task active 78, ADC conversions started 128785, completed 128785, timed out 0, errs 0
Last sensors broadcast 0x00000000 found 0 83 ticks ago, loop time 0
CAN messages queued 42, send timeouts 0, received 2863, lost 575, free buffers 37, min 0, error reg 0
dup 0, oos 362/40/0/0, bm 906, wbm 143999, rxMotionDelay 49112, adv -1369537/37186

dc42

@tom33 thanks.

It looks like that original hard fault report was an old one generated from a previous version of the firmware, so we can discount it.

What I see from the M122 log was that the 1XD board received CAN commands faster than it could process them. As a result, some commands were lost and others were executed much later than they should have been. There are a number of factors contributing to this:

You are using a high XY steps/mm i.e. 32 instead of the more common values of 80 to 200. This increases the amount of processor time used to generate step pulses. I suggest you reconfigure the closed loop drivers to use a lower steps/mm.
You are using quite long M569 timings for the driver, i.e. T6:3:6:1. This will also slow down the processor a little (less in firmware 3.3 than in previous versions). The Leadshine drivers should be OK with T2.5:2.5:5:0.
I suspect that the file you are testing with uses lots of short moves. Please share that file so that I can analyse it.

tom33

@dc42 I try reduce the step/mm by reducing the resolution of the stepper driver. (My actual step/mm is 160)

I try this parameters but the motor doesn't move. I need to use 3.5 minimum for the first T value (T3.5:2.5:5:0)

Yes the file is a succession of very short move for some testing X1.gcode

tom33

@dc42 the thing I don't understand is, Now I'm using 3.3 RC+6 and I have no problem but with 3.3 release I have the problem...
Something have changed between both ?

dc42

@tom33 are you using 3.3RC3+6 on both boards, or just one of them? If you are using RC3+6 on both of then, please try upgrading just the 1XD firmware to 3.3 stable, or just the main board firmware to 3.3 stable. If there has been a regression between 3.3RC3+6 and 3.3 stable then it would be helpful to know which firmware it is in. The CAN protocol hasn't changed between 3.3RC3+6 and 3.3 stable, so those versions will talk to each other over CAN.

tom33

@dc42

I try :
1XD : 3.3.0 Main board 3.3.0 Result loss of step / communication
1XD : 3.3 RC+6 Main board 3.3.0 Result : Result loss of step / communication
1XD : 3.3 RC+6 Main board 3.3 RC+6 Result Ok
1XD : 3.3.0 Main board 3.3 RC+6 Result Ok

tom33

@dc42 that helped you ?

dc42

@tom33 thanks, so it's a change in the ~~1XD~~ main board firmware between 3.3RC3+6 and 3.3 stable that has caused the issue. That tells me where to look.

dc42

@tom33 I am working on tracking down and fixing this issue today. Until I have a fix, I suggest you use the combination of RRF 3.3RC3+6 on the main board and 3.3 on the 1XD boards.

dc42

@tom33 this is now fixed in RRF 3.4.0alpha which is available at https://www.dropbox.com/sh/cpwzsx0lrofa7r3/AABllKZQrgJThEvuMog0n4vWa?dl=0. It's compatible with firmware 3.3 stable on 1XD and other expansion boards. It's an alpha build because we don't yet have an official 3.4beta, but the changes since 3.3 are minor. See https://github.com/Duet3D/RepRapFirmware/wiki/Changelog-RRF-3.x-Beta-&-RC#reprapfirmware-34alpha.

We will probably do a 3.3.1 release including this fix.