[3.4beta6] Toolboard missing extrusion and M122 error

Pakue

@jay_s_uk unfortunately a similar result.

M122 b20
Error: Diagnostics for board 20:
Duet TOOL1LC rev 1.1 or later firmware version 3.4.0beta6 (2021-11-06 11:56:11)
Bootloader ID: SAMC21 bootloader version 2.3 (2021-01-26b1)
M122: Response timeout: CAN addr 20, req type 6024, RID=189

after M122

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.4.0beta6 (2021-11-06 11:38:57) running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956BA-NA3TJ-6J9F6-3SD6M-TB96T
Used output buffers: 1 of 40 (12 max)
=== RTOS ===
Static ram: 151136
Dynamic ram: 65936 of which 580 recycled
Never used RAM 129156, free system stack 127 words
Tasks: SBC(ready,0.6%,476) HEAT(notifyWait,0.0%,320) Move(notifyWait,0.1%,252) CanReceiv(notifyWait,0.0%,771) CanSender(notifyWait,0.0%,365) CanClock(delaying,0.0%,340) TMC(notifyWait,7.5%,58) MAIN(running,91.6%,921) IDLE(ready,0.2%,30), total 100.0%
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 00:45:38 ago, cause: power up
Last software reset at 2021-11-28 23:28, reason: User, none spinning, available RAM 143092, slot 1
Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0044a000 BFAR 0x00000000 SP 0x00000000 Task SBC Freestk 0 n/a
Error status: 0x00
Step timer max interval 133
MCU temperature: min 22.5, current 44.4, max 44.9
Supply voltage: min 23.4, current 23.9, max 24.2, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.1, current 12.1, max 12.2, under voltage events: 0
Heap OK, handles allocated/used 99/2, heap memory allocated/used/recyclable 2048/72/0, gc cycles 0
Driver 0: pos 20093, standstill, SG min 0, mspos 8, reads 29477, writes 11 timeouts 0
Driver 1: pos -3588, ok, SG min 0, mspos 394, reads 29464, writes 24 timeouts 0
Driver 2: pos 845, standstill, SG min 0, mspos 888, reads 29464, writes 24 timeouts 0
Driver 3: pos 0, ok, SG min 0, mspos 339, reads 29471, writes 17 timeouts 0
Driver 4: pos 0, ok, SG min 0, mspos 339, reads 29472, writes 17 timeouts 0
Driver 5: pos 0, ok, SG min 0, mspos 339, reads 29472, writes 17 timeouts 0
Date/time: 2021-11-29 16:52:26
Slowest loop: 1002.34ms; fastest: 0.04ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 125, segments created 42, maxWait 2340188ms, bed compensation in use: mesh, comp offset 0.000
=== MainDDARing ===
Scheduled moves 2057, completed 2052, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state 3
=== AuxDDARing ===
Scheduled moves 0, completed 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamber heaters -1 -1 -1 -1, ordering errs 0
Heater 0 is on, I-accum = 0.7
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Segments left: 1
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File* is doing "G1 X105.864998 Y148.003998 E156.572769" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue* is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty
=== CAN ===
Messages queued 26681, received 43884, lost 0, longest wait 2ms for reply type 6049, peak Tx sync delay 1113, free buffers 49 (min 43), ts 13692/13667/0
Tx timeouts 0,0,0,0,0,0
=== SBC interface ===
Transfer state: 4, failed transfers: 0, checksum errors: 0
RX/TX seq numbers: 42985/42985
SPI underruns 0, overruns 0
State: 5, disconnects: 0, timeouts: 0, IAP RAM available 0x2b6b8
Buffer RX/TX: 1536/3080-0
=== Duet Control Server ===
Duet Control Server v3.4-b6
File /opt/dsf/sd/gcodes/Standard_Box v23.gcode is selected, processing
File:
Buffered code: G1 X105.865 Y148.004 E156.57277
Buffered code: G1 X106.767 Y148.004 E156.64097
Buffered code: G1 X143.747 Y111.023 E160.59435
Buffered code: G1 X143.747 Y111.925 E160.66254
Buffered code: G1 X107.669 Y148.004 E164.51948
Buffered code: G1 X108.571 Y148.004 E164.58768
Buffered code: G1 X143.748 Y112.828 E168.34819
Buffered code: G1 X143.748 Y113.730 E168.41638
Buffered code: G1 X109.473 Y148.004 E172.08047
Buffered code: G1 X110.376 Y148.004 E172.14866
Buffered code: G1 X143.748 Y114.632 E175.71632
Buffered code: G1 X143.748 Y115.534 E175.78451
Buffered code: G1 X111.278 Y148.004 E179.25572
Buffered code: G1 X112.180 Y148.004 E179.32392
Buffered code: G1 X143.748 Y116.436 E182.69871
Buffered code: G1 X143.748 Y117.338 E182.76690
Buffered code: G1 X113.082 Y148.004 E186.04526
Buffered code: G1 X113.984 Y148.004 E186.11346
Buffered code: G1 X143.749 Y118.240 E189.29538
Buffered code: G1 X143.749 Y119.142 E189.36357
Buffered code: G1 X114.887 Y148.004 E192.44907
Buffered code: G1 X115.789 Y148.004 E192.51726
Buffered code: G1 X143.749 Y120.044 E195.50633
Buffered code: G1 X143.749 Y120.946 E195.57452
Buffered code: G1 X116.691 Y148.004 E198.46715
Buffered code: G1 X117.593 Y148.004 E198.53535
Buffered code: G1 X143.749 Y121.848 E201.33155
Buffered code: G1 X143.749 Y122.750 E201.39974
Buffered code: G1 X118.495 Y148.004 E204.09950
Buffered code: G1 X119.397 Y148.004 E204.16770
Buffered code: G1 X143.749 Y123.652 E206.77104
Buffered code: G1 X143.750 Y124.555 E206.83923
==> 1536 bytes
Code buffer space: 1416
Configured SPI speed: 8000000Hz, TfrRdy pin glitches: 0
Full transfers per second: 2.77, max wait times: 36.6ms/10.0ms
Codes per second: 0.15
Maximum length of RX/TX data transfers: 3040/1508

jay_s_uk

@pakue looks like you have an issue with your CAN wiring between the 6HC and the toolboard

Pakue

@jay_s_uk It's printing fine except for the missed extrusions. Wouldn't signalling errors be caught by the CAN "lost" counter?

jay_s_uk

@pakue on the toolboard yes.
but you can't get any info from it, hence the

M122: Response timeout: CAN addr 20, req type 6024, RID=189

Pakue

@jay_s_uk I can try to shorten the CAN cable and add some shielding. Let's see if that helps.

dc42

@pakue the length of the CAN cable is not critical, neither does not normally need to be shielded.

The only time I have seen issues with CAN communication caused by other devices is when I had long wires between the PSU and a Duet 3 main board that was driving a powerful 24V bed heater, and there were separate wires from the PSU to the other CAN devices. In this case the differential ground noise caused by the bed heater turning on/off and the long wires to the PSU introduced a lot of ground noise.

The M122 B20 result is strange, because the board is responding to the first request for M122 data, but failing to respond to some of the subsequent requests. Does M122 B20 always run reliably when the machine is not printing?

Pakue

@dc42 I do indeed have a ~300W 24V heated bed, but the wires are >2mm^2, 30cm in length and split at the PSU between heated bed and the rest of the electronics. I can measure the GND offset/noise at the toolboard when everything is heating up to double check.

The M122 seems to be unreliable during printing, but works (at least sends some data) when the printer just started. About after a print; I'd have to check that.

With RFF 3.3 the M122 doesn't show errors.

dc42

@pakue please make the GCode file you are printing available. I will then test it on the bench, including running M122 while a print is running.

Pakue

@dc42 that one is running 0.9mm extrusion width on a 0.4 nozzle, but I can slice it with standard settings if preferred. The issue is reproducible on other settings too.

Standard_Box v19.gcode

Pakue

@dc42 DC offsets between toolboard GND and Duet3 GND input are

17mV default
33mV with bed heater on
430mV with hotend on

For noise measurements I'd have to pull out the scope.

After a print once everything is turned off (fans, heater, etc.) the M122 still throws errors, so I'm guessing noise is not the root cause?

m122 b20
Error: Diagnostics for board 20:
Duet TOOL1LC rev 1.1 or later firmware version 3.4.0beta6 (2021-11-06 11:56:11)
Bootloader ID: SAMC21 bootloader version 2.3 (2021-01-26b1)
M122: Response timeout: CAN addr 20, req type 6024, RID=234

T3P3Tony

@pakue i agree it looks less likely that noise is the issue. Can you try reinstalling the firmware (M997 B20) on the toolboard. ensure you have the 3.4b6 toolboards firmware binary in the /firmware directory on the SD card.

Pakue

@t3p3tony Updating via CAN it gets stuck on "Please wait while updates are being installed..." and the toolboard only shows up again after downgrading the MB6HC to 3.3.
Even resetting the toolboard (both buttons pressed while powering up) doesn't reset it to the original address (121).

V3.3 on the MB6HC returns the full M122 as far as I can tell

M122 B20
Diagnostics for board 20:
Duet TOOL1LC rev 1.1 or later firmware version 3.4.0beta6 (2021-11-06 11:56:11)
Bootloader ID: SAMC21 bootloader version 2.3 (2021-01-26b1)
Never used RAM 2752, free system stack 2788 words
Tasks: Move(notifyWait,0.0%,153) HEAT(notifyWait,0.2%,115) CanAsync(notifyWait,0.0%,64) CanRecv(notifyWait,0.0%,77) CanClock(notifyWait,0.0%,64) ACCEL(notifyWait,0.0%,61) TMC(delaying,3.0%,57) MAIN(running,91.8%,340) IDLE(ready,0.0%,27) AIN(delaying,4.9%,142), total 100.0%
Last reset 00:03:20 ago, cause: software
Last software reset data not available
Driver 0: pos 0, 655.0 steps/mm,standstill, SG min 0, read errors 0, write errors 1, ifcnt 23, reads 34732, writes 13, timeouts 0, DMA errors 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter 2/6, peak Rx sync delay 212, resyncs 0/0, no step interrupt scheduled
VIN voltage: min 23.8, current 24.0, max 24.1
MCU temperature: min 37.2C, current 37.7C, max 37.8C
Last sensors broadcast 0x00000002 found 1 100 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 4005, send timeouts 0, received 1804, lost 0, free buffers 37, min 37, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
Accelerometer detected: yes, status: 00
I2C bus errors 0, naks 3, other errors 0

M122

M122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.3 (2021-06-15 21:45:47) running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956BA-NA3TJ-6J9F6-3SD6M-TB96T
Used output buffers: 1 of 40 (10 max)
=== RTOS ===
Static ram: 150904
Dynamic ram: 61580 of which 640 recycled
Never used RAM 141068, free system stack 200 words
Tasks: SBC(ready,17.9%,346) HEAT(delaying,0.0%,325) Move(notifyWait,0.0%,302) CanReceiv(notifyWait,0.0%,774) CanSender(notifyWait,0.0%,374) CanClock(delaying,0.0%,339) TMC(notifyWait,7.3%,93) MAIN(running,74.7%,922) IDLE(ready,0.0%,29), total 100.0%
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 00:05:21 ago, cause: software
Last software reset at 2021-11-29 23:23, reason: User, none spinning, available RAM 141068, slot 0
Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0044a000 BFAR 0x00000000 SP 0x00000000 Task SBC Freestk 0 n/a
Error status: 0x00
Step timer max interval 148
MCU temperature: min 43.8, current 44.4, max 44.7
Supply voltage: min 23.6, current 24.0, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.1, current 12.1, max 12.2, under voltage events: 0
Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0
Driver 0: position 0, standstill, reads 28365, writes 11 timeouts 0, SG min/max 0/0
Driver 1: position 0, standstill, reads 28360, writes 16 timeouts 0, SG min/max 0/0
Driver 2: position 0, standstill, reads 28360, writes 16 timeouts 0, SG min/max 0/0
Driver 3: position 0, standstill, reads 28362, writes 14 timeouts 0, SG min/max 0/0
Driver 4: position 0, standstill, reads 28362, writes 14 timeouts 0, SG min/max 0/0
Driver 5: position 0, standstill, reads 28363, writes 14 timeouts 0, SG min/max 0/0
Date/time: 2021-11-29 23:28:40
Slowest loop: 26.00ms; fastest: 0.05ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 125, maxWait 0ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages queued 2896, received 6434, lost 0, longest wait 4ms for reply type 6024, peak Tx sync delay 464, free buffers 49 (min 48), ts 1610/1609/0
Tx timeouts 0,0,0,0,0,0
=== SBC interface ===
State: 4, failed transfers: 0, checksum errors: 0
Last transfer: 2ms ago
RX/TX seq numbers: 51414/51414
SPI underruns 0, overruns 0
Disconnects: 0, timeouts: 0, IAP RAM available 0x2c83c
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.4-b6
Code buffer space: 4096
Configured SPI speed: 8000000Hz, TfrRdy pin glitches: 0
Full transfers per second: 3846.85, max wait times: 18.1ms/20.2ms
Codes per second: 0.20
Maximum length of RX/TX data transfers: 2940/800

T3P3Tony

@pakue thanks that's an interesting result! now you have it working and returning the full result can you try the following:

try the firmware upgrade again
then try upgrading the mainboard to 3.4b6

Pakue

@t3p3tony Upgrading the toolboard to the beta6 works, but after upgrading the MB6HC the M122 issue persists:

m122 b20
Error: Diagnostics for board 20:
Duet TOOL1LC rev 1.1 or later firmware version 3.4.0beta6 (2021-11-06 11:56:11)
Bootloader ID: SAMC21 bootloader version 2.3 (2021-01-26b1)
Never used RAM 2752, free system stack 2788 words
Tasks: Move(notifyWait,0.0%,153) HEAT(notifyWait,0.1%,101) CanAsync(notifyWait,0.0%,64) CanRecv(notifyWait,0.0%,77) CanClock(notifyWait,0.0%,64) ACCEL(notifyWait,0.0%,61) TMC(delaying,3.0%,67) MAIN(running,92.0%,350) IDLE(ready,0.0%,27) AIN(delaying,4.9%,148), total 100.0%
Last reset 00:00:20 ago, cause: software
Last software reset data not available
Driver 0: pos 0, 655.0 steps/mm,standstill, SG min 0, read errors 0, write errors 1, ifcnt 92, reads 10344, writes 11, timeouts 0, DMA errors 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter 3/6, peak Rx sync delay 186, resyncs 0/0, no step interrupt scheduled
VIN voltage: min 23.8, current 24.0, max 24.1
MCU temperature: min 37.9C, current 38.1C, max 38.2C
M122: Last sensors broadcast 0x00000002 found 1 246 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 264, send time
Response timeout: CAN addr 20, req type 6024, RID=17

T3P3Tony

@pakue thanks. I will need some input from @dc42 on this one!

Pakue

@dc42 Did you have success recreating the issue on your machine? On 3.4b7 the issue still persists.

dc42

@pakue I did not succeed in recreating this. Please test with 3.4.0rc2.

Pakue

@dc42 The issue still persists with 3.4rc2. I've also tried shortening the CAN cable without success.

M122 B20

m122 b20
Error: Diagnostics for board 20:
Duet TOOL1LC rev 1.1 or later firmware version 3.4.0rc2 (2022-02-22 10:15:02)
Bootloader ID: SAMC21 bootloader version 2.3 (2021-01-26b1)
All averaging filters OK
Never used RAM 2800, free system stack 88 words
Tasks: Move(notifyWait,0.0%,153) HEAT(notifyWait,0.2%,115) CanAsync(notifyWait,0.0%,65) CanRecv(notifyWait,0.0%,76) CanClock(notifyWait,0.0%,65) ACCEL(notifyWait,0.0%,61) TMC(delaying,3.0%,57) MAIN(running,91.9%,351) IDLE(ready,0.0%,26) AIN(delaying,4.9%,142), total 100.0%
Last reset 00:02:11 ago, cause: power up
Last software reset data not available
Driver 0: pos 0, 655.0 steps/mm,standstill, SG min 0, read errors 0, write errors 0, ifcnt 11, reads 243, writes 11, timeouts 0, DMA errors 0, CC errors 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter 2/6, peak Rx sync delay 198, resyncs 0/0, no step interrupt scheduled
VIN voltage: min 23.8, current 24.0, max 24.1
MCU temperature: min 22.7C, current 31.4C, max 31.4C
M122: Last sensors broadcast 0x00000002 found 1 113 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 1956, send tim
Response timeout: CAN addr 20, req type 6024, RID=18

M122 B20 (after a print)

m122 b20
Error: Diagnostics for board 20:
Duet TOOL1LC rev 1.1 or later firmware version 3.4.0rc2 (2022-02-22 10:15:02)
Bootloader ID: SAMC21 bootloader version 2.3 (2021-01-26b1)
All averaging filters OK
M122: Response timeout: CAN addr 20, req type 6024, RID=210

Pakue

@dc42 Would it be any help to record the CAN bus transmission with a scope, or run a firmware version with more verbose logging?

Phaedrux

I'm not sure it will help, but 3.4 final has just released, can you test with that?

https://github.com/Duet3D/RepRapFirmware/releases/tag/3.4.0