Hiccups causing layer shifts

[iratecrayons]

I've been trying to hunt down the cause of layer shifting on my Duet Maestro CoreXY for the last few weeks, an issue I am very certain is not physical in nature. I've adding tracking marks to my belts, swapped out all motors and cables, ramped down speeds, accel and jerks, etc.

When it does shift is isn't on 45 degrees, and the angles and amount is notably varying, and now I am almost certain it is related to hiccups showing up in the debug logs as I always end up with precisely 1 layer shift per hiccups.

I also don't think this is cause by microstepping, I am not using a particularly aggressive configuration and reducing from 32 - 16 microsteps on E and Z has not really resulted in a change. It typically always shifts between 12hours to 12hours of printing.

I am in the process of compiling firmware from source with hasHiccups triggering a stepping error to try and get more result. In the mean time this is my current config.

M569 P0 S0 D3 F2 V1000         ; Drive 0 goes forwards (A)
M569 P1 S0 D3 F2 V1000         ; Drive 1 goes forwards (B)
M569 P2 S0 D3 F2 V1000         ; Drive 2 goes forwards (ZA)
M569 P3 S1 D3 F2 V1000         ; Drive 3 goes backwards (E0)
M569 P4 S0 D3 F2 V1000         ; Drive 4 goes forwards (ZB)
M584 Z2:4 E3                   ; Setup dual Z drives
M906 X1400 Y1400 Z800:800 E800 I50   ; Set motor currents (mA) and motor idle factor in per cent
M350 X16 Y16 Z16 E32 I1     ; Configure microstepping with interpolation
M92 X160 Y160 Z420 E840    ; Set steps per mm
M566 X900 Y900 Z300 E2400      ; Set maximum instantaneous speed changes (mm/min)
M203 X8400 Y8400 Z1200 E2400   ; Set maximum speeds (mm/min)
M201 X1500 Y1500 Z250 E3060    ; Set accelerations (mm/s^2)
M84 S30                        ; Set idle timeout

And the M122 output. (Saving a .txt file from the console seems to have fully stripped line endings, the the formatting may be a bit off as I needed to add them back manually)

10:17:25 PM: M122: === Diagnostics ===
RepRapFirmware for Duet 2 Maestro version 2.02RC6(RTOS) running on Duet Maestro 1.0
Board ID: 08DGM-95762-FD3TD-6JTDD-3SW6S-18AZH
Used output buffers: 1 of 20 (20 max)
=== RTOS ===
Static ram: 19476
Dynamic ram: 97400 of which 0 recycled
Exception stack ram used: 380
Never used ram: 13816
Tasks: NETWORK(ready,680) HEAT(blocked,1300) MAIN(running,1892) IDLE(ready,204)
Owned mutexes:
=== Platform ===
Last reset 31:22:14 ago, cause: power up
Last software reset at 2018-12-20 14:54, reason: User, spinning module GCodes, available RAM 13852 bytes (slot 2)
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0400f000 BFAR 0xe000ed38 SP 0xffffffff Task 0x4e49414d
Error status: 4
Free file entries: 9
SD card 0 detected, interface speed: 15.0MBytes/sec
SD card longest block write time: 0.0ms, max retries 0
MCU temperature: min 38.5, current 39.3, max 40.4
Supply voltage: min 22.6, current 22.9, max 23.1, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: ok, read errors 0, write errors 0, ifcount 28, reads 60883, timeouts 0
Driver 1: ok, read errors 0, write errors 0, ifcount 28, reads 60883, timeouts 0
Driver 2: standstill, read errors 0, write errors 0, ifcount 15, reads 60881, timeouts 2
Driver 3: ok, read errors 0, write errors 0, ifcount 11, reads 60882, timeouts 1
river 4: standstill, read errors 0, write errors 0, ifcount 15, reads 60883, timeouts 0
Driver 5: ok, read errors 0, write errors 0, ifcount 0, reads 0, timeouts 60883
Driver 6: ok, read errors 0, write errors 0, ifcount 0, reads 0, timeouts 60883
Date/time: 2018-12-21 22:17:19Slowest loop: 4581229.00ms; fastest: 0.11ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0
=== Move ===
Hiccups: 1, StepErrors: 0, LaErrors: 0, FreeDm: 150, MinFreeDm: 150, MaxWait: 0ms, Underruns: 0, 0
Scheduled moves: 2552199, completed moves: 2552169
Bed compensation in use: mesh
Bed probe heights: -0.026 -0.090 0.000 0.000 0.000
=== Heat ===
Bed heaters = 0, chamber
Heaters = -1 -1
Heater 0 is on, I-accum = 0.2
Heater 1 is on, I-accum = 0.4
=== GCodes ===
Segments left: 1
Stack records: 3 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is doing ""G1 X96.581 Y75.131 E0.07115"" in state(s) 0
serial is idle in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
lcd is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 4581229.00ms; fastest: 0.03ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
HTTP sessions: 1 of 8
Interface state 5, link 100Mbps full duplex
=== Filament sensors ===
Extruder 0 sensor: ok
10:17:24 PM: Message Log cleared!

Any insight on what else to check would be greatly appreciated.

It is also possible that this change didn't start happening until 2.02RC3. At one point I reverted from RC3 to RC2 and I don't think I was able to reproduce. I updated again at RC5 (and now RC6) and the problem has returned.

[iratecrayons]

Clarification, it happens between 12 to 24 hours of printing, repeating about every 24 hours on ongoing prints.

[iratecrayons]

The result from a hiccup with debugging enabled

DDA: start=[55.863998 49.412998 1.015103] end=[57.040001 50.084000 1.016289] s=1.353967 vec=[0.868561 0.495582 0.000876 0.031352 0.000000]
a=800.000000 d=800.000061 reqv=80.000000 startv=50.267147 topv=50.267147 endv=18.985228 sa=0.000000 sd=1.353967
cks=36658 sstcda=58906 tstcddpdsc=58906 exac=0
DMX: not moving
DMY: not moving
DMZ: not moving
 

And the M122 log

=== Diagnostics ===
RepRapFirmware for Duet 2 Maestro version 2.02RC7(RTOS) running on Duet Maestro 1.0
Board ID: 08DGM-95762-FD3TD-6JTDD-3SW6S-18AZH
Used output buffers: 1 of 20 (15 max)
=== RTOS ===
Static ram: 19484
Dynamic ram: 97416 of which 0 recycled
Exception stack ram used: 388
Never used ram: 13784
Tasks: NETWORK(ready,632) HEAT(blocked,1320) MAIN(running,1792) IDLE(ready,200)
Owned mutexes:
=== Platform ===
Last reset 02:28:09 ago, cause: software
Last software reset time unknown, reason: Hard fault, spinning module Platform, available RAM 13656 bytes (slot 1)
Software reset code 0x0030 HFSR 0x40000000 CFSR 0x02000000 ICSR 0x04432803 BFAR 0xe000ed38 SP 0x200039ec Task 0x4e49414d
Stack: 004184ed 004184f4 61000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 ffffffff 00000032 00000010 20001b68 200062f0 a5a5a5a5 00f9c4ff a5a5a5a5 a5a5a5a5 a5a5a5a5 0043d983 0045ac20 0045faa4
Error status: 2
Free file entries: 9
SD card 0 detected, interface speed: 15.0MBytes/sec
SD card longest block write time: 0.0ms, max retries 0
MCU temperature: min 40.5, current 41.6, max 42.3
Supply voltage: min 23.0, current 23.0, max 23.1, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: ok, read errors 0, write errors 0, ifcount 17, reads 64865, timeouts 0
Driver 1: ok, read errors 0, write errors 0, ifcount 17, reads 64865, timeouts 0
Driver 2: ok, read errors 0, write errors 0, ifcount 216, reads 64865, timeouts 0
Driver 3: ok, read errors 0, write errors 0, ifcount 187, reads 64863, timeouts 2
Driver 4: ok, read errors 0, write errors 0, ifcount 216, reads 64865, timeouts 0
Driver 5: ok, read errors 0, write errors 0, ifcount 0, reads 0, timeouts 64866
Driver 6: ok, read errors 0, write errors 0, ifcount 0, reads 0, timeouts 64865
Date/time: 2018-12-22 22:34:41
Slowest loop: 4581229.00ms; fastest: 0.11ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0
=== Move ===
Hiccups: 1, StepErrors: 1, LaErrors: 0, FreeDm: 137, MinFreeDm: 125, MaxWait: 0ms, Underruns: 0, 0
Steps: 64670372, AvgStepTime: 0.005ms, MaxStepTime: 69.91ms
Scheduled moves: 129437, completed moves: 129407
Bed compensation in use: mesh
Bed probe heights: -0.136 -0.150 0.000 0.000 0.000
=== Heat ===
Bed heaters = 0, chamberHeaters = -1 -1
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Segments left: 1
Stack records: 3 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is doing "G1 X252.132 Y254.041 E0.10519" in state(s) 0
serial is ready with "M122" in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
lcd is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 4581229.00ms; fastest: 0.03ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
HTTP sessions: 1 of 8
Interface state 5, link 100Mbps full duplex
=== Filament sensors ===
Extruder 0 sensor: ok
ok
 

[Veti]

this seems like a strange issue

this user has the same experience on a delta
https://forum.duet3d.com/topic/7809/jerky-motion-in-z-on-maestro/2

[iratecrayons]

@veti Yeah, I saw that post and believe it is the same issue.

It's possible it's related to a problem with the timer, https://forum.duet3d.com/topic/8264/steptimer-getinterruptclocks-isn-t-incrementing/4

[iratecrayons]

I think this is resolved by firmware 2.02, incorporating the timing fixes. I have not had hiccups or layer shifts in the last 2 days, but I'll keep checking for a couple more just in case.

[dc42]

@iratecrayons said in Hiccups causing layer shifts:

I think this is resolved by firmware 2.02, incorporating the timing fixes. I have not had hiccups or layer shifts in the last 2 days, but I'll keep checking for a couple more just in case.

Thanks for confirming!