Homing Inconsistent Duet 3

[tobias_munich]

@dc42 (M400) thats a good point. will test it.
and good to know, that you have it on your list, many thanks.

but again. it's not a major issue. the homing scripts are working independently very well

[Luke'sLaboratory]

I'm a Duet 3 user, with SBC and single 3HC, 3.01 RC3, DSF 1.2.4

My homing is now consistently bugged when using the homeall.g command, regardless of whether I call individual macros or not.

It happens on my X-axis, and behaves like the X Limit Switch is pressed, immediately stopping when X is homed.

I have verified using the machine-specific diagnostics that the trigger is not being pressed, and I've confirmed that it does register correctly when I use my homex.g command and I can see it press, as well as manually pressing it.

I've tried M400 as well as changing homeall.g to incorporate the gcode from homeX, same result, it behaves like the button is always pressed but then processes homeY+homeZ appropriately.

Something weird is afoot here!

[Danal]

@Luke-sLaboratory said in Homing Inconsistent Duet 3:

I'm a Duet 3 user, with SBC and single 3HC, 3.01 RC3, DSF 1.2.4

My homing is now consistently bugged when using the homeall.g command, regardless of whether I call individual macros or not.

It happens on my X-axis, and behaves like the X Limit Switch is pressed, immediately stopping when X is homed.

I have verified using the machine-specific diagnostics that the trigger is not being pressed, and I've confirmed that it does register correctly when I use my homex.g command and I can see it press, as well as manually pressing it.

I've tried M400 as well as changing homeall.g to incorporate the gcode from homeX, same result, it behaves like the button is always pressed but then processes homeY+homeZ appropriately.

Something weird is afoot here!

@Luke-sLaboratory please post your config.g, and homing files.

[Luke'sLaboratory]

@Danal

; Drives
M569 P0.1 S1                    ; physical drive 0.1 goes forwards XY
M569 P0.0 S1                    ; physical drive 0.0 goes forwards XY
M569 P0.2 S0                    ; physical drive 0.2 goes forwards Coupler
M569 P0.3 S1				    ; physical drive 0.3 goes backwards Z
M569 P0.4 S1				    ; physical drive 0.4 goes backwards Z
M569 P0.5 S0                    ; physical drive 0.5 goes backwards Z
M569 P1.0 S0					; Extruder 1
M569 P1.1 S0					; Extruder 2
M569 P1.2 S0					; Extruder 3
M584 X0.1 Y0.0 Z0.3:0.4:0.5 C0.2 E1.0:1.1:1.2                     ; set drive mapping
M350 X16 Y16 E16:16:16 I1                      						  ; configure microstepping with interpolation
M350 Z16 C8 I0														  ; No interpolation on Z Axis
M92 X160.00 Y160.00 Z1600.00 E409.00:409.00:409.00 C100         ; set steps per mm
M566 X900 Y900 Z600 E1800:1800:1800 C2          ; set maximum instantaneous speed changes (mm/min)
M203 X50000.00 Y50000.00 Z840 E10800.00:10800:10800 C5000   ; set maximum speeds (mm/min)
M201 X2000.00 Y2000.00 Z18 E2500:2500:2500 C400      ; set accelerations (mm/s^2)
M906 X2300 Y2300 Z1800 E1100:1100:1100 C600 I60             ; set motor currents (mA) and motor idle factor in per cent
M84 S120                                      ; Set idle timeout
 
;Stall Detection
M915 C S3 F0 H200					; Coupler
 
; Axis Limits
M208 X-17 Y-15 Z0 C0 S1                             ; set axis minima
M208 X622 Y660 Z950 C500 S0                       ; set axis maxima
M671 X290.0:-32:627 Y658:185:186 S20 ; leadscrews at rear middle, front left+right
 
; Endstops
M574 X1 S1 P"!io2.in"                         ; configure active-low endstop for low end on X via pin io2.in
M574 Y1 S1 P"!io0.in"                         ; configure active-low endstop for low end on Y via pin io0.in
;M574 Y2 S1 P"!io1.in"			      ; configure active-low endstop for high end on Y via pin io1.in
 
;Tool Presence Checking
M950 J"io4.in"
 
; Z-Probe
 
M558 P5 C"io3.in" H10 F120 T3600      ; set Z probe type to simp switch and the dive height + speeds
G31 X0 Y0 Z0 P200                        ; set Z probe trigger value, offset and trigger height
;M557 X15:600 Y15:600 S30                     ; define mesh grid - Whole Bed
M557 X220:400 Y220:400 S50                     ; define mesh grid - Center Small

for my homeall

G91                     ; relative positioning
G1 H2 Z5 F600          ; lift Z relative to current position
G90
M400
M98 P"homex.g"
M400
M98 P"homey.g"
M400
M98 P"homez.g"			; this will leave it 5mm above the bed
M400
M98 P"homec.g"
 

HomeX.g - same thing for homey just in y - comments are to try to diagnose this issue.

M400
;M913 X40 Y40
G1 H1 X-800 F6000 ; move quickly to X axis endstop and stop there (first pass)
G1 X5 F3600       ; go back a few mm
G1 H1 X-627 F360  ; move slowly to X axis endstop once more (second pass)
M400
;M913 X100 Y100

[Danal]

Looks pretty much exactly like mine on the Jubilee:

; Home X Axis
 
G91                     ; Set relative mode
G1 X-999 F6000 H1       ; Big negative move to search for endstop
G1 X4 F600              ; Back off the endstop
G1 X-10 F600 H1         ; Find endstop again slowly
G90                     ; Set absolute mode
G1 X0 F6000

I do see an interesting difference in our config.g: Your X endstop:
M574 X1 S1 P"!io2.in" ; configure active-low endstop for low end on X via pin io2.in

Mine:
M574 X1 S1 P"io1.in" ; Set homing switch configuration X1 = low-end, S1 = active-high (NC)

I'm wondering why the invert, the "!"... Are your switches "normally closed"? If they are "normally open", I wonder if the input pin is picking up noise when the motors are moving? I'd switch the physical wiring to "normally closed"... but, as a quick test, maybe add pullup to your definition and see what happens?
M574 X1 S1 P"!^io2.in"

P.S. I notice you are using io0.in for Y endstop. Documentation says don't use 0, it is reserved for PanelDue. However, I have used it; I missed that document at first. It seemed to work just fine.

[Luke'sLaboratory]

@Danal

Normally Open - My RB's were wired like that and I stick with it. no other reason than that.

Not going to say its 100% not the wiring, but this is a setup that has been fully working for a few months without changes, including the io0.in, but you're right, I could move that.

This works 100% of the time I just home X, using the script. If it was mechanical/electrical, I would expect it to behave similarly with the single call.

I'll give pullup a try, but I'm not confident it will make a difference.

[spllg]

@Luke-sLaboratory said in Homing Inconsistent Duet 3:

M201 X2000.00 Y2000.00 Z18 E2500:2500:2500 C400

have you tried reducing accelerations?

[tobias_munich]

@Luke-sLaboratory my problem is exactly the same.
My first guess was that the optical endstop has a problem. I changed my Hypercube to sensoreless homing. Which works great

My homeall.g looks exactly like yours

But anyhow the issue is still the same.

[Danal]

@Luke-sLaboratory said in Homing Inconsistent Duet 3:

Normally Open - My RB's were wired like that and I stick with it. no other reason than that.

Yeah, and it is easy to sit here and say "Change this, change that"... but... I'd still recommend going normally closed. For one thing, it inherently detects any wiring flaws.

"Worked for months". Could be a version change of the firmware, but much more likely to be something that changed, very subtly, in the hardware. Connection becoming slightly flakey, etc.

RE: The io0... I used it for months before I found the document. So I wouldn't change it. Also, it is Y, correct? And your real problem is with X.

In summary: I certainly can't explain how it worked or what changed. Nonetheless, I'd swap to NC.

[Luke'sLaboratory]

@spllg

Curious - what would acceleration do for homing?
Its not actuating the switch from pure inertia, it just thinks the button is always pressed for some reason.

[Luke'sLaboratory]

@Danal

I hear ya, I hear ya

The problem is that N.O can't randomly close the connection, Both ends on my switches are heat shrunk, and the other ends are properly sitting in their crimped spots.

If it was NC, yeah, if i had a poor connection, it would make sense if something broke, but here wires would have to wear+arc, and that isn't the case, at least as long as I can see, nor does it show up in firmware.

I found that If i power restart the machine, the first home works correctly, but any subsequent homes do not until reset again.

We'll see what RC4 brings us.

[A Former User]

@Luke-sLaboratory said in Homing Inconsistent Duet 3:

The problem is that N.O can't randomly close the connection, Both ends on my switches are heat shrunk, and the other ends are properly sitting in their crimped spots.

it has more to do connections breaking rather than being made; if the NO switch gets disconnected your homing will crash. If your NC switch becomes disconnected it can be detected and crashes avoided (although the common homing sequences does not seem to take advantage of that)

[Danal]

Bearer is correct on one of the prime underlying reasons for the industry "best practice" for NC.

Luke, in your case, I don't suspect wiring causing an issue in terms of a wire itself causing a false trigger... I've worked with you on other discords and you "do things right" on crimping and wiring and such... Instead, I suspect "noise" or a "floating wire" or "capacitance" or something similar. Can't prove it... really don't know why it used to work and works once after a power cycle... but I do know that NC puts the wiring in a "known state", namely hard pull to ground, until that switch clicks.

[Danal]

The more I think about that "works once after a power cycle" the more I become convinced this is a "floating wire" issue. After power cycle, that wire is ???? Neutral ??? I don't know what word to use.

After the first home (that works), the wire becomes "charged" or "discharged" or something.

Maybe.

[spllg]

@Luke-sLaboratory i have no idea. the only thing i know is that after setting acceleration to a higher value homing[xy] ceased to work. after lowering acceleration homing was working. --> https://forum.duet3d.com/topic/14614/unable-to-home-xyz-using-my-homing-scripts?_=1584452867098

[Danal]

@spllg said in Homing Inconsistent Duet 3:

@Luke-sLaboratory i have no idea. the only thing i know is that after setting acceleration to a higher value homing[xy] ceased to work. after lowering acceleration homing was working. --> https://forum.duet3d.com/topic/14614/unable-to-home-xyz-using-my-homing-scripts?_=1584452867098

AHHH! New data point! (Maybe I missed it earlier). Let me think about that for a while.

[Luke'sLaboratory]

@bearer

I agree in practice - My comment in this thread was addressing how much harder it is to get a false positive (switch shows ON when it shouldn't be) connection with a N.O switch.

I recognize the preferred convention is to fail safe by having a NC switch that should error check that its a closed circuit.

@spllg No clue there - It has been working with these values for several months, only recently has there been an issue.

@Danal Lmk when you figure it out - no issues sofar in RC4, but that's 3 datapoints

[A Former User]

@Luke-sLaboratory said in Homing Inconsistent Duet 3:

I agree in practice - My comment in this thread was addressing how much harder it is to get a false positive (switch shows ON when it shouldn't be) connection with a N.O switch.

i would think a hard pull down to ground is more stable than a weak pull up when it comes to coupled or induced noise?

[Danal]

@Luke-sLaboratory said in Homing Inconsistent Duet 3:

but that's 3 datapoints

lol

[Garfield]

@T3P3Tony So finally it happened again, again this was during homing procedures.

DuetControlServer does NOT hang or fail in a way that the linux OS understands

This is the message in the console, it is erroneous - whatever mechanism is used to determine the RRF 'availability' is failing and not the homing process - the mechanism causing the 'Transport endpoint is not connected' error is the problem here.

22/03/2020, 12:31:28 	G28 Y
Error: Operation failed (Reason: Transport endpoint is not connected)
22/03/2020, 12:31:28 	Error: G0/G1: insufficient axes homed

This is the report from sysctl - clearly shows duetcontrolserver is still running - but it has stopped communicating with the Pi

pi@duet3:~ $ sudo systemctl -l status duetcontrolserver
● duetcontrolserver.service - Duet Control Server
   Loaded: loaded (/lib/systemd/system/duetcontrolserver.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2020-03-22 00:29:41 GMT; 12h ago
 Main PID: 340 (DuetControlServ)
    Tasks: 15 (limit: 4915)
   Memory: 50.9M
   CGroup: /system.slice/duetcontrolserver.service
           └─340 /opt/dsf/bin/DuetControlServer
 
Mar 22 12:24:54 duet3 DuetControlServer[340]: [info] Executing nested macro file config-override.g on channel Trigger
Mar 22 12:24:54 duet3 DuetControlServer[340]: [info] Trigger: Finished macro file config-override.g
Mar 22 12:24:54 duet3 DuetControlServer[340]: [info] Trigger: Finished system macro file config.g
Mar 22 12:31:02 duet3 DuetControlServer[340]: [info] Executing nested macro file homex.g on channel HTTP
Mar 22 12:31:16 duet3 DuetControlServer[340]: [info] HTTP: Finished macro file homex.g
Mar 22 12:31:16 duet3 DuetControlServer[340]: [info] Executing nested macro file homey.g on channel HTTP
Mar 22 12:31:28 duet3 DuetControlServer[340]: [info] Received file abort request on channel HTTP for all files
Mar 22 12:31:28 duet3 DuetControlServer[340]: [info] Aborted macro file
Mar 22 12:31:28 duet3 DuetControlServer[340]: [warn] HTTP: Out-of-order reply: 'G0/G1: insufficient axes homed'
Mar 22 12:31:28 duet3 DuetControlServer[340]: [error] G0/G1: insufficient axes homed
pi@duet3:~ $ 

The journal contains the same messages.

pi@duet3:~ $ sudo journalctl -fu duetcontrolserver     
-- Logs begin at Thu 2019-02-14 10:11:59 GMT. --
Mar 22 12:24:54 duet3 DuetControlServer[340]: [info] Executing nested macro file config-override.g on channel Trigger
Mar 22 12:24:54 duet3 DuetControlServer[340]: [info] Trigger: Finished macro file config-override.g
Mar 22 12:24:54 duet3 DuetControlServer[340]: [info] Trigger: Finished system macro file config.g
Mar 22 12:31:02 duet3 DuetControlServer[340]: [info] Executing nested macro file homex.g on channel HTTP
Mar 22 12:31:16 duet3 DuetControlServer[340]: [info] HTTP: Finished macro file homex.g
Mar 22 12:31:16 duet3 DuetControlServer[340]: [info] Executing nested macro file homey.g on channel HTTP
Mar 22 12:31:28 duet3 DuetControlServer[340]: [info] Received file abort request on channel HTTP for all files
Mar 22 12:31:28 duet3 DuetControlServer[340]: [info] Aborted macro file
Mar 22 12:31:28 duet3 DuetControlServer[340]: [warn] HTTP: Out-of-order reply: 'G0/G1: insufficient axes homed'
Mar 22 12:31:28 duet3 DuetControlServer[340]: [error] G0/G1: insufficient axes homed

The message 'executing nested macro' is misleading - these macros are not nested unless the Pi or something in the background is doing this - my homex, homey and homez are all independent, when this failure occured I was homing axes individually. The error insufficiant axes homed is also erroneous. I was actually homing at the time of the error and not executing any command that required the axes to be homed.

Anything in the queue remains in the queue and the interface appears to be hung - it is NOT hung - it is waiting for a macro to complete that never will. When this transport error occurs the system needs to clear any outstanding queued commands and attempt to re-establish communication.

The time is also incorrect - the systemctl reports that the system has been running for 12 hours - it has not, it was started at approx 10am

-- Logs begin at Thu 2019-02-14 10:11:59 GMT. --
Mar 22 12:45:01 duet3 CRON[17414]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Mar 22 12:45:01 duet3 CRON[17410]: pam_unix(cron:session): session closed for user root
Mar 22 12:47:53 duet3 sudo[17864]:       pi : TTY=pts/0 ; PWD=/home/pi ; USER=root ; COMMAND=/usr/bin/journalctl -fu duetcontrolserver
Mar 22 12:47:53 duet3 sudo[17864]: pam_unix(sudo:session): session opened for user root by pi(uid=0)
Mar 22 12:48:00 duet3 sudo[17864]: pam_unix(sudo:session): session closed for user root
Mar 22 12:49:47 duet3 sudo[18117]:       pi : TTY=pts/0 ; PWD=/home/pi ; USER=root ; COMMAND=/usr/bin/journalctl -fu
Mar 22 12:49:47 duet3 sudo[18117]: pam_unix(sudo:session): session opened for user root by pi(uid=0)
Mar 22 12:49:47 duet3 sudo[18117]: pam_unix(sudo:session): session closed for user root
Mar 22 12:49:51 duet3 sudo[18142]:       pi : TTY=pts/0 ; PWD=/home/pi ; USER=root ; COMMAND=/usr/bin/journalctl -f
Mar 22 12:49:51 duet3 sudo[18142]: pam_unix(sudo:session): session opened for user root by pi(uid=0)
 

Is the time a clue ? and why is it so messed up .... I'm not a linux fan and this isn't helping that opinion