Movement after M112 E-Stop!

hpiz

@timschneider So I'm starting to lose my mind, and have begun talking to myself on video (lol, really so I have to type less, but I am starting to get quite lost)

Disclaimer: While my Z stepper can and has moved after a first stop of M112, it is far far less consistent, more than 80% of the time it stops only once as it should (compared to like 2% for the servos). But while the hardware differences clearly are making a behavioral difference, I believe the underlying cause (that I am currently convinced of) is still a problem. I'll get there...

@timschneider said in Movement after M112 E-Stop!:

Active disabled/low external drivers

I'm starting to narrow onto this or something about the enable signal (although the pulse/step signal is still misbehaving imo), and it could be as simple a fix as a wiring change.... maybe, I'm still very confused though. And my gut tells me no wiring change can fix this.

I'm getting confused about the effect of the Enable jumper, I can have it on either position and the drivers are active with no other changes to Duet config or driver parameters.

Also within config.g what does enable high (R1) on a driver mean if the enable pin is actually a ground sink, does that mean enable high (R1) means the transistor is on for the Enable(-) pin meaning it's actually pulled to ground, because that appears to be what I'm seeing. Enable low (R0) keeps my enable wire high since I have it fed by 5V through the optoisolater at the servo. And when I switch to enable high (R1) the enable wire is low, indicating current is passing through the opto on the driver.

@timschneider said in Movement after M112 E-Stop!:

check if the stepper motors are active or disabled after the final stop, are you able to move them freely or not? If they are blocked/active that could describe the first and second stop.

M18 test + EStop drive enable check - Conclusion: Stepper and servo motors are disabled after final stop

@timschneider said in Movement after M112 E-Stop!:

put a M112 in the job file, somewhere in the middle.

M112 mid job - Conclusion: Same effect

Oh, I didn't stop there (warning: the oscilloscope will make an appearance)

M112 in job with M291 right after - Conclusion: M291 is not executed but G movements are

M112 at end of job expecting no strange movement - Conclusion: Still moved after a stop...

Investigating that last test a little deeper - Conclusion: Yup, it did still move, and past where it should have. Ok, obviously the buffering of move commands is superseded by the M112 which executes as soon as it's read.

What about the Z stepper, it does behave somewhat better - Conclusion: I'd say you can see a few steps after the first pause, but much better than my other drives.

Ok let's test the x axis - Conclusion: You can see a clear stop and restart, then final stop just like the Y. While I say in-vid it's about the servo's, I'm not sure they are to blame, there's more.

I kept going off video to try reversing the enable polarity in both the servo params and the duet, I found some quirks I mention in the next vid but it's really not in my focus right now. Anyways, none of that changed anything about the behavior.

At this point I felt I had no option but to pull out the scope. I'm analyzing the following video with the following assumptions...

There is no signal being generated by the driver, the signals seen on scope are entirely from the Duet, I feel that's a pretty safe bet.
The difference in behavior between my Servo's and my Z stepper is down to some kind of buffer/safe time between disabling the drive and re-enabling. I would have to either also assume most drivers have said wait time, or my board is acting differently.

Watching the control signals on scope in realtime

After this test, I really can't blame the servo's at all. They're doing exactly what they are being told to do by the Duet, think it might be time give up. Why would there be steps still coming out of the duet after a stop? Why would the enable signal go to disable->enable->disable? I really am having a hard time trying to think of any configuration or parameter I can change to just have the Duet not send steps after an M112 let alone deactivate, then reactivate a driver.

hpiz

@hpiz I don't have the energy today, but I plan to show the same signals going to my z driver, but being ignored by the driver, which I assume will be the case. I mean why would one drive output behave differently from another?

timschneider

@hpiz Thanks again for all these tests and measurements that go far beyond what I would have asked for.

I guess I can't help you any further and @dc42 needs to take over, as it appears to me, that there is a problem with the following configuration:

Duet 3 Main Board 6XD standalone mode
CNC Mode
Firmware 3.4.6
Emergency Stop (DWC, gcode (M112 even within the jobfile itself) and external trigger (trigger0) confirmed)
Maybe: two motors on one axis

Observed behaviour:

movement commands are queued and the Movement System starts to move the drives
M112 is processed
all drives are disabled
at least one movemend command is still being processed in the queue and this will reactivate the drives again
all drives are disabled again (is it done by the M112 or maybe a stop.g/cancle.g?)

maybe I can reproduce it with a simple bench setup.

timschneider

@dc42 I was able to reproduce this behaviour with an brand new of the shelf 6XD WD32096 v1.01 firmware 3.4.6 reflashed to be sure. In standalone mode.

Enable Jumper in active Enable.

Following config.g

; Enable network
M552 P0.0.0.0 S1
    
;CNC Mode
M453                            ; Select CNC device mode
M669 K0 S2 T1

M569 R0 P0.4 S0 T3:3:6:0       ; physical drive 0.4 goes backwards
M569 R0 P0.5 S0 T3:3:6:0       ; driver 5 requires an active high enable, 2.5us minimum step pulse, 2.5us minimum step interval, 5us DIR setup time and no hold time
M569 R0 P0.3 S0 T3:3:6:0       ; driver 5 requires an active high enable, 2.5us minimum step pulse, 2.5us minimum step interval, 5us DIR setup time and no hold time

M569 P0.0 S0 R0 T5:5:9:0       ; physical drive 0.0 goes backwards
M569 P0.1 S1                    ; physical drive 0.1 goes forwards
M569 P0.2 S1                    ; physical drive 0.2 goes forwards

M584 X0.3 Y0.4:0.5 Z0.0             ; set drive mapping

M92 X160.00 Y320.00 Z640.00     ; set steps per mm
M566 X1800.00 Y180.00 Z120.00     ; set maximum instantaneous speed changes (mm/min)
M203 X15000.00 Y12000.00 Z5000.00 I5  ; set maximum speeds (mm/min)
M201 X400.00 Y200.00 Z200.00        ; set accelerations (mm/s^2)

;M564 H0                         ; Allow stepper movement before homing
; Axis Limits
M208 X0 Y0 Z0 S1                ; set axis minima
M208 X600 Y1200 Z170 S0         ; set axis maxima

;Machine Initialization
M18                          ; disable all steppers
M17                          ; enable z stepper so it doesn't fall

test_movement.gcode

G90
G21
 
 
G1 Y500 F4000
G1 Y600 F4000
 
G1 Y500 F4000
G1 Y600 F4000
 
G1 Y500 F4000
G1 Y600 F4000
 
G1 Y500 F4000
G1 Y600 F4000
 
G1 Y500 F4000
G1 Y600 F4000

M122
Screenshot 2023-11-03 at 12-03-32 My Duet.png

EmergencyStop pressed mid job via DWC

Channel 1 (yellow) is STEP-
Channel 2 (pink) is ENABLE-

there is a ENABLE low, when the Emergency Stop is pressed, then high again, then low, high, low again (low is off in this setup). I've tried it also the other was around with R1 - it just inverts the curve.

The Setup:

gloomyandy

@timschneider If you still have the test system available, is there any chance you could run the test again but use M112 entered in the console rather than using the emergency stop button. The reason I ask is because the emergency stop button in DWC runs M112 followed by M999 so some of what you are seeing may be down to the software reset (M999) operation.

timschneider

@gloomyandy sure. I did even more tests.
run M112 from console will change the behaviour a bit, the last disable / enable toggle will disappear. It appears to me, that a reset will also toggle the driver enable once.

I've changed the test setup to be able to test the sbc setup.
Unfortunately, it will also fail, so there is also movement after the M112.

the following screens are all taken with SBC setup 3.4.6

Screenshot 2023-11-03 at 13-43-18 rockpi-4cplus.png
and the following config.g

M669 K0 X1:0:0 Y0:1:0 Z0:0:1 S1 T1                      ; Cartesian kinematics

; Drives
M569 P0 R0 T1:1:2:0
M569 P1 R0 T1:1:2:0
M569 P2 R0 T1:1:2:0
M569 P3 R0 T1:1:2:0
M569 P4 R0 T1:1:2:0
M569 P5 R0 T1:1:2:0

M569 P0 S1
M569 P1 S1
M569 P2 S1
M569 P3 S1
M569 P4 S1
M569 P5 S1

G90                                                     ; send absolute coordinates...
M584 X0 Y4:5 Z3                                         ; set drive mapping

M92 X400 Y400 Z800                                      ; set steps per mm 4000 steps/rev / 10 mm pitch - 5mm on z

M566 X120.0 Y120.0 Z120.00 P1                           ; set maximum instantaneous speed changes (mm/min) and apply jerk on every move
M203 X10000.00 Y10000.00 Z7000.00                       ; set maximum speeds (mm/min)
M201 X400.00 Y500.00 Z1000.00                           ; set accelerations (mm/s^2)
M204 P250 T1250                                         ; Set printing and travel accelerations
M84 S30                                                 ; Set idle timeout

; Axis Limits
M208 X0 Y0 Z0 S1                                        ; set axis minima
M208 X1330 Y1036 Z278 S0                               ; set axis maxima

; Mode Selector
M453                                                    ; Select CNC Mode

without M669 S1 T1

with M669 S1 T1

with M112 in console

with Emergency Stop in DWC

hpiz

@timschneider What a relief to hear this is reproducible, I was starting to get really depressed. And you beat me to yanking my 6xd out of the cabinet and wiring resistors on an unconnected drive output on the 6xd. That was literally gonna be my next course of troubleshooting.

I took a vacation day today, so I'm about to test the Z (although your test all but confirm this is board/firmware behavior). Will post vid shortly.

timschneider

I did some more test, as I was not able to see this behaviour in my last cnc build. So I was recording the STEP and Enable Signal for the other driver 0,1,2,3 and 5 as well, all with M112 mapped to the Y-axis.

M584 X4 Y0:1:3:5 Z2
and
M584 X4 Y0:1:2:5 Z3

btw. the resistors are all 15k

driver0

driver1

driver2

driver3

driver5

timschneider

it looks like the motion system takes some time to shut down - so is it possible that the following will reactivate the drive again after an EmergencyStop?

In the function void DDA::Prepare(SimulationMode simMode) noexcept

hpiz

@timschneider Obviously my scope is way worse than Tim's so I would reference Tim's scope grabs over my shoddy videos, but I'm trying to do what I can...

Z stepper test with M112 in job

Z stepper test with no M112 in job, using DWC EStop

And lastly...
Z stepper test using external estop trigger with T0

That last one, in my mind, all but confirms there is a wait delay in that driver after enable is reactivated, and depending on how long the Duet keeps it enabled, eventually it too does move, but reacts to disabling immediately.

timschneider

@hpiz I was using the 6XD quite a while without noticeing, so I think it is a good teamwork!

and I guess I maybe found the place where the delay kicks in

when the seperate move thread was added, the active switch fall overboard and it appears to me, that TerminateAndUnlink needs some time to terminate and unlink, and in the meantime, new moves are prepared, and the prepare move is enableing the drive again.

droftarts

@timschneider @hpiz Thanks for all your investigative work, I've flagged this to @dc42.

Ian

dc42

Hi all, so does this issue occur only when using external drivers, for example with a 6XD? ~~Does it occur only in SBC mode?~~

@timschneider which source code branch are you looking at? The current 3.5-dev and 3.4-dev source code has this for Move::Exit:

void Move::Exit() noexcept
{
	StepTimer::DisableTimerInterrupt();
	rings[0].Exit();
#if SUPPORT_ASYNC_MOVES
	rings[1].Exit();
#endif
#if SUPPORT_LASER || SUPPORT_IOBITS
	delete laserTask;
	laserTask = nullptr;
#endif
	moveTask.TerminateAndUnlink();
}

That should be sufficient to stop all movement.

@hpiz which firmware version are you using?

PS - when Emergency Stop is pressed in DWC, it sends M112 and then after a little while it sends M999 to reset the Duet. At the reset the driver Enable signal may go to Enabled again, depending on its polarity. But by then the movement queue is empty because RRF has re-initialised.

timschneider

@dc42
sorry I was referring to the commit where the Move Task was added.
But you give the right hint, I was looking in the wrong direction, as it can bee seen from the oscilloscope that the stop of the step generation is delayed. So I guess, another code before Move::Exit is delayed or the very first line StepTimer::DisableTimerInterrupt is slow.

I'll be back in the office on monday, then I can do the above tests with Duet 3 6HC.

hpiz is using 3.4.6 - from the debug log

power up + 00:00:00 [info] Running: Duet 3 MB6XD v1.01 or later: 3.4.6 (2023-07-21 14:11:58)

hpiz

@dc42 Hi, I have a Duet 2 I could test 3.4.6 with to see steps on my analog scope. Not sure if that helps...

dc42

All, when you see this behaviour is it just the remainder of the current move that continues to execute after a pause? The Move::Exit code empties the movement queues but I'm wondering whether the step interrupt might be getting re-enabled and perhaps that might cause the current move to complete.

hpiz

@dc42 From my experience, it would seem it is just continuing with the moves it was already performing. For example, I don't really see any behavior difference between the following two GCode jobs...

G90
G21


G1 Y500 Z100 F4000
G1 Y600 Z0 F4000

G1 Y500 Z100 F4000
G1 Y600 Z0 F4000

G1 Y500 Z100 F4000
G1 Y600 Z0 F4000

M112


G1 Y500 Z100 F4000
G1 Y600 Z0 F4000

G1 Y500 Z100 F4000
G1 Y600 Z0 F4000

vs

G90
G21


G1 Y500 Z100 F4000
G1 Y600 Z0 F4000

G1 Y500 Z100 F4000
G1 Y600 Z0 F4000

G1 Y500 Z100 F4000
G1 Y600 Z0 F4000

M112

The M112 seems to be executed as it's read into the buffer, around the second movement loop, it then tries to complete the third loop after a brief stop (0.25 secs), nothing after M112 seems to ever try to execute, so that's good. Not sure if this matters, but the continued moves seem to move beyond where they should, which in my mind could only happen if more steps are sent than should. I'm gonna test my Duet 2 ethernet today (will post back maybe 2 hours from now) and see if it's got the same step continuation.

As far as the code goes, I wish I could help more but I mostly make websites in C#. Though, I will keep trying.

hpiz

@dc42 My Duet 2 does not produce pulses after M112 running 3.4.6

Note: The scope is in trigger mode

Duet2: M112 in job

Duet2: Using DWC Endstop

dc42

Thanks all, I'll be away at FormNext for a week but I'll look into this as a matter of urgency when I return.

timschneider

@dc42 we have to thank you for your very fast response and in the middle of the preperation for the Formnext!

@dc42 said in Movement after M112 E-Stop!:

All, when you see this behaviour is it just the remainder of the current move that continues to execute after a pause?

To check that, I did another test. Setup as above with SBC. Jobfile the following:

G90
G21

G1 Y0 F60
G1 Y1 F60

G1 Y0 F60
G1 Y1 F60

G1 Y0 F60
G1 Y1 F60

G1 Y0 F60
G1 Y1 F60

G1 Y0 F60
G1 Y1 F60

G1 Y0 F60
G1 Y1 F60

G1 Y0 F60
G1 Y1 F60

G1 Y0 F60
G1 Y1 F60

So every move is around 1 sec (F60 is within the jerk so i guess there is not much acceleration). Based on that, the delay should be less than 1-1.5s, but in fact, it is again around 3.5s (3.46s) as in all the samples above, regardless of the movement length and speed.

So from my understanding, it is not only the remainder of the current move.