My quad carriage machine is failing on Tools. AGAIN.

MaracMB

tried proposed config. nothing new. same results.

it homes all the first time.
but then when i call to homeall again, it fails to home. It either does not move left or right Z motor, or it fails to trigger the endstop.

MaracMB

correction, as i can't see both endstop states i sent home all from phone and monitored Endstops triggered from Settings>machine specific on the screen.
Ran homeall until it fails to home correctly.

It shows endstop as triggered but motor (in this case left Z) keeps grinding and does not stop. It also does this in it's own random pattern on the right side.

Endstops are OMRON optical endstops, fed by 5V from the CONN_SD.
I set the endstops voltage jumper select on the Duex5 to 5V. The endstops on duex are now wired and fed from their designated pins. Same results no matter how i connect them. No matter what the endstop voltage jumper position is.

MaracMB

after failed homing:
as said, on failed Z homing, while motors grinding and the endstop for Z showed as triggered. I hit emergency stop, board restarts. Endstop for Z shows as not triggered

jay_s_uk

Any i2c errors being reported in M122 when it fails to home?

MaracMB

@jay_s_uk when i hit homeall, and again until it fails (it's like a 3rd or fourth attempt) it grinds until i hit emergency stop.
Running m122 right after restart gives:

=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later + DueX5
Board ID: 08DGM-9T6BU-FG3SN-6J9F6-3SN6L-KUWVG
Used output buffers: 3 of 24 (18 max)
=== RTOS ===
Static ram: 27980
Dynamic ram: 97812 of which 44 recycled
Exception stack ram used: 264
Never used ram: 4972
Tasks: NETWORK(ready,348) HEAT(blocked,1224) DUEX(suspended,160) MAIN(running,1816) IDLE(ready,80)
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 00:00:14 ago, cause: software
Last software reset at 2020-11-09 12:27, reason: User, spinning module GCodes, available RAM 4700 bytes (slot 0)
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0441f000 BFAR 0xe000ed38 SP 0xffffffff Task MAIN
Error status: 0
MCU temperature: min 35.3, current 35.8, max 36.2
Supply voltage: min 24.1, current 24.2, max 24.3, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: standstill, SG min/max not available
Driver 1: standstill, SG min/max not available
Driver 2: standstill, SG min/max not available
Driver 3: standstill, SG min/max not available
Driver 4: standstill, SG min/max not available
Driver 5: standstill, SG min/max not available
Driver 6: standstill, SG min/max not available
Driver 7: standstill, SG min/max not available
Driver 8: standstill, SG min/max not available
Driver 9: standstill, SG min/max not available
Date/time: 2020-11-09 12:28:02
Cache data hit count 23635152
Slowest loop: 6.31ms; fastest: 0.13ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 1.1ms, write time 0.0ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 169, MinFreeDm: 169, MaxWait: 0ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 15.71ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.23
WiFi MAC address 84:f3:eb:83:45:0d
WiFi Vcc 3.43, reset reason Unknown
WiFi flash size 4194304, free heap 23352
WiFi IP address 192.168.0.125
WiFi signal strength -48dBm, reconnections 0, sleep mode modem
Socket states: 0 0 0 0 0 0 0 0
=== DueX ===
Read count 1, 4.07 reads/min

so, i'd say no.

Phaedrux

@Phaedrux said in My quad carriage machine is failing on Tools. AGAIN.:

;Mapping drives M584 X0 Y2 Z5:6 U1 V3 W4 ; Extruder mapping M584 E7:8:9:10 ; Extruder mapping

I think we've defined too many drivers, haven't we?
5 on the duet, 5 on the duex. 0-9 for 10 total, Driver10 doesn't exist.

So you have both Z motors and 3 extruders on the Duex? And XYUVW on the Duet?

When you test the endstops individually and check them with M119 they all work and match the expected axis?

MaracMB

Yes @Phaedrux, all endstops work at mapped axis.

It has 11 motors, so there is motor 10 since you count from 0. not sure how would i count and set them otherwise?

(5) XYUVW is on duet
(2) Z1 and Z2 are on duex
(3) Then there are 3 extruders at motors 7,8,9
(1) and the 4th extruder is external wired to CONN_LCD

5+2+3+1=11

Phaedrux

@MaracMB said in My quad carriage machine is failing on Tools. AGAIN.:

(1) and the 4th extruder is external wired to CONN_LCD

I didn't see you you were using an external driver as well.

Phaedrux

Does homeZ by itself work? G28 Z

MaracMB

@Phaedrux yes. But with occasional “glitching” as described.

Phaedrux

One thing I can think of to help narrow down the issue is to change over the motor mappings to have the Z motors on the Duet and move 2 of the carriages over to the Duex. If the problem homing moves to the motors on the Duex then we can be reasonably certain it's a comms issue between the duex and duet.

MaracMB

@Phaedrux this was already done. Having Z motors on duex instead of second and fourth X axis got me to here.

engikeneer

@MaracMB have you managed to isolate whether it is endstop related or motor related?

I'm wondering if you could keep the Z motors on the Duex, but swap the endstops to the Duet? Not sure if there's any firmware/hardware limitations to stop you? I know RRF3 gives you some more freedom...

Equally, have you tried swapping either the motor or endstops to different drivers/pins on the Duex?

Might give some indication of if it is comms or a hardware/board issue.

MaracMB

@engikeneer i suspect it’s endstop related because on a single color print the layers are perfect and motors in question move nicely.

Can’t use just the Duet board for the endstops as there are 7 endstops needed for movement.

I will try moving and remapping the endstops if it’s something board specific
and also swap the PSU, but doubt that will do any difference. Will see.
If that does not work, i’ll revert to v2 firmware and see if that does any good.

Optical enstops is what i like due to accuracy and lack of mechanical clicking. But maybe these Omrons aren’t best choice for duet as they are normally closed and need inverted logic... not sure why it works on everything else.

It is just that i got to 99%, started planing a kit, parts sourcing and suppliers, BOM list for DIY’ers, but having such a glitch is unacceptable and stopped my work.

Troubleshooting this little glitch is taking more time than getting all the way to here.

MaracMB

After almost giving up with this, i reverted to RRF 2.5.1

I suspect that something is wrong with having current versions of RRF3 in duet2 board. While i was unable to find a reason for homing glitches, i did experience slight temperature fluctuations of about +-0.2 C. Almost as there was a noise in the readings. I first suspected suboptimal PID tuning, but the fluctuations were there even at reboot, heaters in idle and at room temperature.

And that actually was the only thing that seemed wrong and gave it away. M122 said nothing.

Many have advised, not a single thing helped.
I have rewired everything 3 times (A LOT OF WIRING), fluctuations persisted on all thermistors.
Having rerouted the cables to boards, not a single thread crossed a ribbon cable between the Duet and Duex. Nothing changed.
I have disconnected everything and only kept VIN power and bed thermistor Still, fluctuation and glitches persisted.
This behaviour (homing twitches and temperature redings noise) was observed on all RRF3 versions.
Not a single variation of config.g did any good.

Then, i reverted to 2.5.1

Temperature readings are now stable, slowest i2c loop is well under 5ms. I have no doubt the occasional homing glitches experienced are also gone. will test this today, but i am very optimistic.

So i guess RRF3 is just not made (well..?) for duet2 boards.

With reverting to RRF2 I will loose conditionals, one axis with two endstops homing, flexibility with connecting stuff, new PID... but i can live with that if the machine at least works as expected and i can print with it.

It is sad that this took so long. it is not that i was the only person using RRF3 with duet2.

Phaedrux

@MaracMB said in My quad carriage machine is failing on Tools. AGAIN.:

I have no doubt the occasional homing glitches experienced are also gone. will test this today, but i am very optimistic.

Please let us know what you find.

dc42

This is a long thread and various things have been changed/reported. I think the problem you are having is that when using RRF3, the axes that use endstop switches connected to the DueX5 don't always home reliably. Is that correct?

I have it on my list to add code re-establish the endstop states on the DueX immediately prior to any homing move. I will try implement this in time for RRF 3.2beta5.

dc42

@MaracMB

Was I correct in my description of the problem in my previous post?
Did reverting to RRF 2.05.1 solve the problem?

dc42

@MaracMB said in My quad carriage machine is failing on Tools. AGAIN.:

@jay_s_uk when i hit homeall, and again until it fails (it's like a 3rd or fourth attempt) it grinds until i hit emergency stop.
Running m122 right after restart gives:

What we really need is a M122 report when it has failed, before you press emergency stop. Can you run it at lower current, so that the "grinding" doesn't do any damage and that you can get this report?

We intend to finalise RRF 3.2RC1 in the next few days, and I would really like to get to the bottom of the issue you are having so that we can implement any necessary fixes first.

MaracMB

@dc42 reverting to 2.5.1 really didn't do much sadly as in the end it didn't compute the endstops (altough registered as triggered) and me using v3 for so long i find v2 just too rigid to stick with it. I have moved ALL the wires away from IDE cable connection and now use the 3.2-beta4 (2020-11-26).

Here's how it stands now:
Machine homes more or less reliably, it does fail to home one axis ( U ) after doing quadruple printing. It always fails to home after print is finnished (G28 X U V W) . That makes it grind like there is no tommorrow on U.
The end code it fails to execute properly is:

G10 P5 S0 R0 		; turn off all heaters
M140 S0 		; turn off bed
T-1		; deselect tools
G28 X U V W		; home all heads
G1 Y140 F2500 		; move Y to front
M84 		; turn all motors off

But i can live with that for now as i can/will/should adjust the routine.

Interestingly, it does not fail to home at start with only "G28". So Yeeey!

I have a long print going on atm, but will send the M122 report tommorrow with a quick quadruple print where the issue happens.

Another issue that is present with RRF3, or at least with my setup, is pretty obvious temperature readings noise. It is most prominent for heaters at idle, but is for sure present also when heaters are on. No matter the PID tuning...

What to look at in the above picture is the jagged temperature curve of Heater 2, 3 and 4.
I didn't experience that with 2.5.1, but as said, i couldn't setup the endstops correctly and i just gave up with that. I need conditionals anyways.

If something is done about endstops in latest RRF3, that would be Fan-Tastic. I guess the temperature readings noise is minor. For now and me at least.