3.3 Final destroys my CoreXYUV (but RC3 is OK)

[deckingman]

Short version.

Running this macro with RRF R3.3 (final) causes the XY gantry to attempt to do wildly different moves to the UV gantry. But with all prior versions of firmware up to and including 3.3 RC3, this macro works fine. The ensuing mayhem using 3.3 (final) tears out the Bowden tubes, water tubes, and much of the wiring inter-connecting the two gantries.

;End print gcode 

G10 ; retract
G4 P500; pause a bit

M98 P"0:/macros/ToolTemps/ToolTemps0.g" ; set the tool temperatures to zero

G4 P500; pause a bit

G91 ; set relative
G1 Z5 F240 ; move bed down 5mm
G90 ; back to absolute
G1 X165 U165 A165 Y300 V300 B300 F9000 ; move to rear

M140 S0 ; turn off bed
M106 S0 ; turn off part fan

M291 P"Print finished" R"Post-Print Macro" S1 T10

This is repeatable - it happens every time with RRF3.3 (final) but reverting back to RC3 cures the problem and that macro runs just fine (as it always has done). I can't tell whether the XY gantry tries to do something different, or if it's the UV gantry, or if it's both.

Longer version

I've been going back to basics with my latest hot end and was testing pressure advance. It appears not to be doing anything with my mixing hot end but that's another story for another thread. Before starting that thread and pre-empting the likely response I would get, I thought that I had better make sure that I was running the latest stable firmware and realised that I was still running 3.3 RC3. So I updated to the latest stable 3.3 package using the zip file. After installing the firmware, I verified that all 4 boards (one 6HC and three 3HC expansion boards were all reporting the correct firmware versions, which they were. I then cycled the power just to be sure and re-verified that all 4 boards were still reporting the same (and the correct) firmware version.

I re-ran my pressure advance test which is a short gcode file that I put together using notepad++ but which calls that post print macro at the end (to move the head out of the way, and turn off the heaters). The file itself ran fine but mayhem ensued when the macro ran at the end. When the G1 XYUVAB move executes, the XY gantry does one thing but the UV gantry tries to do some thing completely different. And because there are assorted wires, and tubes inter-connecting the two gantries, each one will try to drag the other in conflicting directions. The resultant tug of war between 4 powerful Nema 23s is alarming to behold. But the GI XYUVAB moves within the test file itself all executed without any problems - it's that sequence in the macro that causes the problem. Maybe it has something to do with switching to relative for the Z move and then back to absolute? Maybe the firmware misses that G90 and tries to move XYUVAB relative to where they are instead of to the absolute co-ordinates? Who knows?

After making the necessary repairs, I tentatively tried again with one hand on my emergency stop button. It was just as well because the exact same thing happened so I hit the button before too much damage was done. I then reverted back to RC3 and ran the macro again with no such problems.

So all is sweetness and light again with RC3 (apart from the fact that pressure advance seems to be doing nothing with any values up to 0.6 for all 6 extruders) but carnage and mayhem occur if I run the latest stable 3.3 firmware.

[fcwilt]

@deckingman

As a test what if you took the multiple-axis G1 move and did the moves separately, to the degree that was possible.

Frederick

[oliof]

Maybe add an G4 P200 after the G90 to ensure the switch back to absolute mode has been processed and completed before the G1 move.

[deckingman]

@oliof said in 3.3 Final destroys my CoreXYUV (but RC3 is OK):

Maybe add an G4 P200 after the G90 to ensure the switch back to absolute mode has been processed and completed before the G1 move.

I could try that. But I have many macros and that same sequence might exist in those too. So I'd need to go through every one and apply that fudge to every macro and remember to always use that fudge whenever I create a new macro. So I'd prefer it if whatever changed in firmware between rc3 and final which causes this problem was fixed.

[o_lampe]

@deckingman
Maybe you can find a way to avoid switching to G91 at all?
The object model can tell you the momentary Z-position, just add 5mm to it and calculate an absolute move.

[deckingman]

@o_lampe said in 3.3 Final destroys my CoreXYUV (but RC3 is OK):

@deckingman
Maybe you can find a way to avoid switching to G91 at all?
The object model can tell you the momentary Z-position, just add 5mm to it and calculate an absolute move.

Yes, no doubt if I throw enough time and money at it, I could possibly find another fudge to get around another new firmware problem. I've had to do that on numerous occasions in the past. But why the hell should I?

I appreciate you are trying to help, but what I'm trying to do is bring a firmware issue to the attention of those who need to know and who can fix it. I'm not really looking for fudges that I could apply to get around the problem. In fact, I have a fudge which works - use rc3 instead of final.

[o_lampe]

@deckingman
I don't see my solution as a workaround. I wanted to point out, that using G91 as you did is 'oldfashioned'. Firmware devs should look forward.
Ofcourse it's right to point out the obvious timing issue of newer FW and demand it to be fixed, but IMHO not because of the reasons you brought up.

Just my opinion
Olaf

[dc42]

@deckingman there are a couple of known issues in RRF 3.3 although I suspect neither them affects you:

• If you have more than 20 objects on the build plate and your GCode file uses M486 commands to label the objects (e.g. as latest Prusa Slicer does), this can cause unexpected behaviour, typically the firmware crashes
• Homing and other system macro files are supposed to ignore workplace coordinate offsets, but they don't

I presume you think it is this line that isn't working:

G1 X165 U165 A165 Y300 V300 B300 F9000 ; move to rear

I can't think of any reason why the behaviour of that line should be any different in 3.3. Can you try running it with the F9000 changed to a low speed, giving you time to see what is happening and then press the emergency stop button?

Also, can you try firmware 3.4beta3 ?

[deckingman]

@dc42 I don't think it's that G1 XYUVAB line on it's own because the preceding gcode file completes just fine and it contains many such lines. My gut feel is that it's this sequence.

G91 ; set relative
G1 Z5 F240 ; move bed down 5mm
G90 ; back to absolute
G1 X165 U165 A165 Y300 V300 B300 F9000 ; move to rear

When that runs, as far as I can tell, the G1 Z move completes just fine but when the G1 XYUVAB move executes, one or both gantries attempts to do some sort of move which is very different to those co-ordinates. It's hard to tell which gantry misbehaves because they are interconnected with numerous wires and Bowden tubes. So one gantry will try to drag the other which results in belts jumping and motors skipping as they attempt to tear out the wiring and Bowden tubes.

If the G90 was being skipped, then I would expect both gantries to move in sync until they reach the limits, but that isn't what happens. One or other gantries does something completely different to the other. As if G90 was being applied to one gantry but not the other so (say) XY tries to go to 165/300 but UV tries to move by 165/300.

What might be significant is that X and Y (and Z) are on expansion board 3 but UVA and B are on the main board.

To be clear, there are no problems with 3.3RC3 but I do get the problem with 3.3 final.

Given the ensuing damage that results when this goes wrong, (broken and torn out wiring, torn out Bowden tubes and clips, fluid leaks as a result of broken joints in the cooling system pipes), I'm reluctant to try any firmware other than 3.3 RC3 without having some confidence that a cause has been found and a possible solution exists. Trying 3.4. beta3 on the basis of "we don't know if it'll fix so suck it and see" is not an option that I would consider.

EDIT. I had something similar happen some time back which was that one or other gantry tried to make a wild excursion but this was seemingly a rare and random event. This is different in that it is repeatable and neither random nor rare. IIRC, at the time I believe you thought it was possibly caused by a race condition or some such, which you fixed. Up until now, I haven't seen any such repeats of that same problem but maybe it's related?

[dc42]

@deckingman I suspect it is the same fault as these:

https://forum.duet3d.com/topic/24553/rrf-3-3-does-not-respect-max-accel-in-first-moves-after-tc/8
https://forum.duet3d.com/topic/24660/issues-after-prime/71

In which case, what triggers the fault is the combination of G4 P500 (which waits for the move queue to empty), G1 Z5 (which AFAIR in your configuration involves only motors connected to expansion boards), and G1 X165 U165 A165 Y300 V300 B300 F9000 (which involves motors connected to the main board).

If that's the case, then it will not occur using 3.4beta3. A workaround for RRF 3.3 would be to use a M400 or G4 Pxxx command before the G1 X165... command.

[deckingman]

@dc42 That sounds reasonable. Now I have to make some decisions. Do I try M400 or G4 in RRF 3.3? In which case there are probably numerous other instances of similar sequences in other macros that I'll need to look at. On the other hand, do I try 3.4 beta3 which, because it is a beta, might cause me other issues? Whichever route I choose, do I disconnect all the tubes and wires that connect the two gantries together (which is a lot of work) or do leave them as is and try with one hand on the emergency stop in the hope that I can hit it before too much damage occurs?

On balance, maybe I'll just stick with 3.3 RC3 and get on with the rest of my life. I'll test 3.4 when it progresses to a release candidate status.

[resam]

@deckingman can you turn down your motor currents to reduce potential havoc? something just above "barely moving" and below "a single finger can stop the carriage".

[Phaedrux]

@deckingman said in 3.3 Final destroys my CoreXYUV (but RC3 is OK):

On the other hand, do I try 3.4 beta3 which, because it is a beta, might cause me other issues?

On the plus side, you may be able to catch any other issues present before it's too late to make the final release.

[deckingman]

@resam said in 3.3 Final destroys my CoreXYUV (but RC3 is OK):

@deckingman can you turn down your motor currents to reduce potential havoc? something just above "barely moving" and below "a single finger can stop the carriage".

Possibly. The trouble is that my XY gantry has a moving mass of about 1.5 Kgs and the UV gantry is about 3 Kgs. So it takes a fair bit of motor current to accelerate that mass, and I can't reduce it too much. Also, once the gantries are in motion, there is a fair bit of kinetic energy which might still cause some damage when those gantries are travelling in opposing directions. For sure reducing the motor current will help - in fact it's something I do as a matter or course in my homing files in case an end stop were to develop a fault. But it won't eliminate the potential havoc.

[deckingman]

@phaedrux said in 3.3 Final destroys my CoreXYUV (but RC3 is OK):

On the plus side, you may be able to catch any other issues present before it's too late to make the final release.

Moral blackmail again? If the final release has issues it'll be because I didn't test the beta versions. Is it really my job to beta test every firmware version? If so, can I get paid for doing so because it takes up an awful lot of my time.

I've been keeping an eye on things and note a number of threads highlighting issues that have thus far arisen with 3.4 betas. On the basis that I've done more than my fair share of testing over the years in the past, and spent far too many hours evaluating problems and potential solutions, I think I'll wait a while and let others do their bit. I did say that I'll try 3.4 when it gets to release candidate status.

As an aside, I tend to create a sub folder for every firmware that I download before I upload it to my machine. It makes it easier to roll back if I need to. So I have a record of all the firmware versions that I've tried and I see that I have sub folders for (among others) 3.3beta1, 3.3beta1a, 3.3beta2, 3.3RC2, 3.3RC3 and 3.3Final. So I tested 3 betas and 2 release candidates but still ended up with problems when "upgrading" to the stable version.

[Phaedrux]

@deckingman said in 3.3 Final destroys my CoreXYUV (but RC3 is OK):

Moral blackmail again?

How about we just forget I said anything then. Have a good day Ian.

[o_lampe]

@deckingman To protect your installation once and for all you could connect the gantries with a steel cable. The cable(s) would be a bit shorter than the bowden tubes or wiring.

[deckingman]

@o_lampe said in 3.3 Final destroys my CoreXYUV (but RC3 is OK):

@deckingman To protect your installation once and for all you could connect the gantries with a steel cable. The cable(s) would be a bit shorter than the bowden tubes or wiring.

Hmmm, it's a thought... But I like having a certain degree of freedom of movement. Currently I can move the XY gantry about 15mm in all directions relative to the UV gantry. This allows me to use a script which generates UV moves for the extruder gantry (UV) such that it follows the hot end gantry but doesn't replicate every small move. For example the extruder gantry will "park" in the centre of a hole or cylinder while the hot end gantry prints the circles. So I'd need to set the length of the steel wires fairy accurately such that they allow (say) 12mm of movement.

But then what would happen if a firmware bug meant that one gantry tried to do something wildly different to the other? I suspect that I'd likely end up with bent rails or broken gantry parts. The problem with having relatively high moving mass is that one needs to use relatively powerful motors which are capable of doing damage if they are constrained from doing what they want to do.

[o_lampe]

@deckingman
...that's how I destroyed my CNC.

Another crazy idea would be to add a strain gauge or some other emergency switch to the steel cable, but using bullet proof Firmware would be best.
Better have and don't need, than need and don't have they say in Germany

[deckingman]

@o_lampe said in 3.3 Final destroys my CoreXYUV (but RC3 is OK):

@deckingman
...that's how I destroyed my CNC.

Another crazy idea would be to add a strain gauge or some other emergency switch to the steel cable, but using bullet proof Firmware would be best.
Better have and don't need, than need and don't have they say in Germany

Before I converted to a driven gantry for the extruders (UV), I used a passive gantry. That is to say, there were no motors on the UV gantry and it was connected to the hot end gantry with two strings (not steel wires). So the hot end kind of dragged the extruders around. But when I did long fast moves, once the 3Kg or so of 6 extruders got up to speed, it didn't want to stop and change direction so the strings used to snap often, and/or they would jerk the hot end causing it to lift off the print