After Firmware update from 3.4.5 to 3.5.2 my printer crashes

Chrashem

that's interesting! I checked my g-code flavor in the slicer, but it's already set to RepRap. I barely remember, but in my opinion the slicer was set to klipper by default at the installation . When I configured it, I set it to "RepRap".

This kind of code which causes the illegal character issue is in the "custom-code" area of the slicer. To adjust it you have to do it manually.
I know that I was wondering about it several times, but I never touched it since 1.5 years because the g-code still worked.

Now after the firmware update it was the first time the g-code from the slicer isn't working.
After deactivating these lines, the code runs (after a restart of the mainboard, otherwise it leads directly to a crash).

Summarised these are the following lines which a had to deactivate:

SET_GCODE_OFFSET Z=0

START_PRINT EXTRUDER_TEMP={first_layer_temperature+extruder_temperature_offset} BED_TEMP=[first_layer_bed_temperature]

{if extrusion_role=="InternalInfill"}SET_VELOCITY_LIMIT SQUARE_CORNER_VELOCITY=10 {endif}
{if extrusion_role!="InternalInfill"}SET_VELOCITY_LIMIT SQUARE_CORNER_VELOCITY=5 {endif}

I assume all of this are klipper commands?

droftarts

@Chrashem said in After Firmware update from 3.4.5 to 3.5.2 my printer crashes:

I assume all of this are klipper commands?

Yes, they look like Klipper commands. Maybe at some point the profiles got updated, and if you have RatRig selected, it put those commands in. Being in the custom Gcode section, they would still be output despite 'RepRap' being selected as the Gcode flavour.

I don't know if the RRF Gcode interpreter has changed to make the underscore _ an illegal character recently (it is possible, though I don't think there has been any change to this), but I can't see anything in the release notes saying this. I think the error is being generated because it is seeing the T in 'SET_...' as a tool change command, and then the underscore is not a valid character for the 'T' tool command.

Ian

Chrashem

I cleared all the commands and I did my first test print. But I have still a little issue with the movement. Sometimes my movement is a bit jerky.
For example, if the printer should print a straight line in a continuous movement, it's printing a straight line, but the movement isn't continuous.
It's hard to explain. Its not like a layer shift, its more like a "stopping" for a fraction of a second" You can also her hear that the movement isn't continuous.

I figured out, that it's not depending in which direction the movement is going.
So unfortunately it's not just one stepper. It's happening in diagonal or straight moves.

Edit:
I think I have to take a closer look at my belt drive system. In my opinion it isn't working fluently.
When I my move the printhead by hand it seems that there a some points with a lot of friction.
Maybe my damage is bigger than I thought.

droftarts

@Chrashem said in After Firmware update from 3.4.5 to 3.5.2 my printer crashes:

Sometimes my movement is a bit jerky. For example, if the printer should print a straight line in a continuous movement, it's printing a straight line, but the movement isn't continuous. It's hard to explain. Its not like a layer shift, its more like a "stopping" for a fraction of a second" You can also her hear that the movement isn't continuous.

While it is possible this is due to damage to the drive system, if you're using mesh compensation, it may also be due to the Z axis having to move when the mesh compensation requires it. Having slow Z speeds can make this more of an issue. Currently, your speeds are:

M350 X16 Y16 Z16 E16 I1                    ; configure microstepping with interpolation
M92 X80.00 Y80.00 Z800.00 E686.5           ; set steps per mm LDO Orbiter 2.0  (Nach Kalibrierung am 30.10. Esteps von 690 auf E686,5 gesetzt)
M566 X400.00 Y400.00 Z6.00 E300.00         ; set maximum instantaneous speed changes (mm/min) ;LDO Orbiter 2.0
M203 X10800.00 Y10800.00 Z1000.00 E7200.00 ; set maximum speeds (mm/min) ; LDO Orbiter 2.0
M201 X12000.00 Y12000.00 Z150.00 E10000.00 ; set accelerations (mm/s^2)

If you can increase the speed of the Z axis, particularly the M566 instantaneous speed change, that may help.

Ian

Chrashem

@droftarts
this is also a possibility. But on the other hand I could observe this jerky behaviour during a plane xy movement which is included in my homeall macro. During this there should be no Z movement.

But raising the speed is in general a good hint. I will try this. But I will also check the mechanical side of the gantry system. If there is something wrong which is causing the issue, i guess its better to fix this

Phaedrux

@Chrashem said in After Firmware update from 3.4.5 to 3.5.2 my printer crashes:

When I my move the printhead by hand it seems that there a some points with a lot of friction.

I'd be looking for a bad bearing, uneven belt tension, or skew in the gantry.

Chrashem

@droftarts @Phaedrux
at the moment I learned a lot about my printer....
...unfortunately on the hard way...

From the movement side now it seems okay. In the mechanic I found a bit of friction caused by skew in the gantry. I could fix it by aligning the gantry. The movements by manually pushing the print head were fluent and smooth. But printing a flat surface showed again a bit of jerking.
So as @droftarts mentioned I raise up the z-speed changes. (Maybe I carried it to far, because I raised also the speed and the acceleration)

M566 X400.00 Y400.00 Z50.00 E300.00 
M203 X10800.00 Y10800.00 Z2000.00 E7200.00
M201 X10000.00 Y10000.00 Z300.00 E10000.00

The first test was fine! I Printed two filled layers and it seems that there is no XY-Jerking anymore and the surface quality was also looking good.

But unfortunately a bit later my printer crashed again...

My printer was homed and I started a simulation of a g-code.
This g-code seems to work because its the test part with the surface, I printed before... I got the message that the simulation was done successfully.
After this I tried to do a bed levelling by G32 and at this moment the printhead didn't move in a xy direction and the z axis was just lifting the bed without stopping against the printhead... Like the crash in my first post here...
Unfortunately the z-speed was very high, but fortunately the printhead was on a position where the bed could tilt, so there is no heavy damage. Just one piece from the z-axis is broken and I can replace it.

It feels like the root of evil is anywhere in the combination of the simulation and the G32....
I thought, that I got maybe a state where the printer isn't homed and no x and y movements are allowed? But in this case I thought that the z axis should stop when the probe is triggering the bed... (and the probe is working).

Is there a possibility to do a kind of a simple snapshot of the OM before and after the simulation?

For more testing I will try to reduce the travel speed during the probing and raise the diving height (just to get generating more time to react):

M558 P5 C"io6.in" H15 F300 T1000

Hopefully I can test more tomorrow, but first I have to replace the broken part of the z-Axis.

An other point which I noticed:
Is it possible, that the connection to DWC is more unstable? I recognised sometimes that the browser is generating a new connection to DWC.
With the old firmware I never got this problem. The position of the printer is the same, my laptop is the same and the router is also the same ( router and printer in the same room, distance just 3m away from each other...)

EDIT:
I could confirm it:
If I using a "G32" after a simulation (it doesn't matter if its a successful simulation or an failed one) my printer isn't moving the print head, but try to lifts the bed up till the crash.
I did several tests to get a better clue about it. After a simulation, DWC is reporting full movable axis. I could use the Buttons to move the axis and the printer is doing it fine. But if I click the bed levelling (G32) the printer is just lift the bed and would crash.
During this it doesn't matters which position my z-axis have. For example: If I have the probing height h set to 25 mm and the actual z position is 5 mm, normally the printer would gain the space to the probing height.
After the Simulation the printer is just lifting the bed...

If I do a homing of all axis after the simulation, and do then a G32 the printer is doing a fine levelling of the bed.
But if I don't do a full homing and do a single homing the behaviour depends on which axis I homed:

Simulation and homing x-axis: Lifting the bed -->crash
Simulation and homing y-axis: Lifting the bed -->crash
Simulation and homing x and y axis: Lifting the bed --> crash
Simulation and homing the z axis --> its just moving the y axis to the probing position (not the x axis) and tries to probe there. In my case the probe was not on the bed so I had to stop the printer manually to prevent a crash
Simulation and homing x y z axis --> G32 worked perfectly

Phaedrux

Can you enable some additional logging on the SBC to share here?

https://github.com/Duet3D/DuetSoftwareFramework/wiki/SBC-Setup-Guide#increasing-log-level

Chrashem

@Phaedrux
For sure!
I set up the logging. What movements or commands should I do with the logging on?

What do you want to see?

For the crash situation I have the problem, that I have to cut the power supply immediately I recognised, that the printer is out of control. I don't know if the logging will work under these conditions.

Chrashem

@Phaedrux @droftarts

At the moment I'm at home and I can run some tests for you with the logging. Can you tell me which movements do you want to see with the additional logging?

How can I share the log? Should I copy the text?

Phaedrux

Can you reliably make the printer crash with certain movements? We just want to see if anything comes up in the logging.

Another thing to test would be to setup in standalone mode without the SBC and see if the issue remains.

Chrashem

@Phaedrux
Today I run the test and I have the logging of it.

2024_08_23_Logging_Error_G32.txt

Just an additional Information:
To be sure that I got no crash but don't have to cut the power supply, I raised the dive height and lowered the feed rate to generate some time to react.

Unfortunately the error is a bit different. Normally the bed crashed against the printhead, but with this configuration I didn't get a crash, but I got a strange feedback:

Here are the steps which I did in the test:

16:02 - Restart the Machine
16:02 - Home all Axis
16:05 - G32 to level the Bed --> Successful -->Leadscrew adjustments made: 0.152 0.166 0.166, points used 3, (mean, deviation) before (0.161, 0.006) after (-0.000, 0.000)
See here (answer is late because the movement took some time in reason of the low feed rate):

Aug 23 16:09:52 Duet3 DuetControlServer[7125]: [debug] HTTP: Finished code G30 P2 X275 Y12 Z-99999 S3 ;probe near a leadscrew and calibrate 3 motors => Leadscrew adjustments made: 0.152 0.166 0.166, points used 3, (mean, deviation) before (0.161, 0.006) after (-0.000, 0.000)

16:10 simulation of a job
16:11 G32 to level the Bed again --> Error happened

Aug 23 16:16:12 Duet3 DuetControlServer[7125]: [debug] HTTP: Finished code G30 P2 X275 Y12 Z-99999 S3 ;probe near a leadscrew and calibrate 3 motors => Error: Some computed corrections exceed configured limit of 5.00mm: -14.332 -14.352 -14.354

Phaedrux

@Chrashem said in After Firmware update from 3.4.5 to 3.5.2 my printer crashes:

Error: Some computed corrections exceed configured limit of 5.00mm: -14.332 -14.352 -14.354

This error is due to the requested corrections being more than you have allowed in config.g.

Now the question is, is this actually a correct amount of correction? And if not, why?

If you alter the correction limit from 5mm to 15mm, does the correction actually take place? Is it always a similar amount?

Phaedrux

@Chrashem said in After Firmware update from 3.4.5 to 3.5.2 my printer crashes:

2024_08_23_Logging_Error_G32.txt

@chrishamm Can you take a look through these DCS logs?

Chrashem

@Phaedrux said in After Firmware update from 3.4.5 to 3.5.2 my printer crashes:

This error is due to the requested corrections being more than you have allowed in config.g.

Now the question is, is this actually a correct amount of correction? And if not, why?

This is absolutely not the correct amount of correction. I tried to showed this with the first bed levelling before the simulation! I can do hundreds of bed levellings and they are okay. After each step the corrections are getting smaller and the bed is perfectly levelled.

If I do one simulation and I do a G32 the software tells me this high amounts. Yesterday I tried three times the G32 after a simulation and every time I get this high amounts which are higher than the allowed limit. So in general you can think that the bed is not levelled. BUT if I do after this a homing of the axes and try again a levelling, the results are very small (like the ones without the simulation).

If you alter the correction limit from 5mm to 15mm, does the correction actually take place? Is it always a similar amount?

To be honest, I don't like to try it. Because the bed is so near to the printer head, it would crash!

After a simulation it feels like the software have lost internally the height of the bed. Because if I lower the dive height of the probe, I get after the simulation the behaviour that the printer just pushes the bed against the printhead without take notice of the probe and the axis limits. The printer pushes the bed against the bearings of the z axis and their housings are going to break.

So in my opinion there is something wrong after a simulation. And it doesn't matter which part I simulate. I can print these parts without an issue.

chrishamm

@Phaedrux i'm pretty sure this is an issue in the firmware so probably one for @dc42. Unfortunately, my machine which has independent Z drives isn't operational at this point so I can't attempt to reproduce it at the moment.

cosmowave

@Chrashem
A help to prevent from damage could be to reduce your motor currents during probing, if your hardware allows it.
I do that on my delta.
I cache the current in a variable before reducing it. After probing, i set the current back like in at was in config.g

I use the following code in bed.g before the probing starts:

; cache actual stepper currents, set lower current for probing
var Xcurrent = move.axes[0].current 						; cache X stepper current
var Ycurrent = move.axes[1].current 						; cache Y stepper current
var Zcurrent = move.axes[2].current 						; cache Z stepper current
M906 X400 Y400 Z400										; less motor current for probing
...
...
...
M906 X{var.Xcurrent} Y{var.Ycurrent} Z{var.Zcurrent}		; set motorcurrent back to value before probing

Chrashem

@cosmowave

This is a really good point! Thank you! I will adapt this to my code! And I think I will also adapt this to all of my homing files.

dc42

@Chrashem it sounds to me that the Z position may not be getting restored correctly after running a simulation. Until I can fix this, a workaround may be to home Z at the start of your bed.g file.

Chrashem

@dc42

thx for your message! You're right, and i do a full homing if do a simulation. Maybe it's helpful for you, but I also figured out, that I can also run into this issue if I abort a print. It's not only restricted to the simulation.