RepRapFirmware 3.6.0-alpha.2 for Duet main boards available

dc42

Over the past few weeks we have been running 3.6.0 Alpha1 and more recently Alpha 2 versions of RepRapFirmware internally, and we have also provided these versions to a few external users. We are very encouraged by the feedback we have received and I believe it is time to make 3.6.0-alpha.2 more widely available. Accordingly I have provided the firmware binaries for Duet main boards at https://www.dropbox.com/scl/fo/vljgygfz32kifka9o9qgp/AMqQAbfBdFyP3a797YjpAKg?rlkey=bala1kdb78x1zwyrkqyhbjiem&dl=0. You can ignore the .map files at that location.

The main feature of RRF 3.6.0 the new input shaping algorithm, which is capable of shaping moves no matter how short they are; so this release will be of particular interest to those users of RRF who still experience ringing on prints.

If you consider trying this version, be sure to to read the release notes at https://github.com/Duet3D/RepRapFirmware/wiki/Changelog-RRF-3.x-Beta#reprapfirmware-360-alpha2 first.

We intend to make this version available to users running in SBC mode in the near future.

I welcome general feedback on user experiences with 3.6.0-alpha.2 in this thread; but if you encounter specific issues, please report them in a new thread and include [3.6.0-alpha.2] in the thread title.

I am particularly interested in:

Whether users get any unexpected halts while printing;
How much Never Used RAM is reported by M122 after doing a print (preferably compared with the same value when RRF 3.5.2 is used).

Notepad

@dc42 This is probably the most excited I have been all year. I have been able to spend a good 5 hours with 3.6.A2 and it makes me so giddy.

The most important thing ignoring the quirks of alpha-2 is the input shaper is a huge quality improvement. The following image shows 3.6.A2 (Left) and 3.5.2 (Right) printed using identical print settings.

This is an absolutely phenomenal improvement to the input shaper efficacy, and I dare say I even think the pressure advance seems to be more effective.

Now onto the debugging as there is a few issues I have tried to get some logs for.

The machine used to do all this testing is a heavy version of the Construct 1 XL, and will not see any hardware changes for any tests to isolate only firmware quality improvements.
The main hardware is as follows:

Duet 2 (Wifi), Hemera XS, CHT Volcano (brass), CoreXY kinematics.
All print jobs are done with the same gcode file which includes a 3D benchy, Orca cube, and Orca cube threaded cap. All models are scaled to 125%.
All accelerations are set to 5000mm/s^2 and jerk values are set to 10mm/s. These settings are specifically chosen as these create the worst quality prints possible with 3.5.2 (higher jerk or accelerations surprisingly improves print quality) and any slicer tweaks that could improve print quality has been disabled in favour of making the worst quality prints possible.
ArcWelder is disabled
print file: 3DBenchy36alpha2noarc_PLA_58m24s.gcode
Config.G: config (2).g

The first thing noticeable when printing with 3.6.A2 is at high speeds the motors audibly 'chirp'. https://youtu.be/7ICvUHX-By8?si=9LWsEbJRurqIOF06
At the 14 second mark is when the chirping is most obvious. This tends to be only in arcing movements.
For the first print only, Arcwelder was enabled and this did make the issue worse, however I do not believe its the actual G2 or G3 commands causing the issue, but probably the dense segmentation/step instructions of arc commands causing the IS to freak out slightly. This also seems coupled with print speed, as the first layer which is printed at a respectable 60mm/s prints fine, however print moves above 140mm/s start to excite the chirping greatly and past 220mm/s tend to be the hard limit before the machine encounters a fault.
It is surprisingly hard to determine if the chirping noise is being generated by the XY motors, or by the vibration in the print head. I know the IS doesn't affect the extruder motor itself however I am very much struggling to isolate the noise location, even when putting my ear to the XY motors.

After a few layers, the printer will throw a Code 3 error;

Error: Movement halted because a step timing error occurred (code 3). Please reset the controller

I have been able to get quite a nice diagnostic log with the following process.

Power cycle the printer
YAT USB connection
M122 as a baseline diagnostic
m111 p4 d1 b2048
Print till failiure
M122 as a final diagnostic log
Estop
Save diaglog
Diagnostic log: Duet 2 3.6.alpha2 benchy test run no arcweld debuglog alternate 2.txt

With further testing I have identified:

Pressure Advance does not affect the behaviour. tested 3 times with values 0.00(off), 0.15, 0.30
Input shaper type does not affect the audible chirping intensity, frequency or what moves it occurs on. Tested with Ei2, MZV and ZVD
Input shaper dampening value does not affect the chirping. Tested with S0.0 and S0.05, S0.1
Jerk does affect the chirping considerably. Reducing the jerk from 10 to 6 actually lets prints finish instead of failing within a few dozen layers. Though the chirping is still present, the magnitude tends to be less. I have been able to print an orca cube (previously pictured) and a 3Dbenchy, which shows on curved surfaces the chirping artefacts, but elsewhere on flatter surfaces it looks beautiful.
Z-Jerk and Z-Acceleration does not appear to affect the chirping. leading me to believe there is no interaction between the bed mesh auto leveling and the XY movement being a factor.

Additional diagnostic logs:
Duet 2 3.6.alpha2 benchy test run no arcweld debuglog.txt
Duet 2 3.6.alpha2 benchy test run no arcweld debuglog alternate.txt
Slightly less helpful longer form (2min) YouTube vid of failure: https://youtu.be/N6iOGXx1eeQ?si=WyFaEtqR06-vxfjG

My next testing steps will be:

to source some TMC2660s for use in an alternative motherboard such as the Mellow super5pro. doing so hopefully will make the only difference the processor or ram capacity (ATSAM4E8E vs STM32H723)
TMC2240's and TMC2209 to see if there is driver dependent behaviour.
TMC2209s on Duet3Mini5+ vs Super5Pro
Further jerk testing
Further acceleration testing
Check for Slicer level options that could trigger the chirping

Hopefully any of this is helpful. If there is any particular tests you require I will happily send over anything I can.
Also a massive thankyou to @gloomyandy and @jay_s_uk for assisting in the diagnostic process and basically holding my hand whilst making sure I cover all bases.

oliof

@Notepad interesting results. I found my test machine (RatRig V-Minion v1.1 with dual rail Z mod) to be more pleasant sound wise than with v3.5.2. I observe the same quality improvements as you, but ran into layer shift issues in Y on smaller round prints. I haven't been able to debug as methodically as you so I can't say yet what the cause is (mechanical or firmware).

Sharing a video with some hardcore Klipper user I was told they thought the machine was running in stealth chop but M569 confirms I'm using spread cycle...

dc42

@Notepad thanks for your extensive feedback.

Unfortunately the debug output when hiccups are inserted has prevented additional debug output when the step error is raised. Please can you re-run the failing print using the Duet 2 firmware binary at https://www.dropbox.com/scl/fo/9wo03jbngipb1fxrsff5s/AGD0hZnFbBpBtA0ZLSDUNBE?rlkey=r5kv33ud8fgg2an38n6k68qp8&dl=0 using the same debug settings as before, and report the output here.

Notepad

@dc42 Im running the new tests now...did something else change for 3.6.A2+1? The chirping is noticeably quieter and doesn't sound like it will fail?

Exerqtor

Oh this is something I've been looking forrward to for a good while! 😮

I'll give this baby a go later today 😁

Notepad

@dc42 I have done a single full run with the new 3.6.A2+1 and the print completed successfully. The quality is fantastic and the chirping is basically gone.

Config.g: config (1).g
Diaglog: Duet 2 3.6.a2+1 no arc weld x64microsteps.txt

Whatever has changed between 3.6.A2 and 3.6.A2+1 has made a huge difference. Do you want me to continue testing with 3.6.A2+1 or return to the previous one for further testing?

dc42

@Notepad the only change in the +1 version that should have any effect on motion is that I turned off the debug output that is generated when a hiccup is inserted. So perhaps that code was slowing down the step ISR to the extent that hiccups were extended excessively. I note that in your latest diag report the cumulative hiccup time was 443ms as against 1522ms for the original one.

Your filename suggests that you are using x64 microstepping. Have you tried using x16 with interpolation instead? This is likely to reduce the number of hiccups. The only case in which we advise using greater than x16 microstepping is on extruders with very low steps/mm (e.g. around 100 or lower).

Please continue your testing using the +1 version.

dc42

I have now provided slightly updated RRF 3.6.0-alpha.2+2 binaries at https://www.dropbox.com/scl/fo/9wo03jbngipb1fxrsff5s/AGD0hZnFbBpBtA0ZLSDUNBE?rlkey=r5kv33ud8fgg2an38n6k68qp8&dl=0. Also there is a 3.6.0.alpha DWC,

Notepad

New batch of testing complete with pretty identical results, unfortunately I have not completed the whole batch I wanted to complete today due to time constraints.

Testing was done using 3.6.A2+1:
X16 with interpolation: Duet 2 3.6.a2+1 no arc weld x16microstepswithinterpolate.txt
X32: Duet 2 3.6.a2+1 no arc weld x32microsteps.txt
X64: Duet 2 3.6.a2+1 no arc weld x64microsteps.txt
X128: Debug was incorrectly saved, however it printed completely fine

X128 running 3.6.A2+2: Debug was incorrectly saved similar to before and printed identically to 3.6.A2+1.

The print quality is overall fantastic. No odd sounds or noticeable layer oddities. One thing that did break at some point is the communication to my equipped PanelDue. Warning/info popups still populate however no temperatures load and the device stays in the "connecting" state.

dc42

@Notepad thanks. Serial ports were not working in alpha2+2. I have replaced those binaries with alpha2+3 ones, still at https://www.dropbox.com/scl/fo/9wo03jbngipb1fxrsff5s/AGD0hZnFbBpBtA0ZLSDUNBE?rlkey=r5kv33ud8fgg2an38n6k68qp8&dl=0.

PS - hiccups did increase with microstepping, from 8 (0.24ms) at x16 to 14385 (443.83ms) at x64. Looks like the CPU is not really fast enough to caclulate the step times at x64.

Exerqtor

Just finished the test on 3.6.0-a2+2, and it's a visible improvement over 3.5.2 in terms of ringing when accelerating, BUT it's been introduced some artifacts during deceleration that kind of looks like underextrusion.

I've tried taking some pictures to show off what i'm talking about, the upper testprint is with 3.6.0-a2+2:
alt text

Edit:
This is being done with the following setup:

Hardware:

Duet 3 Mini 5+ v1.02.
1LC v1.2a - as toolboard for the extruder/hotend.
Revo mini hotend with 0.4mm highflow nozzle.
LGX-lite extruder:

FW/print settings:

Outer wall speed: 120mm/s.
Inner wall speed: 200mm/s.
Outer wall acceleration: 3000mm/s2.
Inner wall acceleration: 3000mm/s2.
Outer wall jerk: 7mm/s.
Inner wall jerk: 7mm/s.
Pressure advance: 0.045mm.
Retraction: 0.45mm.
Inner wall flow approx: 12.9mm3.
Outer wall flow apporx: 7.7mm3.
2 walls.
0% infill.
0 top layers.

On a little sidenote, I also noticed that the printer does a tiny "pause" or waits after each probe move, before when it did mesh leveling etc. it was much more "snappy" and things went faster. Now it takes some sweet time between probes. I've seen you've done several commits to remidy some probe related stuff during development so i suspect you're well aware of this allready but I just wanted to mention it.

Notepad

@Exerqtor Ive noticed my pressure advance seems to be more effective in 3.6.A2x. I have yet to do a PA calibration step yet to see how its been affected. It might be your value has changed.

Also when it comes to retraction/deretraction/PA, it might be worth just noting what extruder & nozzle setup you have + speeds, as well as what flowrate the behaviour is noticed at.
For me my prints are normally at 26mm^3/s out of a max flow of 32mm^3/s on a print with 5000mm/c^2Accel and 10jerk

Exerqtor

@Notepad
Added in printer & print info in the post above 😊

Did a new print with the exact same g-code & settings, just with PA disabled, and it sure did remove the underextrusion i was seeing (top is 3.6.0-a2+2 with PA off, bottom is 3.6.0-a2+3 with 0.045mm PA):

alt text

I'll try to crank up some acceleration and jerk later today and see how that plays out.

But I must say that the delay after a probe move is somewhat annoying lmao.

Notepad

New tranche of tests completed,

The first and most important is the new baseline for 3.6.A2+3
Duet 2 3.6.a2+3 no arc weld x16microsteps.txt
This completed without any issues, the output benchy is identical to A2+2 and the motor chirping is none existent (no chirping detected in A2+2 either). Thankfully the PanelDue is now functional so I can confirm the changes to the serial output has worked with no complications.

Further testing to no ones surprise shows x256 and x128 microstepping failed to complete, and failed in the same code 3 error as in A2+0
Duet 2 3.6.a2+3 no arc weld x128microsteps.txt
Duet 2 3.6.a2+3 no arc weld x256microsteps.txt
The motors sounded awful during this, and the prints failed within 10 layers of starting.

Interestingly, in A2+2 I was able to get a successful print out of x128 microstepping, which was not possible in A2+3. I wonder if the fixed serial connection put extra load onto the MCU and caused it to fail.

As I have confirmed that between A2+2 and A2+3 at x16 microsteps with interpolation are pretty much identical, I have resliced the test print file to increase accelerations from 5k->10k, and jerk values 10->14. The goal is to put as much strain on the MCU as possible to try and find any failure cases.

If the test print was made back in 3.5.2, this print would look absolutely awful and highly risk layer shifts. With A2+3 the print was basically flawless with an average print speed of 220mm/s. The motors also sounded slightly quieter when compared to 3.5.2, though I expect this is because of the smoother transitions in corners not exciting the frame instead of directly being motor noise.

Duet 2 3.6.a2+3 no arc weld x16microsteps 10kaccel14Jerk.txt

The last test I have completed is the same increased speeds as before, however with pressure advance disabled. I wanted to remove any possibility that PA could be slowing the machines XY movements down, and at the same time I can test if PA is doing anything (spoiler, it is working)

Duet 2 3.6.a2+3 no arc weld x16microsteps 10kaccel14Jerk noPA.txt

Both these higher speed / more aggressive prints performed flawlessly. there was no weird noises, and as you can see with the photos, Pressure advance seems to be working effectively.

My next test will be focused on finding a new PA value, as my gut feeling is PA has become more effective, thus my original PA value of 0.022 can be reduced.
After this I intend to focus on 16x microstepping and using alternative input shaping methods and cancelation frequencies to see if specific values (i.e ultra low frequencies with complex shapers) might cause issues.

Interestingly I have not noticed any z-probe hanging akin to @Exerqtor has mentioned, the behaviour of my bed and gantry levelling using an inductive probe feels identical to normal.

Notepad

After a quick investigation on PA, it does appear to be much more effective. Key word is 'appears'. Due to the new input shaper causing cornering artefacts (which from now on I will reference as smoothing) this aids in reducing the amount of pressure advance required. More on this later.

The main thing I have tested in this batch is the behaviour of the different input shapers available. I did so by cutting down the original print file into a smaller 10mm slice so the parts can be printed quicker. The slice was chosen to be at the bottom of the original file as this is where most print errors occurred, which coincides with the most amount of long arcing movements. Print speeds and aggression stayed the same.

Duet 2 3.6.a2+3 16xmicro 10kA 14J MZV 51hz.txt
Duet 2 3.6.a2+3 16xmicro 10kA 14J EI3 51hz.txt
Duet 2 3.6.a2+3 16xmicro 10kA 14J EI2 51hz.txt
Duet 2 3.6.a2+3 16xmicro 10kA 14J zvddd 51hz.txt
Duet 2 3.6.a2+3 16xmicro 10kA 14J zvd 51hz.txt

Over all, the prints performed exactly as expected, and all passed without any issues.
There was one print which did get a Code 3 error, and the main difference was I changed the input shaper type mid printer (after the first layer). This might be an edge case senario but it is interesting that the only time I manually change the input shaper type mid print via inserting custom gcode in the slicer, is the one time it fails. It is also interesting that it didnt fail instantly, it was able to process about 5 seconds more of the print job before the print failed on one of the curved sections.
Duet 2 3.6.a2+3 16xmicro 10kA 14J MZV 51hz fail.txt

After this initial fail, I decided on updating the config.g itself and restarting the mainboard to be the best course of action.

Stacking the print jobs from previous, gives a great example of the input shaper aggression against the amount of smoothing which the input shaper applies.
In order from top to bottom:
MZV, ZVD, ZVDDD, EI2, EI3

The effect of the smoothing is very noticableon the benchy's rear text and the orca slicer logo.
This also leads me to the conclusion that pressure advance should be calibrated to the type of input shaper used, as more aggressive input shapers require lower values of PA for the same clarity.

I personally use EI2 as my go to shaper of choice. and at 51hz I was able to lower my PA value from 0.0225 to 0.0085, which in-turn also sped up my overall print speeds without sacrificing any quality.

One final test I completed (no pictures sadly) is using MZV on an extremely low frequency which would normally never be used in actual machines, in hopes to overload the MSU.
The output print quality was decent, however showed massive amount of smoothing almost 10x worse than EI3 in the previous tests.
This leads me to the conclusion that the input shaper should only be used on frequencies above 40hz, and the lower the frequency you go, the simpler the input shaper should be used. I.E dont use EI3 over MZV at 30Hz.
Duet 2 3.6.a2+3 16xmicro 10kA 14J MZV 20hz.txt

I am starting to run out of things to test on my dedicated test machine, so I have selected a handful(8) of print farm machines which i have deployed 3.6.A2+3 on to. So far the output has been a marked improvement in print quality. No known errors or issues has appeared yet.

Any advice on specific areas I should focus my testing onto would be much appreciated. if not my next batch will be all the input shaper types at very low and very high frequencies.

o_lampe

@Notepad https://forum.duet3d.com/assets/uploads/files/1721499165580-p1086721.jpg

In this picture you can see the uppermost layers show a bit of meltdown on the Ei2 Y-part.
I blame it on very short layer time and the filament got too hot?
Wouldn't printing at too high temp also explain the smoothing?

Notepad

@o_lampe AH, I probably should have mentioned, in the Y section at the top, I use pliers to grip the printed model to pull it off my bed. Im too lazy to take the PEI off or let it cool down so when I grip it causes those indentations.

Exerqtor

I've done some more testing and tuning as well, both PA, Acceleration & Jerk has gotten some adjustments. This new approach to input shapiny is REALLY a huge improvement!

I'll upload some pictures once i get home, but i at least was able to shave down PA to 0.0075s from 0.0450s ,acceleration to 3000mm/s2 from 2500mm/s2 & upped the jerk to 7mm/s (~~i can't remember the Jerk values so i'll update those with the pics later~~). I also turned up the printspeed with 50mm/s across the board.

Here is the "PA-tower" results with a range from 0-0.05s, before i had to run 0.045s PA to get the same result as I get with 0.0072s now.
alt text

The printer sounds happier, and the surface quality in the prints is better than ever. Gread job @dc42 & and others who contributed!😁

Proschi3D

I have also tested the firmware and am very satisfied with the improved quality. I haven’t been able to make any fine adjustments yet, but I have increased the speed by 50mm/s as well.