Verifying Firmware Safety Features - Duet 3 Mini 5+

flobler

Hello everyone

We are getting ready for our first release, supporting the Duet 3 Mini 5+ for the Bear Project. We always verify all firmware safety features before releasing to our users (we do it before every Prusa firmware release).

All tests were performed with:

Duet 3 Mini 5+ v1.0 WiFi and v0.5 Ethernet
RRF 3.3
DWC 3.3.0
PanelDue firmware 3.2.11

I was running through all the checks we are used to from Prusa/Marlin. Here you can see how we are testing for the different errors below: Checking Firmware Safety

Min Temp Error for Hotend: Min Temp Error
Min Temp Error for Bed: Min Temp Error
Thermal Runaway Hotend: Thermal Runaway Hotend
Thermal Runaway Bed: Thermal Runaway Bed
Preheat Error Hotend: Preheat Error Hotend
Preheat Error Bed: Preheat Error Bed

While running through those tests, I discovered a couple of things I wanted to ask about:

Whenever an error kicks in and the heater state switches to "fault", the relevant heater stops heating as expected (bed or hotend, depending on the error). Are the following two behaviors expected?

the "active" and "passive" heater temperatures are not being reset to 0C
if the hotend heater throws an error and switches to "fault", the bed heater keeps heating and vice versa if there is a bed heater "fault" the hotend keeps heating

I do not get an Error for the "Preheat Error Bed" described above. I am trying to heat the bed without a bed heater being connected and even after 10 minutes, there is no error triggered even though the bed is not heating. This works within seconds for the hotend. I should mention that the bed has been PID tuned beforehand, without tuning the error („Error: Heater 0 fault: temperature rising much more slowly than expected 1.2C/sec“) shows up after a few seconds. Is this expected?

Thanks for any input/help with this

dc42

@flobler said in Verifying Firmware Safety Features - Duet 3 Mini 5+:

While running through those tests, I discovered a couple of things I wanted to ask about:

Whenever an error kicks in and the heater state switches to "fault", the relevant heater stops heating as expected (bed or hotend, depending on the error). Are the following two behaviors expected?

the "active" and "passive" heater temperatures are not being reset to 0C
if the hotend heater throws an error and switches to "fault", the bed heater keeps heating and vice versa if there is a bed heater "fault" the hotend keeps heating

That is expected. When a heater fault occurs, you are given some time (default 10 minutes, configurable using M570) to reset the heater fault and continue the print if it is spurious. If you do not respond during that time, all heaters are turned off.

I do not get an Error for the "Preheat Error Bed" described above. I am trying to heat the bed without a bed heater being connected and even after 10 minutes, there is no error triggered even though the bed is not heating. This works within seconds for the hotend. I should mention that the bed has been PID tuned beforehand, without tuning the error („Error: Heater 0 fault: temperature rising much more slowly than expected 1.2C/sec“) shows up after a few seconds. Is this expected?

A normal bed heater should go into the fault state with a "temperature rising more slowly than expected" error. It will take twice the dead time (as specified by M307) for the heater fault to be raised. However, if tuning has found the bed heater to have very low power so that the maximum temperature rise above ambient at full power is less than 34C, then the check is omitted.

flobler

@dc42 thank you for your quick response, I really appreciate it! Based on your feedback I did a couple more tests:

the "active" and "passive" heater temperatures are not being reset to 0C
if the hotend heater throws an error and switches to "fault", the bed heater keeps heating and vice versa if there is a bed heater "fault" the hotend keeps heating

That is expected. When a heater fault occurs, you are given some time (default 10 minutes, configurable using M570) to reset the heater fault and continue the print if it is spurious. If you do not respond during that time, all heaters are turned off.

I re-run the following tests:

Min Temp Error for Hotend: Min Temp Error
Min Temp Error for Bed: Min Temp Error

In both cases, once the error for the tested heater kicked in and the corresponding heater fault occurred, I waited for 20 minutes but the board did not turn off all heaters as it would be expected. When I tested the bed heater, the hotend heater was still on after 20 minutes and vice versa. I did not specify anything with M570, here is my config for reference:

; Configuration file for Duet 3 Mini 5+ (firmware version 3)
; executed by the firmware on start-up

; General preferences
G90                                                   ; absolute positioning
M83                                                   ; set extruder to relative mode
M550 P"duet-bear"                                     ; set printer name

; Network
M552 S1                                               ; enable network and acquire dynamic address via DHCP
M586 P0 S1                                            ; enable HTTP
M586 P1 S1                                            ; enable FTP
M586 P2 S0                                            ; disable Telnet

; Drives
M569 P0.0 V10 S1                                      ; physical drive 0.0 goes forwards  / Z-left  / reduced V parameter to enable stealthChop
M569 P0.1 V10 S1                                      ; physical drive 0.1 goes forwards  / Z-right / reduced V parameter to enable stealthChop
M569 P0.2 S0                                          ; physical drive 0.2 goes backwards / Y 
M569 P0.3 S1                                          ; physical drive 0.3 goes forwards  / X
M569 P0.4 S1                                          ; physical drive 0.4 goes forwards  / E
;
M584 X0.3 Y0.2 Z0.0:0.1 E0.4                          ; set drive mapping
;
M671 X-36.5:293.5 Y0:0                                ; define lead screw positions
;
M350 X16 Y16 Z16 E32 I1                               ; configure microstepping with interpolation
M92 X100.00 Y100.00 Z400.00 E280.00                   ; set steps per mm
;
M566 X480.00 Y480.00 Z48.00 E300.00                   ; set maximum instantaneous speed changes (mm/min)
M203 X24000.00 Y24000.00 Z1200.00 E6000.00            ; set maximum speeds (mm/min)
M201 X800.00 Y800.00 Z100.00 E300.00                  ; set accelerations (mm/s^2)
;
M906 X500 Y500 Z450 E550 I40                          ; set motor currents (mA) and motor idle factor in per cent
;
M84 S60                                               ; Set idle timeout
M564 H0                                               ; allow unhomed movement

; Axis Limits
M208 X0 Y-4 Z0 S1                                     ; set axis minima
M208 X250 Y212.5 Z210 S0                              ; set axis maxima

; Endstops
M574 X1 S3                                            ; configure sensorless endstop for low end on X
M574 Y1 S3                                            ; configure sensorless endstop for low end on Y
M915 X S3 F0 H200 R0                                  ; set the X axis sensitivity
M915 Y S3 F0 H200 R0                                  ; set the Y axis sensitivity
M591 D0 P2 C"io3.in" S1                               ; configure endstop for extruder drive 0 on pin io3.in and turn it on (filament sensor)

; Z-Probe
M558 P5 C"^io2.in" H5 F500 T10000 A1                  ; set Z probe type to switch, dive height, dive speeds, travel speeds and number of probes 
G31 P1000 X23 Y5 Z0                                   ; set Z probe trigger value, offset and trigger height
M557 X24:228 Y6:208 P7:7                              ; define mesh grid

; Heaters
M308 S0 P"temp0" Y"thermistor" T100000 B4725 C7.06e-8 A"Bed"    ; configure sensor 0 as thermistor on pin temp0
M950 H0 C"out0" Q50 T0                                          ; create bed heater output on out0 and map it to sensor 0, PWM frequency 50Hz
M307 H0 B0 S1.00                                                ; disable bang-bang mode for the bed heater and set PWM limit
M140 H0                                                         ; map heated bed to heater 0
M143 H0 S110                                                    ; set temperature limit for heater 0 to 110C
;
M308 S1 P"temp1" Y"thermistor" T100000 B4725 C7.06e-8 A"Nozzle" ; configure sensor 1 as thermistor on pin temp1
M950 H1 C"out1" T1                                              ; create nozzle heater output on out1 and map it to sensor 1
M307 H1 B0 S1.00                                                ; disable bang-bang mode for heater and set PWM limit
M143 H1 S300                                                    ; set temperature limit for heater 1 to 300C

; Fans
M950 F0 C"out5" Q500                                  ; create fan 0 on pin out5 and set its frequency
M106 P0 C"Part Cooling" S0 H-1                        ; set fan 0 name and value. Thermostatic control is turned off
;
M950 F1 C"!out3" Q25000                               ; create fan 1 on pin !out3 and set its frequency
M106 P1 C"Hotend" S1 H1 T45                           ; set fan 1 name and value. Thermostatic control is turned on
;
M950 F2 C"out6" Q500                                  ; create fan 2 on pin out6 and set its frequency
M106 P2 C"Case" S1 H1 T45                             ; set fan 2 name and value. Thermostatic control is turned on

; Tools
M563 P0 D0 H1 F0                                      ; define tool 0
G10 P0 X0 Y0 Z0                                       ; set tool 0 axis offsets
G10 P0 R0 S0                                          ; set initial tool 0 active and standby temperatures to 0C
M302 S180 R180                                        ; allow extrusion and retraction starting at 180°C

; Miscellaneous
M575 P1 S1 B57600                                     ; enable support for PanelDue
M501                                                  ; load saved parameters from non-volatile memory
T0                                                    ; select tool 0

Do I have anything configured wrong for the reset to work?

I do not get an Error for the "Preheat Error Bed" described above. I am trying to heat the bed without a bed heater being connected and even after 10 minutes, there is no error triggered even though the bed is not heating. This works within seconds for the hotend. I should mention that the bed has been PID tuned beforehand, without tuning the error („Error: Heater 0 fault: temperature rising much more slowly than expected 1.2C/sec“) shows up after a few seconds. Is this expected?

A normal bed heater should go into the fault state with a "temperature rising more slowly than expected" error. It will take twice the dead time (as specified by M307) for the heater fault to be raised. However, if tuning has found the bed heater to have very low power so that the maximum temperature rise above ambient at full power is less than 34C, then the check is omitted.

For the bed heater I am using the stock Prusa MK3 heater: https://shop.prusa3d.com/de/mk3mk3smk3s/198-magnetic-heatbed-mk52-24v-assembly.html

Here is my PID auto-tuning results for the bed:

M307 H0 R0.302 C72.487:72.487 D10.44 S1.00 V24.2 B0 I0

I am not sure if I correctly understand your feedback though. Would this qualify as a "very low power heater"?

Thank you for your help!

dc42

@flobler I have added this to my list of investigations and tests to do. I expect to do it later this week or early next week.

flobler

@dc42 Thank you! I am looking forward to the results. If you need anything, please do not hesitate to reach out to me directly or ping me here

flobler

@dc42 said in Verifying Firmware Safety Features - Duet 3 Mini 5+:

@flobler I have added this to my list of investigations and tests to do. I expect to do it later this week or early next week.

I just wanted to verify if you had the time to further investigate

dc42

@flobler I'm sorry, this is still on my list to look at but I haven't had time yet. I will not have a chance to look at it until next week.

From your report I understand that the heater fault detection is working and turns the heater off, but the secondary action of cancelling the print and trying to turn off power to the Duet if there is no user intervention isn't working on your system.

jay_s_uk

@dc42 on my duet 3 mini, I use a separate 12v PSU for the bed.
I have on more than on occasion started a print without turning it on. The printer sits there waiting for the bed to heat but never faults.
I'm running 3.3 + SBC

flobler

@dc42 No problem at all

From your report I understand that the heater fault detection is working and turns the heater off, but the secondary action of cancelling the print and trying to turn off power to the Duet if there is no user intervention isn't working on your system.

Correct.

I have on more than on occasion started a print without turning it on. The printer sits there waiting for the bed to heat but never faults.
I'm running 3.3 + SBC

This seems to match the second question I had if I understand correctly.

flobler

@dc42 said in Verifying Firmware Safety Features - Duet 3 Mini 5+:

@flobler I'm sorry, this is still on my list to look at but I haven't had time yet. I will not have a chance to look at it until next week.

Just a short ping to see if there is an update @dc42

gregsaun

@dc42 Just a little bump to see if there is an update for our problem. Thank you in advance for the help

Phaedrux

@flobler @gregsaun In this situation is the print happening from local SD card, or from a USB gcode sender like repetier/octo, etc?

Phaedrux

@jay_s_uk said in Verifying Firmware Safety Features - Duet 3 Mini 5+:

@dc42 on my duet 3 mini, I use a separate 12v PSU for the bed.
I have on more than on occasion started a print without turning it on. The printer sits there waiting for the bed to heat but never faults.
I'm running 3.3 + SBC

the fault detection for initial heating expects the heating rate to be within the expected value less a tolerance. If the expected value is very low then it may be less than the tolerance; but in that case the heater is probably intrinsically safe anyway.
but that's not the same issue as in the OP anyway, because in the OP the heater does fault

flobler

@phaedrux no print at all in this case. I am just testing the safety features while I am standing in front of the printer. So hotend and bed temps are set from the PanelDue.

This guide explains what I am doing:
https://guides.bear-lab.com/Guide/Checking+Firmware+Safety/25?lang=en

My initial post describes what safety features we were hoping to verify.

Phaedrux

@flobler Ok, this may explain the issue then. The timeout will only be active when the heater is running during a print from a file on the SD card.

Snnn Integer timeout in minutes (can be set to 0) for print to be cancelled after heater fault (Firmware 1.20 and later). If the S parameter timeout occurs (which only happens if a SD print is in progress), RRF will also try to turn off power via the PS_ON pin.

https://duet3d.dozuki.com/Wiki/Gcode?revisionid=HEAD#Section_M570_Configure_heater_fault_detection

flobler

@phaedrux thanks, I will re-try the tests with an active print!

the "active" and "passive" heater temperatures are not being reset to 0C
if the hotend heater throws an error and switches to "fault", the bed heater keeps heating and vice versa if there is a bed heater "fault" the hotend keeps heating

That is expected. When a heater fault occurs, you are given some time (default 10 minutes, configurable using M570) to reset the heater fault and continue the print if it is spurious. If you do not respond during that time, all heaters are turned off.

I assume this means that heaters are only reset after a heater fault with the default time of 10 minutes if there is an ongoing print from SD card, right?

One more comment: If I disconnect the heater from the hotend and try to heat it, the heater fault is being triggered after a few seconds even without a print from the SD card and without having defined the fault detection via M570 in config.g. When I try to do the same with the bed the error is not being triggered. Is there a reason why hotend and bed are behaving differently?

I will do some tests this week and report back after.

Thanks!

dc42

@flobler said in Verifying Firmware Safety Features - Duet 3 Mini 5+:

One more comment: If I disconnect the heater from the hotend and try to heat it, the heater fault is being triggered after a few seconds even without a print from the SD card and without having defined the fault detection via M570 in config.g. When I try to do the same with the bed the error is not being triggered. Is there a reason why hotend and bed are behaving differently?

From my previous response:

A normal bed heater should go into the fault state with a "temperature rising more slowly than expected" error. It will take twice the dead time (as specified by M307) for the heater fault to be raised. However, if tuning has found the bed heater to have very low power so that the maximum temperature rise above ambient at full power is less than 34C, then the check is omitted.

Could that be the reason?

flobler

@dc42 thanks for your response. I am not sure what you would classify as "normal bed heater". I am using a stock Prusa MK3 bed.

https://shop.prusa3d.com/en/mk3mk3smk3s/198-magnetic-heatbed-mk52-24v-assembly.html

I doubt this would qualify as a "very low power" heater and the maximum temperature difference at full power should be significantly higher than 34C. Am I misunderstanding maybe? Is there a way for me to find out if the tuning procedure has found it to be a low powered heater?

Also to double check I am understanding everything correctly:

The heater fault (when no heater is connected but the printer is trying to heat) should be triggered anytime, even wihout an ongoing print for both hotend and bed heaters?
The timout which cuts off all temperatures after the fault was triggered only kicks in if there is an ongoing print from SD?

Thank you!

dc42

@flobler said in Verifying Firmware Safety Features - Duet 3 Mini 5+:

Is there a way for me to find out if the tuning procedure has found it to be a low powered heater?

Yes, send M307 H0 (assuming that heater 0 is the bed heater) and tell us what parameters it reports. The product of the R and first C value is the estimated maximum temperature rise above ambient.

Also to double check I am understanding everything correctly:

The heater fault (when no heater is connected but the printer is trying to heat) should be triggered anytime, even wihout an ongoing print for both hotend and bed heaters?

Yes, unless the expected rate of temperature rise is very small, either because the estimated maximum temperature rise above toom temperature is very low (<= 34C) or the heater is already hot.

The timout which cuts off all temperatures after the fault was triggered only kicks in if there is an ongoing print from SD?

Correct. When printing from SD card, after a heater fault is raised, if the user doesn't reset it within the timeout period then the system performs an emergency stop. It also tries to turn off power via the PS_ON pin, as if M81 had been commanded.

flobler

@dc42 thanks again for the quick reply!

Here is the output:

Heater 0 model: heating rate 0.302, cooling time constant 72.5, dead time 10.44, max PWM 1.00, calibration voltage 24.2, mode PID
Computed PID parameters: setpoint change: P56.6, I2.058, D413.7, load change: P56.6, I2.930, D413.7

That does indeed indicate it is considered to be a low powered heater if I understand you correctly.

What is interesting though is, that my other printer which I finished building later and which I have no tested the heater fault with (same bed is used there) has a significantly higher value for C

Heater 0 model: heating rate 0.279, cooling time constant 456.0, dead time 11.53, max PWM 1.00, calibration voltage 23.6, mode PID
Computed PID parameters: setpoint change: P55.5, I0.765, D447.8, load change: P55.5, I1.683, D447.8

It looks like I should re-run the PID routine and re-test. I will do that and report back.