Problem with daemon.g timing

[fcwilt]

@awitc

I don't think M226 is the command you want.

And where are the bits of code that exit the while loop?

Thanks.

Frederick

[awitc]

@fcwilt The M226 is maybe not the best, but it is needed for when I am stopping the print due to problems with a custom extruder (temps, pressure errors) or pausing it from gcode to wait for initial pressure to raise. In this post I was focused on the while true and G4 Px duo, where I want an infinite loop that checks several things ever x milliseconds. That is also why the loop doesn't ever exit. Also I remember to have read somewhere that without it the daemon.g executes slower, like every several seconds or so.

[fcwilt]

@awitc

Well the docs for M226 say it is to be used from within the GCODE file being printed:

Use M226 when a pause is required in the GCode file being printed, for example to pause after a particular layer has completed. It waits until all the moves in the queue have been completed.

I think M25 is the correct code but if the docs are correct it doesn't stop the print, just pauses it.

The daemon code is run every 10 seconds but in your case I have no idea what is going to happen since you don't allow the while loop to exit.

Perhaps @dc42 will notice and address your issues.

Frederick

[infiniteloop]

@awitc

have read somewhere that without [the while loop] the daemon.g executes slower, like every several seconds or so.

As a self-assigned expert for infinite loops, I can confirm that . But it is always better to study the documentation. For daemon.g, see the respective sections in Macros and GCode meta commands.

With M226, the Notes state:

Use M25 when a pause is required from a different source of GCodes (such as the web interface console, PanelDue or a Macro).

Which, in your case, is true. Looking up M25, however, it says:

Note that if a pause is commanded while a macro is being executed, the pause will be deferred until the macro has completed.

… which puzzles me a bit, because your macro - the daemon - will never run up to completion. So, the only advice I can give on this one is to test both variants. Then use whichever of them works for you.

Talking of tests: In order to spot the code lines where the daemon spends most of its time, you might want to comment-out single statements (or blocks of statements) temporarily. Then look whether the response times improve significantly.

Some of your code can certainly be optimised, but the M98 calls are my primary suspect: Every time a macro is invoked, it has to be fetched from the SD card (Note:.on every iteration of the loop!). You can either unravel the macro code into daemon.g or try a fresh, high quality (e.g. very fast) SD card.

[dc42]

@awitc as @infiniteloop says, executing a M98 command frequently in a loop should be avoided, in particular the one at line 70 in your file.

[awitc]

@fcwilt The idea of the while true loop was taken from this forum itself I think, to have it check faster than every 10 seconds. According to @dc42:

@dc42 said in daemon.g usage cases:

@OwenD said in daemon.g usage cases:

Does G4 pass control back to the main process when used in a loop?

Yes.

From this I understand that I effectively get a daemon.g running every G4 Px milliseconds at least (not taking into account the time it takes for the daemon to execute).

The choice between M226 and M25 was made because of this description:

Initiates a pause in the same way as if the pause button is pressed, except that execution of all prior GCode commands in the same input stream is completed first.

On the other hand M25:
M25 attempts to execute as quickly as possible

Even though the GCode dictionary mentions M25 to be the command to stop a print from DWC or some other source, the M226 in my case is like an extension of Gcode. I had to come up with a conditional check during printing.

The flow is as follows:

Print started -> Start Gcode toggles -> Custom macro starts extruder (controlled by a different board than the Duet3), Gcode paused by M226 in start gcode -> print paused till daemon.g checks for appropriate pressure (M24) -> continuous checks for any pressure jumps.

I imagined this as an extension of the Gcode, running along, not a definite break in the flow - like the M25.

The cases where I use my M226/M24 weren't causing problems yet, my print pauses at start, communicates with external controller for the extruder, resolves when pressure builds up fine. During the creation of this topic, these if statements were not checked, the delays have to stem from something else.

[awitc]

@infiniteloop @dc42

This sounds like it! I will remove this macro from the daemon and figure out a different way of doing this.

Is there a good way of saving parameters while the Duet3 is running? This is how I have done it so far:

; macro for updating the value

echo >"/sys/coef.g" "set global.coef = {global.coef}"

G4 P1
M98 P"coef.g"
echo "Updated val: ", global.coef

This is how the logic looks like (not direct code, because I don't have access to the printer right now), I have a variable in config.g, then the file coef.g contains an overwrite,
which is using a new value. In config, the macro is run at every start-up to load the new value from SD card. The global.coef can be overwritten during printer operation by other macros etc.

Looking at this code, I am feeling quite silly that I completely missed having a G4 P1 executing at EVERY loop iteration. I have focused on the code I added (the feeder checks) and that is when I noticed delays. The SD card saving macro I added couple of weeks ago, but didn't test it thoroughly (my oversight). Thanks a lot!

TLDR: avoid M98 in daemon.g, especially when it introduces another G4 delay.

edit:

@infiniteloop said in Problem with daemon.g timing:

As a self-assigned expert for infinite loops

took me a while

[dc42]

@awitc yes that's a good way of saving values, provided you don't call that macro too often, because that might cause excessive wear on the SD card. I believe the G1 P1 call in that macro is not needed, because the echo command will complete and close the file before the M98 command is called. But why do you need the M98 call in that macro at all? Surely it just resets global.coeff to the value it already has?

[awitc]

@dc42

I didn't have the macro for changing nozzle size at hand, so I copied my other one and replaced this line, so that it only demonstrates the saving. Should look like this, but it is for volume measurement parameters:

echo "Previous val: ", global.coef
echo "Reference volume: ", global.reference_volume
echo "Actual volume: ", global.actual_volume

echo >"/sys/coef.g" "set global.coef = "^{global.coef * global.reference_volume/global.actual_volume}^""

G4 P1
M98 P"coef.g"
echo "Updated val: ", global.coef

Here the result is calculated and stored in the coef.g file and then loaded so the printer has the updated value, too.

[infiniteloop]

@awitc

echo >"/sys/coef.g" "set global.coef = {global.coef}"

Extending on what @dc42 stated above, it’s fine to permanently store a variable like this, but it comes at a cost. You only need the file storage for some parameters to survive a reboot, reset or shutdown. During runtime, global variables do the job.

Now, if you have two macros, one to store a set of globals, and one to restore these from disk (SD card), you just have to spot some events who potentially lead to a reset/shutdown, such as pause or end of print, and call the ”store” macro on these events.

The ”restore” macro has to be called only once, either at the end of your config.g or when you start a print - whatever suits your needs.

Essentially, you just have to write the ”store” macro, as it creates (by using ”echo”) an executable file which then IS the ”restore” macro. If you need some sample code on how to compose complex expressions, just drop a note.

I completely missed having a G4 P1 executing at EVERY loop iteration.

Well, that’s just a millisecond. However note the usage of G4 at this line in your daemon.g:

G4 P{var.dt} ; loop delay, 500ms, P = [ms], S = [s]

This one is essential for granting time to parallel processes. For details, look up the ”daemon” sections from the links in my previous post.

If a macro has to wait for the completion of some code in the printing queue or from another macro, don’t use a delay, use M400 instead.

[awitc]

@infiniteloop @dc42

Is there a mechanism for the Duet3 to monitor things on a separate process than the gcode execution?

I have tested the behaviour of my daemon.g, like so:

var pump_started = false

var fans_started = false
var fans_time_elapsed = 0 ; [s]
var fans_time_disable = 30
var fans_timer_reset = true

var feeder_time_elapsed = 0 ; [s]
var feeder_time_disable = 8

var dt = 500 ; target loop time [ms]
var loop_start = 0
var loop_end = 0
var logic_duration = 0
var delay_time = 0


while true
	set var.loop_start = (state.upTime * 1000) + state.msUpTime
	
	; monitor current pressure
	if {global.nozzle_pressure > global.nozzle_pressure_threshold}
		M98 P"/macros/disable_extruder.g" ; stop extruder
		M226 ; stop print
		M118 S"Error: Nozzle pressure is too high!"; error message
	
	if {global.pump_pressure > (global.target_pressure + global.pressure_difference_threshold)}
		; if state.status == "processing"
		M98 P"/macros/disable_extruder.g" ; stop extruder
		M226 ; stop print
		M118 S"Error: Pump pressure not within correct limits!"; error message
		
	elif {abs(global.target_pressure - global.pump_pressure) < global.pressure_difference_threshold}	; if pressure is at the expected levels, resume print
		if state.status == "paused"
			M24 ; resume print job
	
	var t1 = (state.upTime * 1000) + state.msUpTime
	M118 S{"T1 - After pressure checks: " ^ (var.t1 - var.loop_start)}
	
	; monitor temperature
	if sensors.analog[9].lastReading  > global.pump_trigger_temp && var.pump_started == false
		set var.pump_started = true
		set var.fans_started = true
		set var.fans_timer_reset = true
		
		M42 P1 S1		; enable pump		
		M106 P3 S255	; enable fans
		
	if sensors.analog[9].lastReading  < global.pump_trigger_temp && var.pump_started == true
		set var.pump_started = false
		set var.fans_started = false
		
		M42 P1 S0 		; disable pump immediately
		
	var t2 = (state.upTime * 1000) + state.msUpTime
	M118 S{"T2 - After temp logic: " ^ (var.t2 - var.t1)}
		
	if var.fans_started == true
		set var.fans_time_elapsed = var.fans_time_elapsed + {var.dt} / 1000 ; [s]

		if var.fans_time_elapsed >= var.fans_time_disable
			set var.fans_time_elapsed = 0
			M106 P3 S0 		; disable fans
			set var.fans_started = false
			
	var t3 = (state.upTime * 1000) + state.msUpTime
	M118 S{"T3 - After fans logic: " ^ (var.t3 - var.t2)}
	
	; monitor pellet feeder
	if global.feeder_started == true
		set var.feeder_time_elapsed = var.feeder_time_elapsed + {var.dt} / 1000 ; [s]

	  	if var.feeder_time_elapsed >= var.feeder_time_disable
			set var.feeder_time_elapsed = 0
			set global.feeder_started = false
			M42 P5 S0		; disable feeder		
			M118 S"Pellet feeder disabled"
			
	var t4 = (state.upTime * 1000) + state.msUpTime
	M118 S{"T4 - After feeder logic: " ^ (var.t4 - var.t3)}
	
	set var.loop_end = (state.upTime * 1000) + state.msUpTime
	set var.logic_duration = var.loop_end - var.loop_start
	set var.delay_time = var.dt - var.logic_duration

	M118 S{"Loop time: " ^ var.logic_duration ^ " ms"}

	if var.delay_time > 0
		G4 P{var.delay_time} ; loop delay, 500ms, P = [ms], S = [s]

	; if nozzle_size changed, save the new value and update volume
	;M98 P"/macros/update_nozzle_size.g"
	;set global.reference_volume = global.volume_coefficient * global.nozzle_size
	

and obtained the following output:

5/28/2025, 7:24:11 PM Loop time: 1496 ms
5/28/2025, 7:24:11 PM T4 - After feeder logic: 230
5/28/2025, 7:24:10 PM T3 - After fans logic: 230
5/28/2025, 7:24:10 PM T2 - After temp logic: 406
5/28/2025, 7:24:10 PM T1 - After pressure checks: 480

This is for a state where none of the if statements are satisfied, so it is just the time of the expressions being evaluated to false.

Later on, I used this:

if 0 > 0
	; nothing
if 0 > 0
	; nothing
elif 0 > 0
	; nothing

which lead to:

5/28/2025, 7:30:12 PM Loop time: 1191 ms
5/28/2025, 7:30:12 PM T4 - After feeder logic: 205
5/28/2025, 7:30:12 PM T3 - After fans logic: 180
5/28/2025, 7:30:12 PM T2 - After temp logic: 351
5/28/2025, 7:30:11 PM T1 - After pressure checks: 305
5/28/2025, 7:30:11 PM Loop time: 1397 ms
5/28/2025, 7:30:10 PM T4 - After feeder logic: 225
5/28/2025, 7:30:10 PM T3 - After fans logic: 205
5/28/2025, 7:30:10 PM T2 - After temp logic: 485
5/28/2025, 7:30:09 PM T1 - After pressure checks: 302

Other data is code without the T1-4 messages runs at around 900ms, just time measuring runs at 100ms.

I do not want to put a bottleneck on the printing performance, therefore I am worried. I have set the dt to 2000ms, which should leave time for the printer to do stuff. Anyhow, I am looking for a solution which could avoid the coupling between these checks and gcode execution.

EDIT:
the dt = 2000ms is not great, as I want the pressure to be monitored in shorter intervals. Also, I assume this loop might grow in the future, which means I should definitely look for some other ways of doing this.

[infiniteloop]

@awitc

Is there a mechanism for the Duet3 to monitor things on a separate process than the gcode execution?

Daemon.g IS a separate process. However, all processes share a single resource, the MCU, which can become a bottleneck.

This does not mean that you have to count every single millisecond, just try to keep your code in daemon.g tight.

Now, your case is quite special, you intend to substitute the feeder logic built into RRF with your own control mechanism. Using a scripting language, execution times are calculated in seconds, that’s several orders of magnitude slower than the C++ code of RRF. IMHO you have three options:

1. Set up a minimal (but functional) prototype variant using daemon.g, run realistic test prints with it to see if the print runs smoothly. If it stutters or becomes jerky, take one of the other options.

2. Modify the firmware. That’s quite a significant effort, but you certainly get kind of realtime behaviour, even for complex tasks.

3. Use a separate controller (Arduino, ESP or such) as a loosely coupled sub-system. Here, interfacing the hardware (fans, feeder, pressure sensor etc.) is the tricky part, the software is - relatively - trivial to write.

the dt = 2000ms is not great

Keep it at 500ms as proposed in the documentation.

[dwuk3d]

Haven't read whole tread, but for regular trigger times or timeouts - rather than counting executions and adding delay times, in my daemon.g I am just having global 'next trigger time' variables - which I am checking against the upTime in every daemon.g execution, which I am finding is working quite well.

Once triggered - the global variable is then just set to the current upTime plus the next trigger time required.

I don't have any g4's or loops in the daemon.g - I am just letting it run whenever it likes - with most of the time it does nothing.

PS/. If you are using echo's to determine when things are running I have also found that sometimes echo's seem to get ignored - which caused me quite a lot of confusion until I worked out what was happening.

[infiniteloop]

@dwuk3d

rather than counting executions and adding delay times, in my daemon.g I am just having global 'next trigger time' variables - which I am checking against the upTime in every daemon.g execution

Good hint, that’s neat.

I don't have any g4's or loops in the daemon.g

Then, it is called every 10 seconds - which, depending on your needs, may be fine. From the GCode meta commands, section ”daemon.g”:

It is recommended to use a while loop inside the daemon.g file if you are using it to prevent the firmware having to open it every 10 seconds.

So, re-opening of the daemon.g file every 10 secs comes at a cost.

If you are using echo's to determine when things are running I have also found that sometimes echo's seem to get ignored

…not to mention the execution time of the echo command itself which has an impact on all measurements.

[dwuk3d]

@infiniteloop Interesting thanks - I didn't realise it only ran every 10 seconds - the way I read the docs made me think the 10 seconds relates to frequency of checking for file existence, rather than execution frequency - not a big issue for me though as I'm only using it for timing things out - like heaters after filament load, or electro magnets.

Re the upTime+ approach - I am also using it in long running loops in other things like parallel tool change timing loops. I actually ended up setting the G4 amounts within the loops to a slightly different value for every loop - so it easy to see where the processing is if it seems to be stuck - by looking at the current statement being executed from M122 output.