[3.5b1+]OoM after a while when using lots of local array vars

Diamondback

Hey,

currently playing around with the new custom array feature and trying to figure out how to make the most of it and what the limitations are. (that's why the code is intentionally very memory inefficient)

I am using a 8x8 dotstar grid on my toolhead for display purposes and here's a little macro I wrote to show a number:

var states = {{0,1,1,1,1,1,1,0},{1,1,1,1,1,1,1,1},{1,1,1,0,0,1,1,1},{0,1,1,1,0,0,0,0},{0,0,0,1,1,1,0,0},{0,0,0,0,0,1,1,1},{1,1,1,1,1,1,1,1},{1,1,1,1,1,1,1,1}}
var color = {0,0,255}
M98 P"/macros/Misc/ToolheadDisplay/8x8Grid" S{var.states} C{var.color}

With the second macro being this:

var states = param.S
var color = param.C
var brightness = 10
if exists(param.B)
	set var.brightness = param.B
while iterations < #var.states
	var lineData = var.states[iterations]
	while iterations < #var.lineData
		M150 R{var.color[0]} U{var.color[1]} B{var.color[2]} P{var.lineData[iterations] > 0 ? var.brightness : 0} S1 F1
M150 R0 U0 B0 P0 S1 F0 ;announce last LED with a dummy

Now, this code happens to be run once a second from daemon.g during testing and after a while, the board resets with an out of memory error.
I was expecting my local variables to not linger around forever and cause any issues, but then again I don't know anything about the memory management and possible GC work from meta gcode...

So question now: Is that behavior expected? Do custom arrays just stay in memory forever?

dc42

@Diamondback please try the binary at https://www.dropbox.com/sh/6s1sbescl9di7d2/AAAaiB9VF-XT_VpIRYL8Lq-0a?dl=0.

OwenD

@Diamondback
I can't speak for how long the variables stay in memory, however your macro and method of calling it is very inefficient in terms of memory usage.
You're creating (the same) arrays in both macros and duplicating the individual array elements in the second macro.

Edit
This looks very odd??

P{var.lineData[iterations] > 0 ? var.brightness : 0}

dc42

@Diamondback which board is this running on?

You can use M122 to track the memory usage of the garbage-collected heap, which is where arrays are stored.

Diamondback

@OwenD said in [3.5b1+]OoM after a while when using lots of local array vars:

I can't speak for how long the variables stay in memory, however your macro and method of calling it is very inefficient in terms of memory usage.
You're creating (the same) arrays in both macros and duplicating the individual array elements in the second macro.

Yep I know, this is on purpose as I want to see how RRF deals with this and how careful I need to be when using custom arrays.

Diamondback

@dc42 Duet 3 6HC

Diamondback

@OwenD said in [3.5b1+]OoM after a while when using lots of local array vars:

This looks very odd??

What do you mean? This line just sets the desired brightness of a pixel based on whether or not it's supposed to be on.

OwenD

@Diamondback said in [3.5b1+]OoM after a while when using lots of local array vars:

@OwenD said in [3.5b1+]OoM after a while when using lots of local array vars:

This looks very odd??

What do you mean? This line just sets the desired brightness of a pixel based on whether or not it's supposed to be on.

Gotcha.
I wasn't seeing how it resolved to an integer.
Had to go look at the syntax.
I learned something

Diamondback

@dc42 said in [3.5b1+]OoM after a while when using lots of local array vars:

You can use M122 to track the memory usage of the garbage-collected heap, which is where arrays are stored.

What exactly shall I look at? When running M122 from time to time (with the memory guzzling code active) I can see the "never used RAM" go down in fairly large chunks once in a while until it resets.

dc42

@Diamondback I will try to reproduce it.

dc42

@Diamondback some of the memory allocated for storing the nested arrays is not being reclaimed. I will investigate.

The line from the M122 report that I was referring to is this one:

Heap OK, handles allocated/used 495/450, heap memory allocated/used/recyclable 30720/28968/168, gc cycles 23

dc42

@Diamondback please try the binary at https://www.dropbox.com/sh/6s1sbescl9di7d2/AAAaiB9VF-XT_VpIRYL8Lq-0a?dl=0.

Diamondback

@dc42 This seems to fix it the memory leak Now it only resets after a while due to the issue mentioned here (same behavior as without using the local arrays all the time)
https://forum.duet3d.com/topic/31168/3-5b1-reset-reason-stuckinspinloop