Output buffers running out

Adrian52

@dc42 I haven't found anything that makes it consistently better or worse. It's not concurrent sessions in my case. I have just got rrf3 running on my system - will let you know how I get on with that.

Adrian52

@dc42 the problem persists for me with rrf3 alpha 12. After printing about 1000mm filament, M122 gave no output at the dwc console, but remained connected. Yat M122 showed 12 of 24 (24max). The print was still in progress at this point, and I noticed that yat was receiving a line feed for every print move. Not sure where these are coming from - maybe its a slicer problem? I'm using kisslicer.

dc42

@Adrian52 said in Output buffers running out:

@dc42 the problem persists for me with rrf3 alpha 12. After printing about 1000mm filament, M122 gave no output at the dwc console, but remained connected. Yat M122 showed 12 of 24 (24max). The print was still in progress at this point, and I noticed that yat was receiving a line feed for every print move. Not sure where these are coming from - maybe its a slicer problem? I'm using kisslicer.

That's interesting, and it could be related to running out of output buffers. Do you also get this linefeed in YAT when printing with RRF2?

Adrian52

@dc42 yes I do. Looking more closely, the LF's don't occur with every move. They mostly seem to happen with retractions, but sometimes a few retractions go by without a LF. I am using firmware retraction which kisslicer seems to handle well.

nhof

I have had this issue intermittently causing loss of response, it occurs much less frequently in 2.04 compared to 2.03. Used to happen every 1-8 hours, since the update it has happened only once or twice in several weeks.

Only happened on wifi controller (not ethernet). Happened regardless of whether any clients were connected to DWC or whether a gcode print was running or not (even happened with machine idle).

It never happens when the network is disabled even in 2.03.

Unfortunately I haven't found any steps to reproduce the issue reliably.

Adrian52

@dc42 have been looking further at when line feeds occur, and whether they relate to buffers running out.
speed of printing and frequency of retraction seem to be important. When printing a tall object with a simple profile, I got virtually no LF, and printing 11,000mm filament over 4hr the buffers remained at 3 of 24 (13max).
To generate lots of retractions, I printed a 40x40x6mm cube, cut half depth by five 3mm cuboids in each direction (so that 25 little cubes stick up). The bottom half without cubes printed at about 50mm/sec didn't generate many LF's, and the buffers were 4 of 24 (15 max) . The top half with the cubes generated lots of LF, mainly coinciding with retractions. By the end of the top layer, buffers were 15 of 24 (24 max), and sending M122 from the dwc console gave a temporary disconnection.
I am using firmware retraction with M207 S2.7 R0.0 F2500 T1500 Z0.4 - not sure if this is out of the ordinary.

dc42

@Adrian52 said in Output buffers running out:

@dc42 have been looking further at when line feeds occur, and whether they relate to buffers running out.
speed of printing and frequency of retraction seem to be important. When printing a tall object with a simple profile, I got virtually no LF, and printing 11,000mm filament over 4hr the buffers remained at 3 of 24 (13max).
To generate lots of retractions, I printed a 40x40x6mm cube, cut half depth by five 3mm cuboids in each direction (so that 25 little cubes stick up). The bottom half without cubes printed at about 50mm/sec didn't generate many LF's, and the buffers were 4 of 24 (15 max) . The top half with the cubes generated lots of LF, mainly coinciding with retractions. By the end of the top layer, buffers were 15 of 24 (24 max), and sending M122 from the dwc console gave a temporary disconnection.
I am using firmware retraction with M207 S2.7 R0.0 F2500 T1500 Z0.4 - not sure if this is out of the ordinary.

Thanks for the extra information. Please provide your config.g file and the GCode file that demonstrates the linefeeds well.

Adrian52

@dc42 This is my config.g - sorry its a bit messy, but I have options for rrf2 and rrf3.

; Think3DPrint3D configuration file for Mini Kossel for testing Duet WiFi

; Communication and general
M111 S0                             	; Debug off
M550 PKosselXL		; Machine name and Netbios name (can be anything you like)
M551 Preprap                        	; Machine password (used for FTP)
;*** If you have more than one Duet on your network, they must all have different MAC addresses, so change the last digits
M540 P0xBE:0xEF:0xDE:0xAD:0xFE:0xED 	; MAC Address
;*** Wifi Networking
M552 S1				; Enable WiFi. Disabled for setup and testing. Enable once set up on your network.

M555 P2                           	; Set output to look like Marlin
M575 P1 B57600 S1			; Comms parameters for PanelDue

G21                                 	; Work in millimetres
G90                                	; Send absolute coordinates...
M83                                 	; ...but relative extruder moves

; Axis and motor configuration
M569 P0 S0				; Drive 0 goes backwards - changed to backwards for robodigg clamps
M569 P1 S0				; Drive 1 goes backwards - "
M569 P2 S0				; Drive 2 goes backwards - "
M569 P3 S1				; Drive 3 goes forwards
M569 P4 S1				; Drive 4 goes forwards
M574 X2 Y2 Z2 S1			; set endstop configuration (all endstops at high end, active high)
;*** The homed height is deliberately set too high in the following - you will adjust it during calibration
M665 L360.25 R173.990 H297.084 B120.0 X0.538 Y0.525 Z0.000
M666 X1.493 Y-1.213 Z-0.280 A-1.03 B-0.66		; put your endstop adjustments here, or let auto calibration find them
M579 X1.00 Y1.005 Z1.00		        ;correct x y z scaling
M350 X16 Y16 Z16 I1	    		; Set 16x microstepping with interpolation
M350 E16 I1				; Set exruder to 16x with interpolation
M92 X200 Y200 Z200			; Set axis steps/mm
M906 X1200 Y1200 Z1200 E1200 I60	; Set motor currents (mA) and increase idle current to 60%
M201 X6000 Y6000 Z6000 E1500		; Accelerations (mm/s^2)
M203 X10000 Y10000 Z10000 E6000		; Maximum speeds (mm/min)
M566 X2000 Y2000 Z2000 E1500 P1		; Maximum instant speed changes mm/minute, switch on 2.03 jerk policy

; Thermistors
;****rrf2******
M305 P0 T100000 B3950 R4700 H30 L0	; Put your own H and/or L values here to set the bed thermistor ADC correction
;****rrf3******
;M308 S0 P"bedtemp" Y"thermistor" T100000 B4138      ; configure sensor 0 as thermistor on pin bedtemp
;M950 H0 C"bedheat" T0                               ; create bed heater output on bedheat and map it to sensor 0
;****rrf2******
M305 P1 T100000 B4388 R4700 H30 L0	; Put your own H and/or L values here to set the first nozzle thermistor ADC correction
M305 P1 X200				;PT100 sensor for first nozzle
M143 H0 S120                                        ; set temperature limit for heater 0 to 120C
M307 H0 B0 S1.00                                    ; disable bang-bang mode for the nozzle heater and set PWM limit
;****rrf3******
;M308 S1 P"spi.cs1" Y"rtdmax31865"                   ; configure sensor 1 as thermocouple via CS pin spi.cs1
;M950 H1 C"e0heat" T1                                ; create nozzle heater output on e0heat and map it to sensor 1
M143 H1 S295                                        ; set temperature limit for heater 1 to 280C
M307 H1 B0 S1.00                                    ; disable bang-bang mode for the nozzle heater and set PWM limit

; ****Fans rrf2
M106 P1 T45 H1 				; thermostatic mode for fan 1 45 deg
M106 P2 H-1				; get fan 2 onto dwc
M106 P2 H-1 F25000 I1 C"Extra cool"	; correct pwm for 3 wire fan

; ****Fans rrf3
;M950 F0 C"fan0" Q500                                ; create fan 0 on pin fan0 and set its frequency
;M106 P0 S0 H-1                                      ; set fan 0 value. Thermostatic control is turned off
;M950 F1 C"fan1" Q500                                ; create fan 1 on pin fan1 and set its frequency
;M106 P1 S1 H1 T45                                   ; set fan 1 value. Thermostatic control is turned on
;M950 F2 C"!Fan2+^exp.pb6" Q25000
;M106 P2 S0 H-1
; Tools
M563 P0 D0 H1 F0                                    ; define tool 0
G10 P0 X0 Y0 Z0                                     ; set tool 0 axis offsets
G10 P0 R0 S0                                        ; set initial tool 0 active and standby temperatures to 0C

; Tool definitions
M563 P0 D0 H1 F0:2                     ; Define tool 0 with fan 0, fan 2 linked
;M563 P0 D0 H1 F0			; tool 0 with fan 0 only
G10 P0 S0 R0                        	; Set tool 0 operating and standby temperatures
;*** If you have a single-nozzle build, comment the next 2 lines
;M563 P1 D1 H2                      	; Define tool 1
;G10 P1 S0 R0                       	; Set tool 1 operating and standby temperatures
M92 E462                        	; Set extruder steps per mm
M572 D0 S0.08				; pressure advance tool 0
M207 S2.7 R0.0 F2500 T1500 Z0.40             ; firmware retraction setting

; Z probe and compensation definition
;*** If you have a switch instead of an IR probe, change P1 to P4 in the following M558 command
;M558 P1 X0 Y0 Z0			; Z probe is an IR probe and is not used for homing any axes
M558 P8 R0.4 F1000 H15 A10 S0.03		;Z probe smart effector
;G31 P100 X0 Y0 Z-0.30			;probe offset for glass bed
G31 P100 X0 Y0 Z-0.12 			;probe offset for pei bed
;G1 Z0.1
;G92 Z0
;G31 X3 Y-15 Z1.1854 P500			; Set the zprobe height and threshold for IR probe(put your own values here)

;laser filament monitor settings
;****rrf2******
M591 D0 P5 C10 R200:400 E10 S0 A1		;extruder 0, laser w/o switch, e0 endstop, 40-160% range, 10mm, inactive (S1 active), check all motion
;****rrf3******
;M591 D0 P5 C"connlcd.encb" R40:360 E10 S0 A1	;version for reprap3

;*** If you are using axis compensation, put the figures in the following command
M556 S100 X0 Y0 Z0                  	; Axis compensation here
M593 F35.5				; Damp ringing at 35.5Hz
M501					;use override-config.g parameters
M208 S1 Z-0.2				; set minimum Z
G28					;Home axes
G29 S1				;load height map and activate mesh compensation
;
T0					; select first hot end

The test gcode file I referred to : 40x40x6 25 blocks red pla 220 speed 72.gcode

dc42

Thanks. I have messy config with both RRF2 and RRF3 options too! I leave both enabled and I ignore the warning messages on startup.

You don't need the M540 command, it looks like a hangover for when you were using an older Duet.

Did you test whether the linefeeds occur with RRF2 as well as RRF3?

PS - have you tried running the print in simulation mode, to see if you still get the linefeeds?

DeltaCon

In my case (Duet 0.6) going from 2.03 to 2.04 the problem was also much lees frequent. It went from a few times per hour to a few times a day. Also I noticed that sometimes DWC succeeds in a reconnect, but that does not last for long. DC have you programmed some sort of buffer purge or something? A powercycle is needed to get any work done in DWC. However the printjob keeps going without issues. I am using KISSlicer mostly too, but doubt that it is causing this issue.

At first I thought the conversion from TL lighting to LED lighting in my office was responsible for the less frequent issue because the TL used to flicker a lot and I was thinking about electrical noise. But in retrospect I think it was DC's effort in the 2.04 version

Adrian52

@dc42 The line feeds and buffers running out behaved very similarly in rrf2 and rrf3. No line feeds during simulation with 2 or 3.

dc42

Thanks. I'm running your print on my delta now. No linefeeds in YAT so far, but it's still on the first layer.

dc42

OK, I am getting linefeeds now!

Adrian52

@dc42 they really get going for me in the top 3mm

dc42

It's not to do with retractions. I commented out all the G10 and G11 commands in the file, and it still produces the linefeeds.

dc42

Looks like it's the M106 commands in the file that provoke the linefeeds.

dc42

I have found the reasons for the line feeds, and a plausible (but not conclusive) explanation for the loss of output buffers. However, I was unable to reproduce the issue with loss of output buffers. Would you be willing to test a firmware build that deliberately leaves the linefeeds but (hopefully) fixes the output buffer issue? If so, would you prefer a RRF2 or RRF3 build?

Adrian52

@dc42 that would be fine. I am set up for both 2 and 3, so whatever is convenient.

dc42

@Adrian52, when you have had the output buffers being used up, have you ever run M122 and checked the "Error status" field in the Platform section? I have a feeling that bit 3 (i.e. value 8 ) may be set. Of course there would have to be enough output buffers left for that section to be present in the M122 report.

sdavi

@dc42 btw there is a missing buf->whenQueued = millis(); in OutputBuffer::Allocate. Probably more a issue only affecting the LPC port due to the smaller UART ringbuffer, but was causing some buffers to incorrectly timeout.