Power failure and resurrection on a CNC machine

[Nightowl]

@infiniteloop
Blimey, that's not a 30 minute read

I had a wander through the Object Model and realised there's a lot of information in there, but interacting with it I wasn't sure about. It's clear where 'things' are defined, like a spindle or a heater, but finding non-permanent values took a little getting used to.

What didn't help (me) was having several web browser windows open to try and see the impact of what I was doing, but every time I ran a file, it seemed all the windows went to the Dashboard screen - and what showed up in the Console screen was a bit random, too, appearing in one screen but not the other!

At least the Pause and Resume files are working now, with just one small glitch I think I can iron out, and then it's on to the next project

[infiniteloop]

@nightowl999 said in Power failure and resurrection on a CNC machine:

Blimey, that's not a 30 minute read

… but a reference I use almost all of the time

[Nightowl]

@infiniteloop
Actually, I had a look when trying to resolve the pause/ressume issues I was having, and it helped.

Thanks for the pointer!

[dc42]

@nightowl999 said in Power failure and resurrection on a CNC machine:

@infiniteloop

Thanks for that. Is this also the case with a 24v powered machine?

With 24V power the Duet should be able to save the important data to the resurrect.g file. However, there may not be enough power to complete movement commands in the M911 power fail script.

[Nightowl]

@dc42
The only 'mechanical' things, as I see it, would be to raise the Z height to clear the workpiece (same as a printer I guess), and switch the spindle off.

The problem might be how to switch the spindle on, at the correct speed, from within the resurrect-prologue.g file. I appreciate a variable is probably volatile so I wouldn't be able to get the spindle speed from there, but would it be saved in state.restorePoints[].spindleSpeeds[] of the Object Browser on a power fail - unexpected or otherwise?

Thanks

[Nightowl]

On from this a bit, as I've just had two power failures and hadn't configured M911 in the config.g file, so don't have a resurrect.g to recover from (I have now!) I'd be interested in looking at your...

...'customised recovery file generator'...

... @infiniteloop to see if I can find a way of saving the value of a variable in the resurrect.g file

[Nightowl]

@infiniteloop

I found the printed version...

[infiniteloop]

@nightowl999

to see if I can find a way of saving the value of a variable in the resurrect.g file

That’s difficult. If you have M911 configured near the end of your config.g, it will automatically generate resurrect.g and write it on the SD card. In a second step, M911 will execute the commands you told it with the P parameter. There, you might be able to invoke your own macro generator - although, I’m not quite sure about how to put a macro call into the P string …

Depending on whether resurrect.g has already been closed at this stage, you might be able to append own gCode commands - at the end of the file, which is the wrong place for what I think you want to achieve.

Another possibility is to save your own file, which can then be called from resurrect-prologue.g. To get an idea where in the recovery process resurrect-prologue.g is called, please have a look at this document.

The third option is to ignore resurrect.g completely and to generate your own recovery file instead.

Aside from the document linked above, you should thoroughly study the resurrect.g file you now have on your SD card. Best is to draw a copy onto your PC and to comment each single gCode you find. Also take care of the parameters and their meaning. Use the GCode Dictionary to spell the lines, until you can imagine what resurrect.g does and why so.

Then, you are finally able to decide whether you can get along with some supplemental file which is called from within resurrect-prologue.g, or if you have to bite the apple and write your own recovery mechanism.

In the meantime, I’ll prepare some code samples for you. Sorry, but this may take until tomorrow, as I have an Arduino on my desk which needs a severe debugging treatment

[Nightowl]

@infiniteloop
Thanks for that, and I'm pretty sure I can wait until tomorrow!

Yeah, I didn't realise the resurrect.g file was overwritten every time, but I should have!

Now my thinking is when the power fails the relevant information should be saved into
state.restorePoint[0].spindleSpeed and state.restorePoints[0].toolNumber, which I thought it would make sense for it to do that anyway, but it seems not.

Maybe I could write something into the M911 string to save that info into the restore points and 'recover' it from the resurrect-prologue.g file?

[infiniteloop]

@nightowl999

when the power fails the relevant information should be saved into
state.restorePoint[0].spindleSpeed and state.restorePoints[0].toolNumber

Sorry, but the object model is read-only. This said, I think I know what you mean: these are the properties you want to restore on top of resurrect.g.

Maybe I could write something into the M911 string to save that info into the restore points and 'recover' it from the resurrect-prologue.g file?

You must write a macro to generate a file with these data points. That file, in turn, is a macro itself, containing the data as arguments of GCode instructions who put these data into place.

Whether you can call this file from resurrect-prologue.g or not depends on the required order of commands: for instance, you should not turn the spindle on before the proper tool has been selected. That’s why you should study your resurrect.g in detail.

[infiniteloop]

@nightowl999 This time, the debugging session took not as long as I expected, so here’s some fish for the cat:

; --- just a sample resurrect.g file: don't ever execute! ---

√M141 P0 S50.0										; activate chamber heater
√G29 S1												; load the height map from file and activate bed compensation
√T-1 P0												; de-select tools, don't run macros
√G92 X-101.068 Y6.281 Z9.100							; set position of head without moving it
√G60 S1												; store current position in slot #1
M98 P"resurrect-prologue.g"
√M116												; wait for all temperatures to reach their set values
√M290 X0.000 Y0.000 Z0.100 R0						; restore babystepping offsets
√G10 L2 P1 X0.00 Y0.00 Z0.00							; L=2: this sets the origin of the coordinate system number
G10 L2 P2 X0.00 Y0.00 Z0.00							;	 specified by the P parameter (1 to 9) to the specified X, Y, X... values
G10 L2 P3 X0.00 Y0.00 Z0.00
G10 L2 P4 X0.00 Y0.00 Z0.00
G10 L2 P5 X0.00 Y0.00 Z0.00
G10 L2 P6 X0.00 Y0.00 Z0.00
G10 L2 P7 X0.00 Y0.00 Z0.00
G10 L2 P8 X0.00 Y0.00 Z0.00
G10 L2 P9 X0.00 Y0.00 Z0.00
G54													; select coordinate system #1
√G10 P1 S0 R0										; set active(S) and standby(R) temperatures for tool #1
√T1 P0												; select tool #1, don't run macros
√G10 P0 S230 R225									; set active(S) and standby(R) temperatures for tool #0
√T0 P6												; select tool #0, run macros (0b0110) - tfree=1, tpre=2 and tpost=4
M106 S0.70											; set fan speed (fan defaults to #0)
√M106 P0 S0.70										; 	now, with dedication to every possible fan
√M106 P1 S0.00
M106 P2 S0.00
M106 P5 S0.00
M106 P7 S0.00
M106 P8 S0.00
√M116												; wait for all temperatures to reach their set values
√G92 E0.00000										; set position of extruder without moving it
√M83													; extruder to relative
√M23 "0:/gcodes/Grundplatte II.gcode"				; select file for printing
√M26 S8015948 P0.000									; set pointer into file (P=proportion of the first move to be skipped, default 0.0, must be less than 1.0)
√G0 F6000 Z11.200									; place Z 2 mm above the printing position
√G0 F6000 X-91.068 Y6.281							; drive X and Y to the printing position
√G0 F6000 Z9.200										; lower Z for 2 mm into its printing position
√G1 F3000.0 P0										; use speed F, set port bits to 0 (remainder from RRF 2?)
G21													; set units to mm
√M24													; resume print (after a pause)

That’s an older resurrect.g file that I used as a first template for my own recovery macro. I’ve added comments to understand what’s going on, every GCode I then implemented in my counterpart has a checkmark.

Depending on the editor and font used, the comments are badly misaligned. For better readability, you might want to paste the content of the black box into your editor of choice and ameliorate the text at will.

Recently generated "resurrects" look a bit different, so I suggest you to publish your latest resurrect.g - I'd like to have a look at that.

[Nightowl]

@infiniteloop

Here is the latest resurrect.g file:

; File "0:/gcodes/Calibration Toolpath.gcode" resume print after power failure at 2022-05-29 16:49
G21
G92 X182.895 Y658.930 Z66.000
G60 S1
T-1 P0
M98 P"resurrect-prologue.g"
M116
M290 X0.000 Y0.000 Z0.000 R0
; Workplace coordinates
G10 L2 P1 X147.00 Y584.00 Z67.00
G10 L2 P2 X0.00 Y0.00 Z0.00
G10 L2 P3 X0.00 Y0.00 Z0.00
G10 L2 P4 X0.00 Y0.00 Z0.00
G10 L2 P5 X0.00 Y0.00 Z0.00
G10 L2 P6 X0.00 Y0.00 Z0.00
G10 L2 P7 X0.00 Y0.00 Z0.00
G10 L2 P8 X0.00 Y0.00 Z0.00
G10 L2 P9 X0.00 Y0.00 Z0.00
G54
M106 S0.50
M106 P0 S0.50
M116
G92 E0.00000
M83
M486 S-1
G17
M23 "0:/gcodes/Calibration Toolpath.gcode"
M26 S249
G0 F6000 Z68.000
G0 F6000 X182.895 Y658.930
G0 F6000 Z66.000
G1 F2500.0 P0
G21
M24

I was under the impression this file was overwritten with a new one every time there was a 'power failure', and it wasn't user-editable?

[infiniteloop]

@nightowl999 said in Power failure and resurrection on a CNC machine:

I was under the impression this file was overwritten with a new one every time there was a 'power failure', and it wasn't user-editable?

Your first impression is correct. Whether resurrect.g is write-protected or not? Can't tell you as I currently have no such file at hand …

Nevertheless, here's the first lesson - by the way, it demonstrates how M911 generates resurrect.g, up to the fact that the generated file will always be overwritten when we call M98 P"MWriter.g" again

; MWriter aka macro writer
; Creates the macro "HelloWorld.g" on the SD card which in turn can be executed
; Requirements: RRF v3.4

; To keep things simple, we use the /sys directory - which makes sense as any recovery macros should live in there.
; For development and tests, I use DWC, mainly with the "System" tab

var fName = "HelloWorld.g"			; this is the filename for the macro to create


; Here, we create the file - note the use of a single ">" with echo: this overwrites any existing file with this name.
echo >{var.fName} "; HelloWorld.g"
; with ">>", echo appends something to the now existing file:
echo >>{var.fName} "; Macro file created by macro ""MWriter.g"""	; we always write strings - note how to put a string into it (""MWriter.g"")
echo >>{var.fName} ""												; empty line for better readability
echo >>{var.fName} "M291 S2 R""Hello World"" P""Moin Moin"""		; message box, blocking (S2), with title (R) and message (P), localized :-)
echo >>{var.fName} ""
echo >>{var.fName} ";EOF"											; ... old habit of mine

;EOF																; same habit ...

Oh, and thanks for your resurrect.g - I'll try to study it. Did you make a copy and added comments?

[Nightowl]

@infiniteloop

Yes, here it is, but I'm not sure if I've done it correctly...

; File "0:/gcodes/Calibration Toolpath.gcode" resume print after power failure at 2022-05-29 16:49
G21					; sets units to mm
G92 X182.895 Y658.930 Z66.000		; sets the position for X, Y and Z
G60 S1					; save memory slot number
T-1 P0					; deselects the tool
M98 P"resurrect-prologue.g"		; runs the user-defined resurrect-prologue.g
M116					; wait for all temperatures to reach set value
M290 X0.000 Y0.000 Z0.000 R0		; sets babystepping (not CNC)
; Workplace coordinates
G10 L2 P1 X147.00 Y584.00 Z67.00	; origin of coordinate system number 
G10 L2 P2 X0.00 Y0.00 Z0.00		; as above for 3D printers
G10 L2 P3 X0.00 Y0.00 Z0.00		; as above for 3D printers
G10 L2 P4 X0.00 Y0.00 Z0.00		; as above for 3D printers
G10 L2 P5 X0.00 Y0.00 Z0.00		; as above for 3D printers
G10 L2 P6 X0.00 Y0.00 Z0.00		; as above for 3D printers
G10 L2 P7 X0.00 Y0.00 Z0.00		; as above for 3D printers
G10 L2 P8 X0.00 Y0.00 Z0.00		; as above for 3D printers
G10 L2 P9 X0.00 Y0.00 Z0.00		; as above for 3D printers
G54					; select coordinate system 1
M106 S0.50				; turns the fan on at half speed
M106 P0 S0.50				; turns user-configured fan on
M116					; wait for all temperatures to reach set value
G92 E0.00000				; sets the position for the extruder
M83					; sets extruder to relative mode
M486 S-1				; Indicate a non-object, purge tower, or other global feature
G17					; select XY plane for arc moves
M23 "0:/gcodes/Calibration Toolpath.gcode"	; selects the file previously running
M26 S249				; set file offset from the start of the interrupted file
G0 F6000 Z68.000			; rapid move to above saved Z height
G0 F6000 X182.895 Y658.930		; rapid move to saved XY axes
G0 F6000 Z66.000			; reaped move to saved Z height
G1 F2500.0 P0				; controlled linear move  relating to output pins
G21					; sets units to mm
M24					; start/resume SD print

[infiniteloop]

@nightowl999 said in Power failure and resurrection on a CNC machine:

Yes, here it is, but I'm not sure if I've done it correctly...

Thx, and well done! From a first glance, I get the impression that this resurrect.g is missing all the things which are specific to CNC routers.

Could you please try to edit this commented version and insert comment lines with the CNC specific commands you miss? Think carefully, the sequence of commands is essential, else you can wreck your machine in the worst case.

With this "script" ready, we can see if the required additions can be called from resurrect-prologue.g or if we have to write the whole M911 stuff ourselves.

[Nightowl]

@infiniteloop Whew, let's see how this goes...

; File "0:/gcodes/Calibration Toolpath.gcode" resume print after power failure at 2022-05-29 16:49
G21					; sets units to mm
G92 X182.895 Y658.930 Z66.000		; sets the position for X, Y and Z
G60 S1					; I think a user-defined save slot can be 0, 3, 4 or 5 (0 is default)
? T-1 P0				; I'm not sure why the tool is deselected
Assuming this data has been saved in the state.restorePoints nodes...
T0					; select the last tool used - this should be stored in the relevant state.restorePoints[*].toolNumber
Set savedSpindleSpeed			; select the last tool's spindle speed - this should be stored in the relevant state.restorePoints[0].spindleSpeeds
M98 P"resurrect-prologue.g"		; runs the user-defined resurrect-prologue.g
; M116					; not relevant for CNC
; M290 X0.000 Y0.000 Z0.000 R0		; not relevant for CNC
; Workplace coordinates
G10 L2 P1 X147.00 Y584.00 Z67.00	; I think this is necessary 
; G10 L2 P2 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P3 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P4 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P5 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P6 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P7 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P8 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P9 X0.00 Y0.00 Z0.00		; not relevant for CNC
? G54					; select coordinate system 1
; M106 S0.50				; not relevant for my CNC
M106 P0 S0.50				; turns user-configured fan on (I've got a case fan configured)
; M116					; not relevant for CNC
; G92 E0.00000				; not relevant for CNC
; M83					; not relevant for CNC
? M486 S-1				; not relevant for CNC
? G17					; not sure if this is relevant for CNC - it's not in the config.g file
M23 "0:/gcodes/Calibration Toolpath.gcode"
M26 S249				; set file offset from the start of the interrupted file
G0 F6000 Z68.000			; rapid move to above saved Z height
G0 F6000 X182.895 Y658.930		; rapid move to saved XY axes
G0 F6000 Z66.000			; reaped move to saved Z height
? G1 F2500.0 P0				; controlled linear move relating to output pins
G21					; sets units to mm
M24					; start/resume SD print

Before the power failure happens, I know where the spindle information is held, as I use that in my pause.g (which includes a test M28/M29 script) and resume.g files. These may help, so I've added them below...

; pause.g
M28 savedCNC.g
echo "Duet and RepRap are fantastic"
M29
set global.savedSpindleSpeed = spindles[0].active ; sets the global variable
echo "Spindle speed saved at " ^ {global.savedSpindleSpeed} ^ "RPM" ; shows saved spindle speed in the Console
G1 Z{max(move.axes[2].userPosition+5,move.axes[2].max-5)} F2400 ; move the Z axis to a safe height		
G0 X273.5 Y560 ; move XY to a safe place
M5 ; turn the spindle off

; resume.g
if state.currentTool =-1
	echo "No tool active.  Selecting tool zero"
	T0 ; select tool zero
if state.currentTool >= 0
	echo "Spindle state on tool ", state.currentTool, " is ", spindles[state.currentTool].state
if {state.currentTool >= 0} & {tools[state.currentTool].spindleRpm >= 0}
	M3 S{global.savedSpindleSpeed} ; resume saved spindle speed
	G4 S1 ; wait 1 second to allow the spindle to spin up
	echo "Spindle speed resumed at " ^ {global.savedSpindleSpeed} ^ "RPM" ; this should show the setting was successful
G0 R1 X0 Y0 ; move X and Y back to saved work XY location
G1 R1 Z0 F240 ; move Z slowly down to saved work Z location

[infiniteloop]

@nightowl999 Some remarks to your resurrect.g:

1. G60 S1 - this saves the current coordinate values in the memory slot which is later referred to from M24 (resume print). For this to work, the coordinates must have been set before. That’s the G92 line, in front of G60.
2. M486 S-1 - the context is object cancellation, but I don’t quite understand the meaning. In my resurrect.g, the parameter was T-1, which is not documented but could have meant something like „don’t cancel objects“.
3. G1 F2500.0 P0 - this one sets the feed rate, but the P0 is special - from the GCode Reference: „Pnnnn (supported only in some builds of RepRapFirmware)“. I once figured out the port bits (IOBITS), but I have no guess whether P0 is used by our RRF or simply ignored.

All of the other entries have a potential meaning with 3D printers, depending on the setup, but I have not the faintest idea which entries you really need for your mill: do you heat a nozzle, use multiple coordinate systems or do you extrude something? OK, just see your latest post - I’ll go through that in a minute … [contemplating] …

[infiniteloop]

@nightowl999 After some contemplation, I commented your commented resurrect.g:

; File "0:/gcodes/Calibration Toolpath.gcode" resume print after power failure at 2022-05-29 16:49
G21					; sets units to mm



; >>> These two lines belong together and prepare the latter use of M24
G92 X182.895 Y658.930 Z66.000		; sets the position for X, Y and Z
G60 S1					; I think a user-defined save slot can be 0, 3, 4 or 5 (0 is default)



? T-1 P0				; I'm not sure why the tool is deselected
; >>> ALL tools are deselected or, in a sense, deactivated. I’m not sure, but I think that in CNC mode, this would prevent any motorized tool from spinning?

 Assuming this data has been saved in the state.restorePoints nodes...
 T0					; select the last tool used - this should be stored in the relevant state.restorePoints[*].toolNumber
 Set savedSpindleSpeed			; select the last tool's spindle speed - this should be stored in the relevant state.restorePoints[0].spindleSpeeds

; >>> At the time this macro is generated, these data can directly be read from the object model - in a way, you "store" them in resurrect.g for later use



M98 P"resurrect-prologue.g"		; runs the user-defined resurrect-prologue.g
; M116					; not relevant for CNC
; M290 X0.000 Y0.000 Z0.000 R0		; not relevant for CNC
; Workplace coordinates
G10 L2 P1 X147.00 Y584.00 Z67.00	; I think this is necessary 
; >>> Yes.

; G10 L2 P2 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P3 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P4 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P5 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P6 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P7 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P8 X0.00 Y0.00 Z0.00		; not relevant for CNC
; G10 L2 P9 X0.00 Y0.00 Z0.00		; not relevant for CNC

? G54					; select coordinate system 1
;  >>> Needed to activate G10 L2 P1


; M106 S0.50				; not relevant for my CNC
M106 P0 S0.50				; turns user-configured fan on (I've got a case fan configured)
 ; M116					; not relevant for CNC
 ; G92 E0.00000				; not relevant for CNC
 ; M83					; not relevant for CNC
 ? M486 S-1				; not relevant for CNC
; >>> Fine. One problem less :-)

 ? G17					; not sure if this is relevant for CNC - it's not in the config.g file
; >>> Don’t know either - do your .gcode files contain G2 oder G3 commands? I think the larger problem is to get to know the proper command in use from the object model: G17, G18 or G19


M23 "0:/gcodes/Calibration Toolpath.gcode"
M26 S249				; set file offset from the start of the interrupted file
G0 F6000 Z68.000			; rapid move to above saved Z height
G0 F6000 X182.895 Y658.930		; rapid move to saved XY axes
G0 F6000 Z66.000			; reaped move to saved Z height
; >>> Not quite: the first G0 puts Z 2 mm above the saved height (to ensure lee-room for the following X/Y move), the third G0 goes to the bone.
; >>> That's suitable for 3D Printers, don't know how to best navigate a tool on your router ..

? G1 F2500.0 P0				; controlled linear move relating to output pins
; >>> How can I translate "extrusion rate" into the CNC world? Are any "extruders" in use?

G21					; sets units to mm
M24					; start/resume SD print

Fine. That's less work than needed for a 3D Printer to recover properly.

In the next step, I have to figure out what trick you apply to manage your spindle. Was a good idea to add your pause/resume scripts to your post

[Nightowl]

@infiniteloop Thank you

In response to your comments, here are mine...

Line 13 - M3 would stop the spindle spinning
Line 19 - I know where they are, but at what point can the values be 'recovered' and saved
Line 52 - Not in config.g, but I CAN find G3 in the toolpath (gcode) files (e.g. "G3X31.755Y28.580I3.175J0.000" from a Vectric CAD file (these are G2 - clockwise, and G3 - counterclockwise, arc movements)
Line 60 - OK, understood
Line 61 - I think movement is controlled by G1 X*Y*X*  with speed defined with F*, e.g. G1X49.124Y56.098F3683.0
Line 64 - No, there are no extruders

[infiniteloop]

@nightowl999 OK, now I have edited (faked) a resurrect.g, easy to read as it is really short. I've reflected some of your comments…

; File "0:/gcodes/Calibration Toolpath.gcode" resume print after power failure at 2022-05-29 16:49


G21					; sets units to mm
G92 X182.895 Y658.930 Z66.000		; sets the position for X, Y and Z
G60 S1					; save to slot 1 for latter use of M24

T-1 P0					; this enables the macros tpreN.g and tpostN.g to be called by the following T0
T0						; change(!) to tool T0, call the corresponding macros

M3 S10000				; set spindle speed - don’t worry, we create a macro with this line like we created HelloWorld :-)

M98 P"resurrect-prologue.g"		; runs the user-defined resurrect-prologue.g

G10 L2 P1 X147.00 Y584.00 Z67.00	; restore the coordinates for coords-system #1
G54					; select coordinate system 1

G17					; or G18, G19 - needed for circular moves

M23 "0:/gcodes/Calibration Toolpath.gcode"		; select the interrupted CNC file
M26 S249				; set file offset from the start of the interrupted file
G0 F6000 Z68.000			; rapid move to above saved Z height
G0 F6000 X182.895 Y658.930		; rapid move to saved XY axes
G0 F6000 Z66.000			; rapid move to saved Z height

G21					; sets units to mm
M24					; start/resume SD print

With the G0 movements, formerly known as line 61, you misunderstand my concerns: It's not the syntax, but what path is safe for these head moves on a CNC? Are you fine with the movements as they are done in resurrect.g?

So, your homework for now: carefully check my faked resurrect.g, especially with regard to the position of the M3 command: shall the spindle start that early in the recovery process?