Duet 3/Rpi + toolboard on 3.2b3 - G29 fails

[oozeBot]

Just upgraded to 3.2b3 on a Duet3/RPi with toolboard. I've now run the following macro 3 times and it's failed every time after 20-30 probes during the G29. Since it's in the middle of a G29, nothing is reported to the console..

This is new behavior in 3.2b3. Please let me know if there is anything I can do to capture details of what is happening. Thanks

M98 P"homez.g"
M557 X20:280 Y20:280 S20	; 169 mesh probe grid
M561
 
G29 S0
 
G1 X152.4 Y152.4 Z10

[gloomyandy]

What sort of probe are you using? If it is a bl touch do you have deployprobe/retractprobe files? You could try enabling M111 P3 S1 and M111 P16 s1 and posting the output.

[oozeBot]

My mistake for not adding that.. this is a BLTouch 3.1.

deployprobe.g

M280 P0 S10

retractprobe.g

M280 P0 S90

[oozeBot]

@gloomyandy said in Duet 3/Rpi + toolboard on 3.2b3 - G29 fails:

You could try enabling M111 P3 S1 and M111 P16 s1 and posting the output.

I'll try this now and report back.. Also, I'm going to have to reboot as it's stuck in a Busy state trying to process the G29 command.

[oozeBot]

@gloomyandy ok, it stopped again after 50-60 probes.. how can I retrieve the logging? It is still in the G29 loop and nothing has been written to the console.

[gloomyandy]

Connect via USB and the logging should go there.

[oliof]

Did you update the firmware on the toolboard as well?

[oozeBot]

@oliof yes - and updated the bootloader.. will be testing with a usb cable attached at some point here soon.

[oozeBot]

I've had time to run mesh bed leveling again with a serial monitor attached. With M111 P3 S1 and M111 P16 S1 enabled, nothing worth reporting is captured. It simply just stops somewhere around 40~60 probe points in.

Below is what was captured. @dc42 - could you give me any guidance on what else I can try to capture the details of what is happening?

From the snippet below, it stopped on line 18, with the probe extended..

Requesting macro file 'deployprobe.g' (fromCode: true)
HTTP: M280 P0 S10
Requesting macro file 'daemon.g' (fromCode: false)
Macro completed on channel 0
Waiting macro completed on channel 9
Requesting macro file 'daemon.g' (fromCode: false)
Waiting macro completed on channel 9
Requesting macro file 'retractprobe0.g' (fromCode: true)
Waiting macro completed on channel 0
Requesting macro file 'retractprobe.g' (fromCode: true)
HTTP: M280 P0 S90
Macro completed on channel 0
Requesting macro file 'daemon.g' (fromCode: false)
Waiting macro completed on channel 9
Requesting macro file 'deployprobe0.g' (fromCode: true)
Waiting macro completed on channel 0
Requesting macro file 'deployprobe.g' (fromCode: true)
HTTP: M280 P0 S10
Aux: M409 F"d99f"
Macro completed on channel 0
Requesting macro file 'daemon.g' (fromCode: false)
Waiting macro completed on channel 9
Requesting macro file 'daemon.g' (fromCode: false)
Waiting macro completed on channel 9
Requesting macro file 'daemon.g' (fromCode: false)
Waiting macro completed on channel 9
Requesting macro file 'daemon.g' (fromCode: false)
Waiting macro completed on channel 9
Aux: M409 F"d99f"

[oozeBot]

I just tried to lower the routine to a 49 point mesh and it still failed multiple times at different locations.

Sigh.. I understand it's a Sunday, but if anyone sees this and has any ideas on how to capture detailed info on the issue before rolling back to 3.2b1 in a few hours, please let me know.

[gloomyandy]

Run the same test but also run DCS in debug mode and capture the output.

[oozeBot]

@gloomyandy DCS is not an option, unless there is another way?

Debugging disabled for modules: Platform(0) Network(1) Webserver(2) GCodes(3) Move(4) Heat(5) DDA(6) Roland(7) Scanner(8) PrintMonitor(9) Storage(10) PortControl(11) DuetExpansion(12) FilamentSensors(13) WiFi(14) Display(15) LinuxInterface(16) CAN(17)

[gloomyandy]

Sorry should have provided more details. You need to do this on the rPi. This is the official doc: https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Monitoring_optional

I'm not sure if that is 100% correct for 3.2-beta3 though. In particular you may find that when you run DCS as root (which this will do), you can no longer connect via the web interface. But you should still be able to run the G29 via the USB console. You may also find that this produces a lot of output so you may find that changing the command to:
sudo /opt/dsf/bin/DuetControlServer -l debug | tee log
will help. This will send a copy of the output to a file called log in your current directory.

[oozeBot]

@gloomyandy here is the tail end of the output before it fails.. there is nothing valuable. It simply stops:

[info] Finished macro file retractprobe.g
[debug] HTTP: ==> Unfinished starting code: G29 S0
[debug] HTTP: Disposing macro file retractprobe.g
[debug] HTTP: Sent M566 Z600.00, remaining space 1472, needed 32
[debug] HTTP: -> Resumed suspended code
[debug] HTTP: Sent M201 Z600.00, remaining space 1440, needed 32
[debug] HTTP: -> Resumed suspended code
[debug] HTTP: Sent G1 X152.4 Y152.4 Z10, remaining space 1392, needed 48
[debug] HTTP: -> Resumed suspended code
[debug] HTTP: Suspending code M566 Z600.00
[debug] HTTP: Suspending code M201 Z600.00
[debug] HTTP: Suspending code G1 X152.4 Y152.4 Z10
[debug] Macro file deployprobe0.g not found
[debug] HTTP: ==> Starting code G29 S0
[debug] HTTP: Disposing macro file deployprobe0.g
[debug] HTTP: ==> Unfinished starting code: G29 S0
[debug] HTTP: Sent M566 Z600.00, remaining space 1472, needed 32
[debug] HTTP: -> Resumed suspended code
[debug] HTTP: Sent M201 Z600.00, remaining space 1440, needed 32
[debug] HTTP: -> Resumed suspended code
[debug] HTTP: Sent G1 X152.4 Y152.4 Z10, remaining space 1392, needed 48
[debug] HTTP: -> Resumed suspended code
[debug] HTTP: Suspending code M566 Z600.00
[debug] HTTP: Suspending code M201 Z600.00
[debug] HTTP: Suspending code G1 X152.4 Y152.4 Z10
[info] Starting macro file deployprobe.g on channel HTTP
[debug] HTTP: ==> Starting code G29 S0
[debug] Waiting for execution of M280 P0 S10 (macro code)
[debug] Processing M280 P0 S10
[debug] Waiting for finish of M280 P0 S10
[debug] HTTP: Sent M280 P0 S10, remaining space 1496, needed 40
[debug] Completed M280 P0 S10
[debug] Finished codes from macro file deployprobe.g
[info] Finished macro file deployprobe.g
 

I tried running M122. It's added to the queue but since G29 is still running, it's blocked from executing. The only way I know out of this is to reboot, which clears any errors.

[gloomyandy]

There may be something in there that is useful to Christian when he gets to look at it, so if you have the full log file please post it for him to be able to look at.

[oozeBot]

@gloomyandy @chrishamm Here is the log. It also contains a bonus crash at line 362-364 where it lost connection and soft-rebooted itself. I hadn't seen that before..

I'm going to be rolling back to 3.2b1 here soon. Please let me know if there is anything else I can assist with testing..

log.txt

[jbarros]

Just to add some data to this thread, I'm able to finish 210 point Mesh Probing on Duet3 with Toolboard with no SBC running 3.2 beta3:

09/11/2020, 12:27:29	210 points probed, min error -0.063, max error 0.349, mean 0.079, deviation 0.085
Height map saved to file 0:/sys/heightmap.csv

[dc42]

@jbarros said in Duet 3/Rpi + toolboard on 3.2b3 - G29 fails:

Just to add some data to this thread, I'm able to finish 210 point Mesh Probing on Duet3 with Toolboard with no SBC:

09/11/2020, 12:27:29	210 points probed, min error -0.063, max error 0.349, mean 0.079, deviation 0.085
Height map saved to file 0:/sys/heightmap.csv

Thanks, that's a very useful data point. Looks like it's something to do with running the deploy/retract files when they are held on the SBC, because the rest of the G29 probing function doesn't involve the SBC except at the start and end.

[oozeBot]

@dc42 I still have one machine running 3.2b3 if there is anything you or others want me to test on this issue.

I plan on rolling back to 3.2b1, creating a height map, and the moving then back to 3.2b3 for other testing by this afternoon though.

[dc42]

@oozeBot said in Duet 3/Rpi + toolboard on 3.2b3 - G29 fails:

@dc42 I still have one machine running 3.2b3 if there is anything you or others want me to test on this issue.

I plan on rolling back to 3.2b1, creating a height map, and the moving then back to 3.2b3 for other testing by this afternoon though.

@chrishamm, is there anything else you would like @oozeBot to test?