2 SD Cards - SBC - stop after pause.g 3.5beta2

Phaedrux

Any point is fine. I want to see that all the firmware versions are matching.

Phaedrux

When DWC is unresponsive, is the pi still running?

Can you follow these steps to gather some logs on the pi?

https://github.com/Duet3D/DuetSoftwareFramework/wiki/SBC-Setup-Guide#increasing-log-level

IndeX4D

M122 B100
Diagnostics for board 100:
Duet TOOL1LC rev 1.1 or later firmware version 3.5.0-beta.3 (2023-04-13 18:44:05)
Bootloader ID: SAMC21 bootloader version 2.3 (2021-01-26b1)
All averaging filters OK
Never used RAM 5360, free system stack 92 words
Tasks: Move(nWait,0.0%,149) HEAT(nWait,0.2%,85) CanAsync(nWait,0.0%,57) CanRecv(nWait,0.0%,76) CanClock(nWait,0.0%,67) ACCEL(nWait,0.0%,53) TMC(delaying,3.0%,57) MAIN(running,91.7%,346) IDLE(ready,0.0%,27) AIN(delaying,4.9%,142), total 100.0%
Last reset 00:13:22 ago, cause: software
Last software reset data not available
Driver 0: pos 0, 400.0 steps/mm,standstill, SG min 0, read errors 1, write errors 1, ifcnt 79, reads 8000, writes 12, timeouts 0, DMA errors 0, CC errors 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter 6/17, peak Rx sync delay 216, resyncs 0/0, no timer interrupt scheduled
VIN voltage: min 24.0, current 24.1, max 24.1
MCU temperature: min 40.8C, current 41.2C, max 41.3C
Last sensors broadcast 0x00000002 found 1 248 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 16062, send timeouts 0, received 10443, lost 0, free buffers 37, min 37, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
Accelerometer: LIS3DH, status: 00
I2C bus errors 0, naks 3, other errors 0
14.4.2023, 23:10:11	M122 B110
Diagnostics for board 110:
Duet TOOL1LC rev 1.1 or later firmware version 3.5.0-beta.3 (2023-04-13 18:44:05)
Bootloader ID: SAMC21 bootloader version 2.3 (2021-01-26b1)
All averaging filters OK
Never used RAM 5568, free system stack 92 words
Tasks: Move(nWait,0.0%,149) HEAT(nWait,0.2%,101) CanAsync(nWait,0.0%,57) CanRecv(nWait,0.0%,76) CanClock(nWait,0.0%,67) ACCEL(nWait,0.0%,53) TMC(nWait,3.0%,57) MAIN(running,91.8%,336) IDLE(ready,0.0%,27) AIN(delaying,4.9%,142), total 100.0%
Last reset 00:13:16 ago, cause: software
Last software reset at 2022-02-23 14:40, reason: StackOverflow, available RAM 2656, slot 0
Software reset code 0x0100 ICSR 0x0000000e SP 0x20007f34 Task HEAT Freestk 4042 ok
Stack: 20003fb0 20003fe4 00019d07 00000000 00000000 0000000a 0001a407 00000000 00019511 20003298 fffffffd a5a5a5a5 00000000 00000002 00000000 00000002 0001a1f7 00000000 20002130 200020d4 2000207c 00023368 200020d4 20002130 00000032 200021f8 00013ca5
Driver 0: pos 0, 690.0 steps/mm,standstill, SG min 14, read errors 0, write errors 1, ifcnt 80, reads 4925, writes 12, timeouts 0, DMA errors 0, CC errors 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter 5/16, peak Rx sync delay 213, resyncs 0/0, no timer interrupt scheduled
VIN voltage: min 24.2, current 24.2, max 24.3
MCU temperature: min 30.3C, current 36.6C, max 37.1C
Last sensors broadcast 0x00000004 found 1 97 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 12759, send timeouts 0, received 10354, lost 0, free buffers 37, min 37, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
Accelerometer: LIS3DH, status: 00
I2C bus errors 0, naks 3, other errors 0
14.4.2023, 23:10:04	M122 B2
Diagnostics for board 2:
Duet EXP3HC rev 1.01 or earlier firmware version 3.5.0-beta.3 (2023-04-13 18:42:24)
Bootloader ID: not available
All averaging filters OK
Never used RAM 157816, free system stack 202 words
Tasks: Move(nWait,0.0%,162) HEAT(nWait,0.0%,90) CanAsync(nWait,0.0%,63) CanRecv(nWait,0.0%,79) CanClock(nWait,0.0%,70) TMC(nWait,5.8%,105) MAIN(running,92.8%,443) IDLE(ready,0.0%,40) AIN(delaying,1.3%,265), total 100.0%
Last reset 00:13:09 ago, cause: software
Last software reset data not available
Driver 0: pos 0, 80.0 steps/mm,standstill, SG min n/a, mspos 8, reads 63510, writes 11 timeouts 0, steps req 0 done 0
Driver 1: pos 0, 80.0 steps/mm,standstill, SG min n/a, mspos 8, reads 63510, writes 11 timeouts 0, steps req 0 done 0
Driver 2: pos 0, 80.0 steps/mm,standstill, SG min n/a, mspos 8, reads 63511, writes 11 timeouts 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter 3/22, peak Rx sync delay 183, resyncs 0/0, no timer interrupt scheduled
VIN voltage: min 24.0, current 24.0, max 24.0
V12 voltage: min 12.2, current 12.2, max 12.2
MCU temperature: min 37.2C, current 37.2C, max 37.8C
Last sensors broadcast 0x00000000 found 0 106 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 9492, send timeouts 0, received 13406, lost 0, free buffers 37, min 37, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
14.4.2023, 23:10:02	M122 B30
Diagnostics for board 30:
Duet EXP1HCL firmware version 3.5.0-beta.3 (2023-04-14 13:08:48)
Bootloader ID: SAME5x bootloader version 2.4 (2021-12-10)
All averaging filters OK
Never used RAM 53516, free system stack 184 words
Tasks: Move(nWait,0.0%,156) HEAT(nWait,0.0%,82) CanAsync(nWait,0.0%,62) CanRecv(nWait,0.0%,77) CanClock(nWait,0.0%,68) TMC(nWait,39.7%,344) CLSend(nWait,0.0%,152) MAIN(running,58.4%,415) IDLE(ready,0.1%,29) AIN(nWait,1.8%,267), total 100.0%
Last reset 00:13:07 ago, cause: software
Last software reset data not available
Closed loop enabled: yes, pre-error threshold: 8.00, error threshold: 4.00, encoder type rotaryQuadrature, position 0
Encoder reverse polarity: no, raw count 0
Tuning mode: 0, tuning error: 0x1, collecting data: no
Control loop runtime (us): min=5, max=32, frequency (Hz): min=10870, max=17045
Driver 0: pos 0, 80.0 steps/mm,not tuned/calibrated, SG min n/a, mspos 680, reads 20926, writes 20 timeouts 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter 2/18, peak Rx sync delay 202, resyncs 0/0, next timer interrupt due in 4 ticks, enabled, next step interrupt due in 3704500696 ticks, disabled
VIN voltage: min 24.6, current 24.6, max 24.6
V12 voltage: min 12.2, current 12.2, max 12.2
MCU temperature: min 31.1C, current 31.1C, max 34.1C
Last sensors broadcast 0x00000000 found 0 59 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 6327, send timeouts 0, received 13369, lost 0, free buffers 37, min 37, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
Accelerometer: none
I2C bus errors 12, naks 0, other errors 0
14.4.2023, 23:10:00	M122 B21
Diagnostics for board 21:
Duet EXP1HCL firmware version 3.5.0-beta.3 (2023-04-14 13:08:48)
Bootloader ID: SAME5x bootloader version 2.4 (2021-12-10)
All averaging filters OK
Never used RAM 53344, free system stack 184 words
Tasks: Move(nWait,0.0%,156) HEAT(nWait,0.0%,82) CanAsync(nWait,0.0%,63) CanRecv(nWait,0.0%,77) CanClock(nWait,0.0%,68) TMC(nWait,39.3%,344) CLSend(nWait,0.0%,152) MAIN(running,58.6%,411) IDLE(ready,0.2%,29) AIN(nWait,1.9%,267), total 100.0%
Last reset 00:13:05 ago, cause: software
Last software reset data not available
Closed loop enabled: yes, pre-error threshold: 8.00, error threshold: 4.00, encoder type rotaryQuadrature, position -1
Encoder reverse polarity: no, raw count 65535
Tuning mode: 0, tuning error: 0x1, collecting data: no
Control loop runtime (us): min=5, max=32, frequency (Hz): min=10870, max=17045
Driver 0: pos 0, 80.0 steps/mm,not tuned/calibrated, SG min n/a, mspos 248, reads 55305, writes 20 timeouts 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter -1/16, peak Rx sync delay 201, resyncs 0/0, next timer interrupt due in 23 ticks, enabled, next step interrupt due in 3706150520 ticks, disabled
VIN voltage: min 27.3, current 27.3, max 27.3
V12 voltage: min 12.1, current 12.1, max 12.1
MCU temperature: min 29.4C, current 29.4C, max 31.8C
Last sensors broadcast 0x00000000 found 0 145 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 9435, send timeouts 0, received 13331, lost 0, free buffers 37, min 37, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
Accelerometer: none
I2C bus errors 0, naks 0, other errors 0
14.4.2023, 23:09:58	M122 B20
Diagnostics for board 20:
Duet EXP1HCL firmware version 3.5.0-beta.3 (2023-04-14 13:08:48)
Bootloader ID: SAME5x bootloader version 2.4 (2021-12-10)
All averaging filters OK
Never used RAM 53340, free system stack 186 words
Tasks: Move(nWait,0.0%,156) HEAT(nWait,0.0%,82) CanAsync(nWait,0.0%,63) CanRecv(nWait,0.0%,77) CanClock(nWait,0.0%,68) TMC(nWait,39.3%,344) CLSend(nWait,0.0%,152) MAIN(running,58.6%,417) IDLE(ready,0.2%,29) AIN(nWait,1.9%,267), total 100.0%
Last reset 00:13:02 ago, cause: software
Last software reset data not available
Closed loop enabled: yes, pre-error threshold: 8.00, error threshold: 4.00, encoder type rotaryQuadrature, position 0
Encoder reverse polarity: no, raw count 0
Tuning mode: 0, tuning error: 0x1, collecting data: no
Control loop runtime (us): min=5, max=32, frequency (Hz): min=10563, max=17045
Driver 0: pos 0, 80.0 steps/mm,not tuned/calibrated, SG min n/a, mspos 936, reads 27495, writes 20 timeouts 0, steps req 0 done 0
Moves scheduled 0, completed 0, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0
Peak sync jitter 0/11, peak Rx sync delay 203, resyncs 0/0, next timer interrupt due in 15 ticks, enabled, next step interrupt due in 3707792031 ticks, disabled
VIN voltage: min 24.4, current 24.4, max 24.4
V12 voltage: min 12.1, current 12.1, max 12.1
MCU temperature: min 31.3C, current 31.3C, max 34.2C
Last sensors broadcast 0x00000000 found 0 206 ticks ago, 0 ordering errs, loop time 0
CAN messages queued 9408, send timeouts 0, received 13293, lost 0, free buffers 37, min 37, error reg 0
dup 0, oos 0/0/0/0, bm 0, wbm 0, rxMotionDelay 0
Accelerometer: none
I2C bus errors 0, naks 0, other errors 0

IndeX4D

@Phaedrux ah I did not check the pi then.. I´ll do it
Should I do increase log level before?

Phaedrux

It would be good to increase the log level and then provoke the error.

IndeX4D

This post is deleted!

IndeX4D

@Phaedrux

Can you please a small advice how I can open DCS to increase the log level?
Thanks

IndeX4D

IndeX4D

This post is deleted!

IndeX4D

Printer is stucked every pause.
and gives the last info see picture

Pi still lives.
It´s doing this when get stucked.

IndeX4D

is that a normal output?

Phaedrux

I'm not sure. With using closed loop and beta firmware, we will need to wait and see what @dc42 and @chrishamm have to say.

chrishamm

@IndeX4D Please upgrade to 3.5.0-beta.3, send M122 using the G-code console after it gets stuck, and post the output here. That little excerpt from the debug log isn't really useful, I'll need the full log in case the problem persists with 3.5.0-beta.3 and M122 doesn't show anything significant.

IndeX4D

@chrishamm already is 3.5 beta3 and same behaviour.
How can I give you the full log?

Edit:
When I print with an older dwc version then 3.5 beta1, I don´t have this problem.

IndeX4D

tell me when you find your bugs.
I won´t investigate anymore.
Sadly...... when I have more time, I would like to help you more, but can't let my printers standing still for such a long time.
thanks

Phaedrux

@IndeX4D said in 2 SD Cards - SBC - stop after pause.g 3.5beta2:

When I print with an older dwc version then 3.5 beta1, I don´t have this problem.

Thanks. That's useful to know.

IndeX4D

@Phaedrux
I think I have the solution.
I had a resurect.g folder in my system fiels. I deleted it and now it´s fine. Is that a normal behaviour?

Noe, I see the problem after resume.g.
Is there any solution for it?

chrishamm

@IndeX4D Is it DWC (web UI) that hangs or the printer itself, i.e. does refreshing the web page fix it?

If it is DWC, please have a look at the JS Console after pressing F12 in DWC once it's frozen and check for potential errors. If there are any errors, please share them here via copy&paste.

If it is not DWC, please share a full DCS log - all your uploaded logs are incomplete. To obtain that, do the following once the problem has occurred again:

Connect via PuTTY over SSH to your machine
Run journalctl -u duetcontrolserver --no-pager --since today
Right-click on the PuTTY window bar and click "Copy All to Clipboard"
Open Notepad and paste the contents into it
Save the file and upload it here

I am still unable to reproduce it myself here.

PS: Don't forget to change back the log level from debug to info when you're done.

IndeX4D

@chrishamm

´´it´´? means the freezing/stuck when press pause button or the slow down after resume? what you wanna reproduce yourself?
because I can work with the slow down for a while.

Crazy behaviour also: When resuming, sometimes the printer abbreviate some corners. Never seen it before. That´s also better then stucked and dead.

Do I have to delet resurect.g when printing new print or not?
I just wanna know, if it was my fault.....

DWC is showing actual temperatures, but I can´t make anything. No fans etc
Seems it´s wait for something.

but Pi is still working.

IndeX4D

This post is deleted!