Intermittent communication disruption between 6HC and 3HC
-
I'm almost done printing the last order, so I'll have more time to experiment tomorrow. I'll report afterwards.
Thanks for being awesome and helping out!
-
Quick update.
I'm 5 hours into the print, so far the print has not failed, no skipped steps or underextrusion, etc. The 3HC starts flashing quickly immediately, catches sync within 2-3 seconds.
There is one more test I need to do - if all goes well, I'll have it done tomorrow morning - that's to check how the 3HC behaves after a long power off (at least one hour). For now I've had the printer running pretty much 24/7 since Saturday.
I ran a quick M122 just in case you want to see what's going on right now (pasting below). The assertion failure is still there and DHT22 does not work.
Diagnostics for board 2: Duet EXP3HC firmware version 3.3beta (2021-02-01 22:29:11) Bootloader ID: not available Never used RAM 154972, free system stack 0 words Move 80 HEAT 78 CanAsync 72 CanRecv 82 CanClock 74 TMC 30 MAIN 263 AIN 260 Last reset 12:15:33 ago, cause: software Last software reset time unknown, reason: AssertionFailed, available RAM 173348, slot 0 Software reset code 0x0120 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0445f85f BFAR 0xe000ed38 SP 0x2002ff8c Task MAIN Freestk 4294967295 ok Stack: 000002f2 0002a9b0 00020f41 00000000 0002256d 20002ec8 20003100 00000031 00000001 20002980 03800209 0001a72d 0001a709 00000000 00000000 ffffffff 0001de4b 200022d8 20002330 00029c48 ffffffed 00000000 00f00000 e000ef34 c0000000 200041dc 00020ea5 Driver 0: position 19481636, 410.0 steps/mm, ok, reads 39334, writes 0 timeouts 0, SG min/max 0/83, steps req 354599 done 354486 Driver 1: position 0, 80.0 steps/mm, standstill, reads 39334, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Driver 2: position 0, 80.0 steps/mm, standstill, reads 39334, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Moves scheduled 966506, completed 966505, in progress 1, hiccups 0, step errors 0, maxPrep 59, maxOverdue 3, maxInc 3, mcErrs 0, gcmErrs 0 Peak sync jitter 10, peak Rx sync delay 177, resyncs 0, next step interrupt due in 1138 ticks, enabled VIN: 24.1V, V12: 12.1V MCU temperature: min 45.7C, current 45.9C, max 46.1C Ticks since heat task active 16, ADC conversions started 44133758, completed 44133757, timed out 0 Last sensors broadcast 0x00000000 found 0 22 ticks ago, loop time 0 CAN messages queued 1648, send timeouts 0, received 11585, lost 0, free buffers 36, min 36, error reg 0 dup 0, oos 0, bm 0, wbm 0=== Diagnostics === RepRapFirmware for Duet 3 MB6HC version 3.3beta running on Duet 3 MB6HC v0.6 or 1.0 (standalone mode) Board ID: 08DJM-956L2-G43S4-6JKDA-3SJ6T-1B6GH Used output buffers: 1 of 40 (40 max) === RTOS === Static ram: 149800 Dynamic ram: 92100 of which 72 recycled Never used RAM 108700, free system stack 118 words Tasks: NETWORK(ready,228) ETHERNET(blocked,117) SENSORS(blocked,15) HEAT(blocked,280) CanReceiv(blocked,877) CanSender(blocked,337) CanClock(blocked,326) TMC(blocked,16) MAIN(running,616) IDLE(ready,20) Owned mutexes: HTTP(MAIN) === Platform === Last reset 12:15:29 ago, cause: software Last software reset at 2021-02-02 09:54, reason: User, GCodes spinning, available RAM 108908, slot 0 Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0x00000000 Task MAIN Freestk 0 n/a Error status: 0x04 Aux0 errors 331,331,331 Aux1 errors 0,0,0 MCU temperature: min 43.7, current 45.2, max 52.1 Supply voltage: min 23.9, current 24.0, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes 12V rail voltage: min 12.0, current 12.1, max 12.2, under voltage events: 0 Driver 0: position 27530, ok, reads 12882, writes 38 timeouts 0, SG min/max 0/1023 Driver 1: position 2814, ok, reads 12882, writes 38 timeouts 0, SG min/max 0/1023 Driver 2: position 2725, ok, reads 12882, writes 38 timeouts 0, SG min/max 0/1023 Driver 3: position 0, ok, reads 12882, writes 38 timeouts 0, SG min/max 0/1023 Driver 4: position 0, ok, reads 12883, writes 38 timeouts 0, SG min/max 0/1023 Driver 5: position 0, ok, reads 12883, writes 38 timeouts 0, SG min/max 0/1023 Date/time: 2021-02-02 22:09:51 Slowest loop: 211.65ms; fastest: 0.04ms === Storage === Free file entries: 9 SD card 0 detected, interface speed: 25.0MBytes/sec SD card longest read time 4.0ms, write time 152.3ms, max retries 0 === Move === DMs created 125, maxWait 514613ms, bed compensation in use: mesh, comp offset 0.000 === MainDDARing === Scheduled moves 458647, completed moves 458612, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 15], CDDA state 3 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 Heater 0 is on, I-accum = 0.0 Heater 1 is on, I-accum = 0.4 === GCodes === Segments left: 1 Movement lock held by null HTTP is ready with "m122 b0" in state(s) 0 Telnet is idle in state(s) 0 File is doing "G1 X190.481 Y213.861 E0.00113" in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === CAN === Messages queued 1143678, send timeouts 0, received 177174, lost 0, longest wait 21ms for reply type 6024, peak Tx sync delay 468, free buffers 48 (min 33) === Network === Slowest loop: 574.55ms; fastest: 0.02ms Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions HTTP sessions: 1 of 8 - Ethernet - State: active Error counts: 0 0 1 0 0 Socket states: 5 2 2 2 2 0 0 0 -
I'm sorry to be the bearer of bad news, but it's much worse now. I don't know if it's related in any way to the firmware shuffling of late, but now, I can't seem to be able to get a single print off successfully that takes longer than 8 hours.
Almost immediately after my post last night, I noticed that the printer stopped moving and paused on it's own. There were no messages in the console. The printer was just stopped above the print.
I hit resume and the printer seemed to try to move the carriage, but was unable to. It was just stuck in place and jittering around. The display on PanelDue showed significant changes in location, but there was only jittery motion.
I aborted the print and started another, fresh gcode again in hopes that it's just a fluke.
Alas, it was not. A night print that was supposed to take 9 hours froze sometime around the 7th hour (I can't tell for sure).
This time I grabbed diag logs from before I hit resume and after. I'm pasting them below along with my current config.g file.
To be sure - after I hit emergency stop on the panel, everything was moving perfectly fine, so it wasn't something mechanical that would prevent the printer from moving. I did notice that the quality of the print is much worse on the z axis - something I never had an issue with before.
I am now getting very worried that something more serious broke and will have the printer down longer.
Do the logs tell you anything?
Before resume:
m122 b0 === Diagnostics === RepRapFirmware for Duet 3 MB6HC version 3.3beta running on Duet 3 MB6HC v0.6 or 1.0 (standalone mode) Board ID: 08DJM-956L2-G43S4-6JKDA-3SJ6T-1B6GH Used output buffers: 1 of 40 (40 max) === RTOS === Static ram: 149800 Dynamic ram: 92100 of which 72 recycled Never used RAM 108700, free system stack 118 words Tasks: NETWORK(ready,228) ETHERNET(blocked,117) SENSORS(blocked,15) HEAT(blocked,274) CanReceiv(blocked,877) CanSender(blocked,337) CanClock(blocked,326) TMC(blocked,16) MAIN(running,616) IDLE(ready,20) Owned mutexes: HTTP(MAIN) === Platform === Last reset 21:54:30 ago, cause: software Last software reset at 2021-02-02 09:54, reason: User, GCodes spinning, available RAM 108908, slot 0 Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0x00000000 Task MAIN Freestk 0 n/a Error status: 0x04 Aux0 errors 0,0,0 Aux1 errors 0,0,0 MCU temperature: min 44.1, current 44.7, max 45.4 Supply voltage: min 21.6, current 24.0, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes 12V rail voltage: min 12.0, current 12.1, max 12.2, under voltage events: 0 Driver 0: position 17407, standstill, reads 36452, writes 0 timeouts 0, SG min/max 0/1023 Driver 1: position 276, standstill, reads 36452, writes 0 timeouts 0, SG min/max 0/1023 Driver 2: position 8318, standstill, reads 36453, writes 0 timeouts 0, SG min/max 0/1023 Driver 3: position 0, standstill, reads 36453, writes 0 timeouts 0, SG min/max 0/1023 Driver 4: position 0, standstill, reads 36450, writes 2 timeouts 0, SG min/max 0/1023 Driver 5: position 0, standstill, reads 36450, writes 2 timeouts 0, SG min/max 0/1023 Date/time: 2021-02-03 07:48:52 Slowest loop: 49.84ms; fastest: 0.05ms === Storage === Free file entries: 9 SD card 0 detected, interface speed: 25.0MBytes/sec SD card longest read time 3.4ms, write time 0.8ms, max retries 0 === Move === DMs created 125, maxWait 0ms, bed compensation in use: mesh, comp offset 0.000 === MainDDARing === Scheduled moves 935235, completed moves 935235, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 1], CDDA state -1 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 Heater 0 is on, I-accum = 0.0 Heater 1 is on, I-accum = 0.3 === GCodes === Segments left: 0 Movement lock held by null HTTP is ready with "m122 b0" in state(s) 0 Telnet is idle in state(s) 0 File is doing "G1 X110.426 Y107.065 E0.07748 F1134.2" in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === CAN === Messages queued 494969, send timeouts 0, received 140596, lost 0, longest wait 21ms for reply type 6024, peak Tx sync delay 468, free buffers 48 (min 42) === Network === Slowest loop: 10.04ms; fastest: 0.02ms Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions HTTP sessions: 1 of 8 - Ethernet - State: active Error counts: 0 0 1 0 0 Socket states: 2 2 2 2 2 0 0 0 m122 b2 Diagnostics for board 2: Duet EXP3HC firmware version 3.3beta (2021-02-01 22:29:11) Bootloader ID: not available Never used RAM 154972, free system stack 0 words Move 80 HEAT 78 CanAsync 72 CanRecv 82 CanClock 74 TMC 30 MAIN 263 AIN 260 Last reset 21:54:31 ago, cause: software Last software reset time unknown, reason: AssertionFailed, available RAM 173348, slot 0 Software reset code 0x0120 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0445f85f BFAR 0xe000ed38 SP 0x2002ff8c Task MAIN Freestk 4294967295 ok Stack: 000002f2 0002a9b0 00020f41 00000000 0002256d 20002ec8 20003100 00000031 00000001 20002980 03800209 0001a72d 0001a709 00000000 00000000 ffffffff 0001de4b 200022d8 20002330 00029c48 ffffffed 00000000 00f00000 e000ef34 c0000000 200041dc 00020ea5 Driver 0: position 26155512, 410.0 steps/mm, standstill, reads 45397, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Driver 1: position 0, 80.0 steps/mm, standstill, reads 45395, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Driver 2: position 0, 80.0 steps/mm, standstill, reads 45396, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Moves scheduled 1320929, completed 1320929, in progress 0, hiccups 0, step errors 0, maxPrep 0, maxOverdue 0, maxInc 0, mcErrs 0, gcmErrs 0 Peak sync jitter 9, peak Rx sync delay 173, resyncs 0, no step interrupt scheduled VIN: 24.1V, V12: 12.1V MCU temperature: min 45.7C, current 45.9C, max 46.1C Ticks since heat task active 184, ADC conversions started 78871426, completed 78871425, timed out 0 Last sensors broadcast 0x00000000 found 0 190 ticks ago, loop time 0 CAN messages queued 423, send timeouts 0, received 885, lost 0, free buffers 36, min 36, error reg 0 dup 0, oos 0, bm 0, wbm 0After resume (and about 5 seconds of the carriage jittering):
m122 b0 Diagnostics for board 2: Duet EXP3HC firmware version 3.3beta (2021-02-01 22:29:11) Bootloader ID: not available Never used RAM 154972, free system stack 0 words Move 80 HEAT 78 CanAsync 72 CanRecv 82 CanClock 74 TMC 30 MAIN 263 AIN 260 Last reset 21:56:48 ago, cause: software Last software reset time unknown, reason: AssertionFailed, available RAM 173348, slot 0 Software reset code 0x0120 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0445f85f BFAR 0xe000ed38 SP 0x2002ff8c Task MAIN Freestk 4294967295 ok Stack: 000002f2 0002a9b0 00020f41 00000000 0002256d 20002ec8 20003100 00000031 00000001 20002980 03800209 0001a72d 0001a709 00000000 00000000 ffffffff 0001de4b 200022d8 20002330 00029c48 ffffffed 00000000 00f00000 e000ef34 c0000000 200041dc 00020ea5 Driver 0: position 26160116, 410.0 steps/mm, standstill, reads 46672, writes 0 timeouts 0, SG min/max 0/192, steps req 22610 done 22610 Driver 1: position 0, 80.0 steps/mm, standstill, reads 46672, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Driver 2: position 0, 80.0 steps/mm, standstill, reads 46671, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Moves scheduled 1321401, completed 1321401, in progress 0, hiccups 0, step errors 0, maxPrep 59, maxOverdue 1, maxInc 1, mcErrs 0, gcmErrs 0 Peak sync jitter 10, peak Rx sync delay 173, resyncs 0, no step interrupt scheduled VIN: 24.1V, V12: 12.1V MCU temperature: min 45.7C, current 45.9C, max 46.1C Ticks since heat task active 96, ADC conversions started 79008838, completed 79008837, timed out 0 Last sensors broadcast 0x00000000 found 0 102 ticks ago, loop time 0 CAN messages queued 588, send timeouts 0, received 1722, lost 0, free buffers 36, min 36, error reg 0 dup 0, oos 0, bm 0, wbm 0 === Diagnostics === RepRapFirmware for Duet 3 MB6HC version 3.3beta running on Duet 3 MB6HC v0.6 or 1.0 (standalone mode) Board ID: 08DJM-956L2-G43S4-6JKDA-3SJ6T-1B6GH Used output buffers: 15 of 40 (40 max) === RTOS === Static ram: 149800 Dynamic ram: 92100 of which 72 recycled Never used RAM 108700, free system stack 118 words Tasks: NETWORK(ready,228) ETHERNET(blocked,117) SENSORS(blocked,15) HEAT(blocked,274) CanReceiv(blocked,877) CanSender(blocked,337) CanClock(blocked,326) TMC(blocked,16) MAIN(running,616) IDLE(ready,20) Owned mutexes: HTTP(MAIN) === Platform === Last reset 21:56:48 ago, cause: software Last software reset at 2021-02-02 09:54, reason: User, GCodes spinning, available RAM 108908, slot 0 Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0x00000000 Task MAIN Freestk 0 n/a Error status: 0x04 Aux0 errors 0,0,0 Aux1 errors 0,0,0 MCU temperature: min 44.5, current 44.8, max 45.0 Supply voltage: min 23.9, current 24.0, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes 12V rail voltage: min 12.1, current 12.1, max 12.2, under voltage events: 0 Driver 0: position 80, standstill, reads 50864, writes 0 timeouts 0, SG min/max 0/990 Driver 1: position 80, standstill, reads 50864, writes 0 timeouts 0, SG min/max 0/1023 Driver 2: position 9190, standstill, reads 50863, writes 0 timeouts 0, SG min/max 0/1023 Driver 3: position 0, standstill, reads 50863, writes 0 timeouts 0, SG min/max 0/1023 Driver 4: position 0, standstill, reads 50863, writes 0 timeouts 0, SG min/max 19/1023 Driver 5: position 0, standstill, reads 50863, writes 0 timeouts 0, SG min/max 16/1023 Date/time: 2021-02-03 07:51:10 Slowest loop: 49.77ms; fastest: 0.04ms === Storage === Free file entries: 9 SD card 0 detected, interface speed: 25.0MBytes/sec SD card longest read time 3.3ms, write time 0.8ms, max retries 0 === Move === DMs created 125, maxWait 0ms, bed compensation in use: mesh, comp offset 0.000 === MainDDARing === Scheduled moves 935861, completed moves 935861, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 Heater 1 is on, I-accum = 0.4 === GCodes === Segments left: 0 Movement lock held by null HTTP is ready with "m122 b0" in state(s) 0 Telnet is idle in state(s) 0 File is doing "G1 X91.532 Y86.855 E0.01315" in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === CAN === Messages queued 1037, send timeouts 0, received 590, lost 0, longest wait 20ms for reply type 6024, peak Tx sync delay 468, free buffers 48 (min 46) === Network === Slowest loop: 3.99ms; fastest: 0.02ms Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions HTTP sessions: 2 of 8 - Ethernet - State: active Error counts: 0 0 1 0 0 Socket states: 2 2 2 2 2 0 0 0 m122 b2 Diagnostics for board 2: Duet EXP3HC firmware version 3.3beta (2021-02-01 22:29:11) Bootloader ID: not available Never used RAM 154972, free system stack 0 words Move 80 HEAT 78 CanAsync 72 CanRecv 82 CanClock 74 TMC 30 MAIN 263 AIN 260 Last reset 21:56:48 ago, cause: software Last software reset time unknown, reason: AssertionFailed, available RAM 173348, slot 0 Software reset code 0x0120 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0445f85f BFAR 0xe000ed38 SP 0x2002ff8c Task MAIN Freestk 4294967295 ok Stack: 000002f2 0002a9b0 00020f41 00000000 0002256d 20002ec8 20003100 00000031 00000001 20002980 03800209 0001a72d 0001a709 00000000 00000000 ffffffff 0001de4b 200022d8 20002330 00029c48 ffffffed 00000000 00f00000 e000ef34 c0000000 200041dc 00020ea5 Driver 0: position 26160116, 410.0 steps/mm, standstill, reads 46672, writes 0 timeouts 0, SG min/max 0/192, steps req 22610 done 22610 Driver 1: position 0, 80.0 steps/mm, standstill, reads 46672, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Driver 2: position 0, 80.0 steps/mm, standstill, reads 46671, writes 0 timeouts 0, SG min/max not available, steps req 0 done 0 Moves scheduled 1321401, completed 1321401, in progress 0, hiccups 0, step errors 0, maxPrep 59, maxOverdue 1, maxInc 1, mcErrs 0, gcmErrs 0 Peak sync jitter 10, peak Rx sync delay 173, resyncs 0, no step interrupt scheduled VIN: 24.1V, V12: 12.1V MCU temperature: min 45.7C, current 45.9C, max 46.1C Ticks since heat task active 96, ADC conversions started 79008838, completed 79008837, timed out 0 Last sensors broadcast 0x00000000 found 0 102 ticks ago, loop time 0 CAN messages queued 588, send timeouts 0, received 1722, lost 0, free buffers 36, min 36, error reg 0 dup 0, oos 0, bm 0, wbm 0; Configuration file for Duet 3 (firmware version 3) ; executed by the firmware on start-up ; ; generated by RepRapFirmware Configuration Tool v3.1.3 on Sun Jul 12 2020 20:53:52 GMT+0200 (Central European Summer Time) ; General preferences G90 ; send absolute coordinates... M83 ; ...but relative extruder moves M550 P"Voron" ; set printer name M669 K1 ; select CoreXY mode ; Network M552 P0.0.0.0 S1 ; enable network and acquire dynamic address via DHCP M586 P0 S1 ; enable HTTP M586 P1 S0 ; disable FTP M586 P2 S0 ; disable Telnet ; Drives for XY M569 P0.4 S1 ; A motor M569 P0.5 S1 ; B motor ; Drives for Z M569 P0.0 S0 ; Front left M569 P0.1 S1 ; Back left M569 P0.2 S0 ; Back right M569 P0.3 S1 ; Front right ; Wait before addressing the 3HC ; G4 S25 ; Drive for extruder M569 P2.0 S1 ; physical drive 1.0 goes forwards ; Drive mappings M584 X0.4 Y0.5 Z0.0:0.1:0.2:0.3 E2.0 ; set drive mapping ; General drive config like speeds, accel, jerk, etc M350 X16 Y16 Z16 E16 I1 ; configure microstepping with interpolation M92 X80.00 Y80.00 Z400.00 E410.00 ; set steps per mm M566 X300.00 Y300.00 Z300.00 E120.00 ; set maximum instantaneous speed changes (mm/min) M203 X18000.00 Y18000.00 Z2100 E1200 ; set maximum speeds (mm/min) M201 X3000.00 Y3000.00 Z350.00 E250.00 ; set accelerations (mm/s^2) M906 X900 Y900 Z1000 E600 I30 ; set motor currents (mA) and motor idle factor in per cent M84 S30 ; Set idle timeout ; Axis Limits M208 X1 Y0 Z0 S1 ; set axis minima M208 X301 Y305 Z300 S0 ; set axis maxima ; Endstops M574 X2 S1 P"!io1.in" ; X Hall effect endstop M574 Y2 S1 P"!io2.in" ; Y Hall effect endstop M574 Z1 S1 P"io3.in" ; Z microswitch endstop ; Z-Probe M558 P5 C"io4.in" I1 H5 F120 T12000 ; set Z probe type to switch and the dive height + speeds M671 X354:354:-53:-53 Y0:370:370:0 S20 ; Define Z belts locations (Front_Left, Back_Left, Back_Right, Front_Right) G31 P500 X0 Y25 Z0.85 ; set Z probe trigger value, offset and trigger height M557 X15:285 Y25:275 S40 ; define mesh grid ; Hotbed M308 S0 P"temp0" Y"thermistor" T100000 B4138 ; Thermistor M950 H0 C"out0" T0 ; Heater M307 H0 A157.8 C632 D1.4 V24.0 S0.6 B0 ; Heater 0 model: gain 157.8, time constant 632.0, dead time 1.4, max PWM 0.60, calibration voltage 24.0, mode PID M140 H0 ; Map heated bed to heater 0 M143 H0 S120 ; Set temperature limit for heater 0 to 120C ; Hotend M308 S1 P"temp1" Y"thermistor" T100000 B4138 ; Thermistor M950 H1 C"out1" T1 ; Heater ; Mosquitto M307 H1 A593.7 C180.4 D7.6 V24.0 B0 ; Heater 1 model: gain 593.7, time constant 180.4, dead time 7.6, max PWM 1.00, calibration voltage 24.0, mode PID ; Mosquitto HF ; M307 H1 A408.1 C161.3 D4.8 V24.0 B0 ; Heater 1 model: gain 408.1, time constant 161.3, dead time 4.8, max PWM 1.00, calibration voltage 24.0, mode PID ; DHT22 sensor M308 S10 P"0.spi.cs1" Y"dht22" A"Chamber Temp" ; Temperature (connected to cs0 port on the temp daughterboard slot M308 S11 P"S10.1" Y"dhthumidity" A"Chamber Hum[%]" ; Humidity ; Main fans M950 F0 C"2.out6" Q500 ; Part cooling fan M106 P0 S0 H-1 C"Part Fan" ; Default off; thermostatic control off M950 F1 C"out8" Q500 ; Hotend cooling fan ;M106 P1 S1 H-1 ; Default off; thermostatic control off M106 P1 S1 H1 T45 C"Hotend Fan" ; Default on; thermistatic control on above 45C ; Aux fans (option to control daughter board fans based on main board temp sensor coming in RRF3.3, for now has to be manual) M950 F2 C"2.out7" Q500 ; Electronics compartment fan 1 M106 P2 S0 H-1 C"Electronics Fan 1" ; Default off; thermostatic control off M950 F3 C"2.out8" Q500 ; Electronics compartment fan 2 M106 P3 S0 H-1 C"Electronics Fan 2" ; Default off; thermostatic control off M950 F4 C"!out4+out4.tach" ; Noctua exhaust filter fan M106 P4 S0 H-1 C"Exhaust Fan" ; Default off ; Tools M563 P0 D0 H1 F0 ; define tool 0 G10 P0 X0 Y0 Z0 ; set tool 0 axis offsets G10 P0 R0 S0 ; set initial tool 0 active and standby temperatures to 0C ; Push buttons and their config M950 J5 C"io5.in" ; Left button - light M581 P5 T5 ; when button pushed, execute trigger5.g M950 J6 C"io7.in" ; Left button - heat up M950 F5 C"io6.out" ; configure the relay as fan M106 F5 S0 H-1 C"Light" M581 P6 T6 ; When pushed - trigger relay switch on io6.out ; Custom settings are not defined ; Miscellaneous M911 S22 R23 P"M913 X0 Y0 G91 M83 G1 Z3 E-5 F1000" ; set voltage thresholds and actions to run on power loss M575 P1 S1 B57600 ; enable support for PanelDue -
@pkos, thanks for your reports. I'm sorry that you have lost two large prints.
As the head movement was not correct, the problem appears to be occurring on the main board, not on the EXP3HC.
Can you share one of those GCode files, so that I can run it here?
I am about to prepare a 3.2.1 build of the EXP3HC firmware. This should allow you to revert to RRF 3.2 without experiencing the slow startup issue on the EXP3HC.
-
Here we go then.
This is the file that I tried printing since last night where it failed both times.
Great news on the 3.2.1
I know I am probably getting annoying by now, but I'm one of those persistent people. The assertion failure is still there and I see that the bootloader ID is not there. Is this as it should be? Or maybe 3.2.1 will solve the problem?
EDIT: One more note - both prints failed at a different point in the gcode. One much earlier than the second one.
-
@pkos said in Intermittent communication disruption between 6HC and 3HC:
I know I am probably getting annoying by now, but I'm one of those persistent people. The assertion failure is still there and I see that the bootloader ID is not there. Is this as it should be?
The assertion failure report is the saved info from the crash when you were running older firmware. This is clear because the free memory and stack trace are exactly the same as before; whereas if the same crash happened again now, at the very least the free memory would be lower. What's happening is that "normal" software resets such as when you installed the new firmware did not clear out the data.
If you wish, you can replace the software reset data by sending M122 B2 P1004 to provoke a deliberate reset. The software reset data will then report a deliberate zero divide error.
The bootloader ID is not reported because older bootloaders do not report their ID. But please do not update the bootloader at this stage.
Thanks for providing the GCode file. I will run this on the bench to see if I can replicate the problem.
-
And now it's perfectly clear - thank you
I am much more calm now.For now, I'm back on 3.1.1 running prints - so far so good.
I'll wait for the 3.2.1 to see if that helps at all.Then the last thing to solve for me will be the DHT22. 3.1.1 - works perfectly. Switch to 3.2 or 3.3 - nothing. Values jump to 2k.
-
@pkos said in Intermittent communication disruption between 6HC and 3HC:
Then the last thing to solve for me will be the DHT22. 3.1.1 - works perfectly. Switch to 3.2 or 3.3 - nothing. Values jump to 2k.
This is on my list to investigate.
-
The candiate EXP3HC 3.2.1 firmware is now avaiable at https://www.dropbox.com/sh/1lwimb98k6hzz3z/AAApVr_P6roUjnya4riDbGAba?dl=0. This includes the fix to the slow startup issue (LED not starting to blink immediately after power up).
Have you run any long prints using RRF 3.2 yet?
-
Not yet. For now I quickly printed the rest of functional parts in small batches, but I will give the new firmware a try.
-
@pkos said in Intermittent communication disruption between 6HC and 3HC:
Not yet. For now I quickly printed the rest of functional parts in small batches, but I will give the new firmware a try.
OK, in that case bear in mind that 3.2 might have the same issue that you saw with 3.3. You might want to try an air print first to save on filament.
-
Will do.
Quick question - do I need to redo PID for 3.2? I thought I saw that there is a new algorithm, but it's not necessary to redo PID.
I am seeing some very weird behavior on 3.2. Regardless of the temperature I set on the bed, it drops temps without the firmware screaming bloody murder and thermal runaway.
Attaching screenshot. Please note that the temp is 100C, while it's set at 105C.
Hotend temp dropping was my doing.Just in case, log from the main board below.
m122 b0 === Diagnostics === RepRapFirmware for Duet 3 MB6HC version 3.2 running on Duet 3 MB6HC v0.6 or 1.0 (standalone mode) Board ID: 08DJM-956L2-G43S4-6JKDA-3SJ6T-1B6GH Used output buffers: 1 of 40 (21 max) === RTOS === Static ram: 149788 Dynamic ram: 94408 of which 76 recycled Never used RAM 114560, free system stack 129 words Tasks: NETWORK(ready,189) ETHERNET(blocked,109) SENSORS(blocked,19) HEAT(blocked,289) CanReceiv(blocked,848) CanSender(blocked,341) CanClock(blocked,352) TMC(blocked,19) MAIN(running,717) IDLE(ready,19) Owned mutexes: HTTP(MAIN) === Platform === Last reset 00:29:42 ago, cause: power up Last software reset at 2021-02-03 19:30, reason: User, GCodes spinning, available RAM 114560, slot 2 Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0x00000000 Task MAIN Freestk 0 n/a Error status: 0x00 Aux0 errors 0,0,0 Aux1 errors 0,0,0 MCU temperature: min 37.9, current 44.5, max 46.5 Supply voltage: min 23.9, current 24.0, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes 12V rail voltage: min 12.1, current 12.1, max 12.2, under voltage events: 0 Driver 0: position 23945, ok, reads 31637, writes 21 timeouts 0, SG min/max 0/1023 Driver 1: position 3358, ok, reads 31637, writes 21 timeouts 0, SG min/max 0/1023 Driver 2: position 8994, ok, reads 31637, writes 21 timeouts 0, SG min/max 0/1023 Driver 3: position 0, ok, reads 31638, writes 21 timeouts 0, SG min/max 0/1023 Driver 4: position 0, ok, reads 31638, writes 21 timeouts 0, SG min/max 0/1023 Driver 5: position 0, ok, reads 31638, writes 21 timeouts 0, SG min/max 0/1023 Date/time: 2021-02-03 20:01:50 Slowest loop: 44.47ms; fastest: 0.05ms === Storage === Free file entries: 9 SD card 0 detected, interface speed: 25.0MBytes/sec SD card longest read time 3.4ms, write time 0.0ms, max retries 0 === Move === DMs created 125, maxWait 89884ms, bed compensation in use: mesh, comp offset 0.000 === MainDDARing === Scheduled moves 70080, completed moves 70020, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state 3 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 Heater 0 is on, I-accum = 0.2 Heater 1 is on, I-accum = 0.3 === GCodes === Segments left: 1 Movement lock held by null HTTP is ready with "m122 b0" in state(s) 0 Telnet is idle in state(s) 0 File is doing "G1 X178.308 Y129.689 E0.02381" in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === Network === Slowest loop: 79.63ms; fastest: 0.02ms Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions HTTP sessions: 1 of 8 - Ethernet - State: active Error counts: 0 0 1 0 0 Socket states: 2 5 2 2 2 0 0 0 === CAN === Messages queued 73379, send timeouts 0, received 7629, lost 0, longest wait 5ms for reply type 6029, free buffers 48 -
@pkos I would say it's worth trying the new PID tuning. The old values should be usable though.
-
I'll give another PID tune a try then just in case.
Still... weird that firmware didn't alert me at once that the temperature is not rising fast enough - especially since it was below the set temp and actually very slowly dropping - you don't see it on the screenshot above, but by the end of that print, temp was sub 100C. -
You don't have to re-run PID tuning for 3.2 if you are happy with the PID performance under 3.1.1. You do need to re-tune if you want the benefit of hot end heater feedforward to compensate for changes in the speed of the print cooling fan.
-
Update from today.
I ran two tests today on 3.2 (6HC) and 3.2.1 (3HC).
One was a print and generic sync test, the other was a sync test after a long power down.Quick sync, but print failed on 3.2 and 3.2.1
There was no problem with sync. That caught on pretty much immediately.
Print failed very quickly - after about 5 layers, I heard lost steps and the print was shifted. I stopped the print and turned the device off for the second part of the test.Instant sync after a long power down.
I left the printer off for a good 6 hours. Now I came back and turned it on - it seemed almost as if both boards came up at the same time and synced up even before the rest of the system launched.So I guess the sync problem can be considered solved.
A new symptom that I never had before is the behavior of the hotbed. Historically, this printer has been running just fine since May - often printing long prints, never had issues with temperature. I guess it's just coincidence that now I keep seeing so many issues
The problem is that after about an hour of the hotbed being heated up to ABS temps (about 105C) - I notice that the thermistor starts reporting dropping values, but the firmware doesn't complain about it. One screenshot was posted above. Today I had this happen a couple times. I now need to figure out what is going on there. I'm guessing a hardware issue, but I am curious why the firmware doesn't complain.
My hotbed is an 8mm alu plate with an AC Keenovo mat on it (it has the thermistor embedded in it). That mat is connected to the Duet 6HC via a Crydom SSR. There is a thermal fuse between the Crydom and the Keenovo mat (it triggers at 125C). The PSU is a proper Mean Well (bought from TME) not a noname PSU.
If you want me to shift to a new topic to track this, I'll be happy to, but for now - I'll ask here.
Is there a way to see what is being sent to the hotbed output from the 6HC? Forgive the comparison, but there is one nice feature in Klipper and Mainsail that would help debugging here. They show that data in their GUI (screenshot below). I'm wondering if there's a way to see that on a Duet with RRF - this would help debugging a lot since I could see what/if anything is sent to the board when the temps start dropping and how the numbers compare to the time, when temp is held steady at the preset temperature.
Is this kind of behavior normal? I set the temp of the hotbed to 105C, it gets there fine and stays there for a while, but - I see temps start dropping to 100C, but the firmware doesn't complain? Even if I set the temperature to 120C - temps do not rise, but keep falling. It's very slow (1C over 10 or so minutes), but still - it is falling.
I did run a new PID tune on the bed for the temps I usually print at (ABS, so ~105C). But that didn't eliminate the problem.
-
OK. So it seems the Keenovo mat is broken - or at least everything points to it being at fault regarding the temps.
What bugs me though is how come the firmware didn't object when instead of temp being on point, it started dropping...
-
@pkos said in Intermittent communication disruption between 6HC and 3HC:
What bugs me though is how come the firmware didn't object when instead of temp being on point, it started dropping...
The allowed temperature drop before a fault is registered is set by the M570 command. The default is 15C because when we tried setting it lower, too many users reported heater faults, mostly caused by print cooling fans blowing too much cold air at the heater block. I suggest you configure a lower maximum temperature drop, at least for the bed heater.
-
Understood. Thank you.
I have a new mat on the way (lucky to find someone wanting to get rid of theirs locally), but until I get it - I won't be able to do any more testing for the next couple of days.
-
@pkos said in Intermittent communication disruption between 6HC and 3HC:
Understood. Thank you.
I have a new mat on the way (lucky to find someone wanting to get rid of theirs locally), but until I get it - I won't be able to do any more testing for the next couple of days.
No problem.
I think the issue with missed steps you saw with firmware 3.2 was caused by a problem that I suspected to exist in firmware 3.2 on the MB6HC (caused by the code speed increase in 3.2 on MB6HC) which we fixed in 3.3. When we discovered this issue, I was surprised that we had no reports of missed steps with 3.2 on MB6HC. I guess you are the first user to use the high step rates on the MB6HC that are needed to trigger this problem.
I have back-ported the fix for this to release 3.2.1 which is currently undergoing testing.
