@dc42 Running a print successfully as we speak, after 24hr with no issues. Thank you for your work sorting this out!
Posts made by Fletcher
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
@dc42 Another VSSA fault chain on the latest firmware:
4/13/2021, 7:41:44 AM Error: VSSA fault, check thermistor wiring 4/13/2021, 7:41:44 AM m122 === Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.3beta2+1 (2021-04-12 08:36:30) running on Duet 3 Mini5plus Ethernet (SBC mode) Board ID: FDQG2-Q296U-D65J0-40KMG-1K03Z-HXFTZ Used output buffers: 1 of 40 (12 max) === RTOS === Static ram: 100072 Dynamic ram: 91764 of which 64 recycled Never used RAM 51804, free system stack 172 words Tasks: Linux(ready,137) HEAT(delaying,322) CanReceiv(notifyWait,774) CanSender(notifyWait,371) CanClock(delaying,340) TMC(notifyWait,99) MAIN(running,498) IDLE(ready,20) AIN(notifyWait,260) Owned mutexes: HTTP(MAIN) === Platform === Last reset 23:27:06 ago, cause: software Last software reset at 2021-04-12 16:13, reason: User, none spinning, available RAM 51856, slot 1 Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task Linu Freestk 0 n/a Error status: 0x00 Aux0 errors 0,0,0 Aux1 errors 0,0,0 MCU revision 3, ADC conversions started 100481305, completed 100481304, timed out 0, errs 1 tnd=0 rnd=0 wrd=0 wtd=1 war=1 Supply voltage: min 0.0, current 0.1, max 24.4, under voltage events: 1, over voltage events: 0, power good: no Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0 Driver 0: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 218, reads 4487, writes 11, timeouts 0, DMA errors 0 Driver 1: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 220, reads 4487, writes 11, timeouts 0, DMA errors 0 Driver 2: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 192, reads 4489, writes 9, timeouts 0, DMA errors 0 Driver 3: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 218, reads 4486, writes 11, timeouts 0, DMA errors 0 Driver 4: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 218, reads 4487, writes 11, timeouts 0, DMA errors 0 Driver 5: position 0, assumed not present Driver 6: position 0, assumed not present Date/time: 2021-04-13 15:41:41 Cache data hit count 4294967295 Slowest loop: 57.18ms; fastest: 0.08ms === Storage === Free file entries: 10 SD card 0 not detected, interface speed: 0.0MBytes/sec SD card longest read time 0.0ms, write time 0.0ms, max retries 0 === Move === DMs created 83, maxWait 0ms, bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1, chamberHeaters = -1 -1 === GCodes === Segments left: 0 Movement lock held by null HTTP* is doing "M122" in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger* is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === CAN === Messages queued 759838, send timeouts 0, received 1013078, lost 0, longest wait 2ms for reply type 6049, peak Tx sync delay 354, free buffers 17 (min 16) === SBC interface === State: 4, failed transfers: 0 Last transfer: 4ms ago RX/TX seq numbers: 14877/14877 SPI underruns 0, overruns 0 Number of disconnects: 0, IAP RAM available 0x118a8 Buffer RX/TX: 0/0-0 === Duet Control Server === Duet Control Server v3.3-b2 Code buffer space: 4096 Configured SPI speed: 8000000 Hz Full transfers per second: 35.89 Codes per second: 0.00 Maximum length of RX/TX data transfers: 3044/804 4/13/2021, 7:41:39 AM Error: VSSA fault, check thermistor wiring
It was stable for almost 24 hours, which is positive change, at least! Thank you for all the work you're putting in on this issue.
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
@dc42 More M122!
4/11/2021, 12:40:50 PM M122 === Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.3beta2+1 (2021-04-11 10:20:02) running on Duet 3 Mini5plus Ethernet (SBC mode) Board ID: FDQG2-Q296U-D65J0-40KMG-1K03Z-HXFTZ Used output buffers: 1 of 40 (16 max) === RTOS === Static ram: 100072 Dynamic ram: 91712 of which 64 recycled Never used RAM 51856, free system stack 172 words Tasks: Linux(ready,127) HEAT(delaying,322) CanReceiv(notifyWait,774) CanSender(notifyWait,371) CanClock(delaying,340) TMC(notifyWait,99) MAIN(running,415) IDLE(ready,20) AIN(notifyWait,260) Owned mutexes: HTTP(MAIN) === Platform === Last reset 04:43:27 ago, cause: software Last software reset at 2021-04-11 15:57, reason: User, none spinning, available RAM 51804, slot 2 Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task Linu Freestk 0 n/a Error status: 0x00 Aux0 errors 0,0,0 Aux1 errors 0,0,0 MCU revision 3, ADC conversions started 20273300, completed 20273299, timed out 0, errs 0 tnd=0 rnd=0 wrd=0 wtd=0 war=0 Supply voltage: min 0.0, current 36.1, max 36.1, under voltage events: 2, over voltage events: 0, power good: yes Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0 Driver 0: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 164, reads 49562, writes 18, timeouts 0, DMA errors 0 Driver 1: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 164, reads 49561, writes 18, timeouts 0, DMA errors 0 Driver 2: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 144, reads 49561, writes 18, timeouts 0, DMA errors 0 Driver 3: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 164, reads 49561, writes 18, timeouts 0, DMA errors 0 Driver 4: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 164, reads 49561, writes 18, timeouts 0, DMA errors 0 Driver 5: position 0, assumed not present Driver 6: position 0, assumed not present Date/time: 2021-04-11 20:40:50 Cache data hit count 4294967295 Slowest loop: 2.05ms; fastest: 0.11ms === Storage === Free file entries: 10 SD card 0 not detected, interface speed: 0.0MBytes/sec SD card longest read time 0.0ms, write time 0.0ms, max retries 0 === Move === DMs created 83, maxWait 0ms, bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1, chamberHeaters = -1 -1 Heater 1 is on, I-accum = 0.0 === GCodes === Segments left: 0 Movement lock held by null HTTP* is doing "M122" in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger* is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === CAN === Messages queued 126986, send timeouts 0, received 169317, lost 0, longest wait 0ms for reply type 0, peak Tx sync delay 348, free buffers 17 (min 17) === SBC interface === State: 4, failed transfers: 0 Last transfer: 4ms ago RX/TX seq numbers: 18024/18024 SPI underruns 0, overruns 0 Number of disconnects: 0, IAP RAM available 0x118dc Buffer RX/TX: 0/0-0 === Duet Control Server === Duet Control Server v3.3-b2 Code buffer space: 4096 Configured SPI speed: 8000000 Hz Full transfers per second: 35.72 Codes per second: 0.00 Maximum length of RX/TX data transfers: 3940/820 4/11/2021, 12:40:49 PM Error: VSSA fault, check thermistor wiring
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
@dc42 My machine is happy to oblige!
4/10/2021, 8:21:11 AM M122 === Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.3beta2+1 (2021-04-09 14:40:21) running on Duet 3 Mini5plus Ethernet (SBC mode) Board ID: FDQG2-Q296U-D65J0-40KMG-1K03Z-HXFTZ Used output buffers: 1 of 40 (13 max) === RTOS === Static ram: 100072 Dynamic ram: 91712 of which 64 recycled Never used RAM 51856, free system stack 172 words Tasks: Linux(ready,115) HEAT(delaying,322) CanReceiv(notifyWait,774) CanSender(notifyWait,371) CanClock(delaying,340) TMC(notifyWait,99) MAIN(running,374) IDLE(ready,20) AIN(notifyWait,260) Owned mutexes: HTTP(MAIN) === Platform === Last reset 18:32:25 ago, cause: software Last software reset at 2021-04-09 21:48, reason: User, none spinning, available RAM 51804, slot 0 Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00430000 BFAR 0xe000ed38 SP 0x00000000 Task Linu Freestk 0 n/a Error status: 0x00 Aux0 errors 0,0,0 Aux1 errors 0,0,0 MCU revision 3, ADC conversions started 79666166, completed 79666165, timed out 0, errs 4 tnd=0 rnd=0 wrs=0 wrx=4 war=4 Supply voltage: min 0.0, current 0.1, max 24.4, under voltage events: 2, over voltage events: 0, power good: no Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0 Driver 0: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 68, reads 52912, writes 0, timeouts 0, DMA errors 0 Driver 1: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 68, reads 52912, writes 0, timeouts 0, DMA errors 0 Driver 2: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 58, reads 52911, writes 0, timeouts 0, DMA errors 0 Driver 3: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 68, reads 52911, writes 0, timeouts 0, DMA errors 0 Driver 4: position 0, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 68, reads 52912, writes 0, timeouts 0, DMA errors 0 Driver 5: position 0, assumed not present Driver 6: position 0, assumed not present Date/time: 2021-04-10 16:21:10 Cache data hit count 4294967295 Slowest loop: 2.50ms; fastest: 0.09ms === Storage === Free file entries: 10 SD card 0 not detected, interface speed: 0.0MBytes/sec SD card longest read time 0.0ms, write time 0.0ms, max retries 0 === Move === DMs created 83, maxWait 0ms, bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1, chamberHeaters = -1 -1 Heater 1 is on, I-accum = 0.0 === GCodes === Segments left: 0 Movement lock held by null HTTP* is doing "M122" in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger* is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === CAN === Messages queued 450532, send timeouts 0, received 600717, lost 0, longest wait 0ms for reply type 0, peak Tx sync delay 345, free buffers 17 (min 17) === SBC interface === State: 4, failed transfers: 0 Last transfer: 3ms ago RX/TX seq numbers: 30955/30955 SPI underruns 0, overruns 0 Number of disconnects: 0, IAP RAM available 0x118dc Buffer RX/TX: 0/0-0 === Duet Control Server === Duet Control Server v3.3-b2 Code buffer space: 4096 Configured SPI speed: 8000000 Hz Full transfers per second: 35.82 Codes per second: 0.00 Maximum length of RX/TX data transfers: 3148/796 4/10/2021, 8:21:05 AM Error: VSSA fault, check thermistor wiring
Omitted: several hundred similar VSSA fault messages before this one. It was still repeating the error when I checked just now, and I had to
M999
reset to get the web interface responsive enough to copy-paste! -
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
M122 === Diagnostics === RepRapFirmware for Duet 3 Mini 5+ version 3.3beta2+1 (2021-04-09 14:40:21) running on Duet 3 Mini5plus Ethernet (SBC mode) Board ID: FDQG2-Q296U-D65J0-40KMG-1K03Z-HXFTZ Used output buffers: 1 of 40 (11 max) === RTOS === Static ram: 100072 Dynamic ram: 91712 of which 64 recycled Never used RAM 51856, free system stack 172 words Tasks: Linux(ready,115) HEAT(delaying,322) CanReceiv(notifyWait,774) CanSender(notifyWait,371) CanClock(delaying,340) TMC(notifyWait,99) MAIN(running,484) IDLE(ready,20) AIN(notifyWait,260) Owned mutexes: HTTP(MAIN) === Platform === Last reset 04:37:40 ago, cause: software Last software reset at 2021-04-09 21:48, reason: User, none spinning, available RAM 51804, slot 0 Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00430000 BFAR 0xe000ed38 SP 0x00000000 Task Linu Freestk 0 n/a Error status: 0x00 Aux0 errors 0,0,0 Aux1 errors 0,0,0 MCU revision 3, ADC conversions started 19812079, completed 19812078, timed out 0, errs 1 tnd=0 rnd=0 wrs=0 wrx=1 war=1 Supply voltage: min 0.0, current 24.3, max 24.4, under voltage events: 1, over voltage events: 0, power good: yes Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0 Driver 0: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 68, reads 20759, writes 20, timeouts 0, DMA errors 0 Driver 1: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 68, reads 20758, writes 20, timeouts 0, DMA errors 0 Driver 2: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 58, reads 20760, writes 18, timeouts 0, DMA errors 0 Driver 3: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 68, reads 20757, writes 20, timeouts 0, DMA errors 0 Driver 4: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 68, reads 20759, writes 20, timeouts 0, DMA errors 0 Driver 5: position 0, assumed not present Driver 6: position 0, assumed not present Date/time: 2021-04-10 02:26:25 Cache data hit count 4294967295 Slowest loop: 2.05ms; fastest: 0.12ms === Storage === Free file entries: 10 SD card 0 not detected, interface speed: 0.0MBytes/sec SD card longest read time 0.0ms, write time 0.0ms, max retries 0 === Move === DMs created 83, maxWait 0ms, bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1, chamberHeaters = -1 -1 Heater 1 is on, I-accum = 0.0 === GCodes === Segments left: 0 Movement lock held by null HTTP* is doing "M122" in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger* is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === CAN === Messages queued 149946, send timeouts 0, received 199896, lost 0, longest wait 2ms for reply type 6049, peak Tx sync delay 345, free buffers 17 (min 16) === SBC interface === State: 4, failed transfers: 0 Last transfer: 4ms ago RX/TX seq numbers: 6547/6547 SPI underruns 0, overruns 0 Number of disconnects: 0, IAP RAM available 0x118dc Buffer RX/TX: 0/0-0 === Duet Control Server === Duet Control Server v3.3-b2 Code buffer space: 4096 Configured SPI speed: 8000000 Hz Full transfers per second: 35.80 Codes per second: 0.00 Maximum length of RX/TX data transfers: 3148/796
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
Edit: After about 36 hours of uptime, I got the VSSA fault error too..
I have yet to experience issues on my idling machine. I'll probably try a print tonight and see how it goes.When I initially installed the upgrade, my extruder thermistor (which is on toolboard TEMP0) was reporting something absurd like 2000C, but turning my 24v PSU off and then on again fixed that.
It's been sitting idle with no errors for the last 22 hours.Setup: 120v wall power, 24V PSU, Raspberry Pi 3B+ with the Canakit PSU, Duet Web Control 3.3.0-b2, 5x motors, live thermistors for bed/hotend. Bed is a 12v bed connected via a MOSFET board, and it's connected to one of the GPIOs rather than the big high-current bed connector. M115 as follows from up-thread:
4/8/2021, 8:21:47 AM m115 B121 Duet TOOL1LC firmware version 3.3beta2+1 (2021-04-07 10:51:19) 4/8/2021, 8:21:41 AM M115 FIRMWARE_NAME: RepRapFirmware for Duet 3 Mini 5+ FIRMWARE_VERSION: 3.3beta2+1 ELECTRONICS: Duet 3 Mini5plus Ethernet FIRMWARE_DATE: 2021-04-07 13:33:17
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
@dc42 Thank you! I loaded your 7APR firmware at 10:33 AM local time today and left the board sitting idle while I worked. It's now 18:50 with zero VIN under-voltage events, whereas I had at least one during this time-frame each of the last two days (with the machine sitting idle). I'm going to let the machine sit for another 24 hours before I try printing, but, signs are positive!
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
@droftarts Overnight, I had two more events:
I've tried two different PSUs and replaced 100% of the 24v wiring (before finding this thread), so I'm fairly certain it's not a power supply or wiring issue. Only remaining thing I could swap out is the 120V from the power entry module to the PSU, but this really feels like the same issue.
Good news is, I've got a build that seems to reliably generate failures every 12 hours, if you need more beta testers for firmware!
Due to the way my space is laid out, I'd need a wifi bridge (or to re-configure my SBC as a wifi bridge, I suppose) to use the Mini5 Ethernet without the SBC, so I'm down to test firmware for RRF.
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
Ok, I caught a VIN error while idle! See below: The error at 1:42 AM was the print failing. The error at 4:43 PM was the machine sitting idle.
-
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
Also 0.0v. It's possible I mis-read my logs, but I am pretty sure of what I saw. Also of note: for at least some of that time, my heater was on because I am an idiot (but thankfully I didn't burn my house down). Of course I closed the RRF tab in Firefox this morning because my laptop was dragging -- is there any way to retrieve historical console data from SBC, so I could potentially see exactly when those messages happened and if the heater was on at the time?
[edit]On further thought, I'd say it's more likely that I misread my log, than that this is a new problem you haven't seen before. I'm going to leave my machine on and idle again with a console window up and see if I can capture this again.[/edit]
[edit]I can, in fact, Google, so I've enabled logging with
M929 S1 P"vin-debug.log"
and we'll see what we get. Is there any harm in adding this to my config.g considering I'm running on a 32gb SD card and there's no way I'm anywhere close to filling that? Would be nice if we could do auto log rollover (ie: save to a new filename after the file reaches a certain size) but c'est la vie! [/edit] -
RE: Duet 3 Mini 5+ VIN under-voltage issues in SBC mode
I'm seeing this too. DuetMini5+ Ethernet in SBC mode (RPi 3B+), M115 for mainboard and toolboard below:
M115 FIRMWARE_NAME: RepRapFirmware for Duet 3 Mini 5+ FIRMWARE_VERSION: 3.3beta1 ELECTRONICS: Duet 3 Mini5plus Ethernet FIRMWARE_DATE: 2021-02-14 16:24:45 M115 B121 Duet TOOL1LC firmware version 3.3beta1 (2021-02-14 16:34:04)
I left my printer sit powered on this week, and see 3 VIN undervolt events while idle. It also died again 3 hours into a 5 hour print last night. I'm happy to do whatever troubleshooting would help folks gather more info to get the root cause nailed down.