Duet 3 Mini 5+ VIN under-voltage issues in SBC mode

[lodger]

Afternoon,

Sorry to say i have had undervolt again. This time network fully locked up. and i had to plug in a keyboard and monitor to get the M122 off . So sorry about the txt file

M122.txt

Web page ,SSH and remote desktop fully stopped.

[dc42]

@lodger, thanks for that. It's a useful data point, because it eliminates another aspect of the hardware and firmware that is only used when there is an attached SBC.

[Serbitar]

Tripped over the issue again:

Mar 27 21:49:17 ender3 DuetControlServer[22051]: [error] === Diagnostics ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: RepRapFirmware for Duet 3 Mini 5+ version 3.3beta2+1-swcrc running on Duet 3 Mini5plus WiFi (SBC mode)
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Board ID: W4ZW4-V296U-D65J0-40KM0-LR03Z-H6928
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Used output buffers: 2 of 40 (12 max)
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === RTOS ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Static ram: 100008
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Dynamic ram: 98124 of which 0 recycled
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Never used RAM 45572, free system stack 123 words
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Tasks: Linux(ready,145) HEAT(delaying,299) CanReceiv(notifyWait,943) CanSender(notifyWait,358) CanClock(delaying,340) TMC(notifyWait,109) MAIN(running,354) IDLE(ready,20) AIN(delaying,258)
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Owned mutexes: Autopause(MAIN)
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === Platform ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Last reset 00:29:07 ago, cause: software
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Last software reset at 2021-03-27 21:20, reason: User, none spinning, available RAM 45476, slot 2
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0x00000000 Task Linu Freestk 0 n/a
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Error status: 0x00
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Aux0 errors 0,0,0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Aux1 errors 0,0,0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: ADC conversions started 870319, completed 870319, timed out 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Supply voltage: min 0.0, current 24.1, max 25.0, under voltage events: 2, over voltage events: 0, power good: no
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Heap OK, handles allocated/used 0/0, heap memory allocated/used/recyclable 0/0/0, gc cycles 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Driver 0: position -7822, standstill, SG min/max 0/0, read errors 0, write errors 2, ifcnt 10, reads 17204, writes 18, timeouts 2, DMA errors 0, failedOp 0x6a
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Driver 1: position -7466, standstill, SG min/max 0/38, read errors 0, write errors 2, ifcnt 10, reads 17204, writes 18, timeouts 1, DMA errors 0, failedOp 0x72
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Driver 2: position 3887, ok, SG min/max 0/248, read errors 0, write errors 2, ifcnt 18, reads 17196, writes 26, timeouts 0, DMA errors 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Driver 3: position 0, ok, SG min/max 0/422, read errors 0, write errors 2, ifcnt 14, reads 17183, writes 22, timeouts 17, DMA errors 0, failedOp 0x72
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Driver 4: position 0, ok, SG min/max 0/408, read errors 0, write errors 2, ifcnt 14, reads 17194, writes 22, timeouts 8, DMA errors 0, failedOp 0x01
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Driver 5: position 0, ok, SG min/max 0/100, read errors 0, write errors 2, ifcnt 14, reads 17195, writes 22, timeouts 5, DMA errors 0, failedOp 0x41
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Driver 6: position 0, ok, SG min/max 0/114, read errors 0, write errors 2, ifcnt 14, reads 16657, writes 22, timeouts 543, DMA errors 0, failedOp 0x41
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Date/time: 2021-03-27 21:49:16
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Cache data hit count 2632939525
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Slowest loop: 179.79ms; fastest: 0.08ms
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === Storage ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Free file entries: 10
Mar 27 21:49:17 ender3 DuetControlServer[22051]: SD card 0 not detected, interface speed: 0.0MBytes/sec
Mar 27 21:49:17 ender3 DuetControlServer[22051]: SD card longest read time 0.0ms, write time 0.0ms, max retries 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === Move ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: DMs created 83, maxWait 62634ms, bed compensation in use: mesh, comp offset 0.000
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === MainDDARing ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Scheduled moves 19881, completed moves 19881, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 4], CDDA state -1
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === AuxDDARing ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === Heat ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Bed heaters = 0 -1, chamberHeaters = -1 -1
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Heater 0 is on, I-accum = 0.0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Heater 2 is on, I-accum = 0.6
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === GCodes ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Segments left: 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Movement lock held by Autopause
Mar 27 21:49:17 ender3 DuetControlServer[22051]: HTTP* is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Telnet is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: File* is doing "G1 X-85.899002 Y-87.926003 E0.377730" in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: USB is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Aux is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Trigger* is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Queue* is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: LCD is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: SBC is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Daemon is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Aux2 is idle in state(s) 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Autopause is ready with "M122 M115 M913 X0 Y0 G10 P0 R0 S0 G10 P1 R0 S0 G91 M83 G1 Z3 E-5 F10000" in state(s) 71
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Code queue is empty.
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === Filament sensors ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Extruder 0 sensor: ok
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Extruder 1 sensor: ok
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === CAN ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Messages queued 15731, send timeouts 15728, received 0, lost 0, longest wait 0ms for reply type 0, peak Tx sync delay 0, free buffers 17 (min 17)
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Last cancelled message type 4514 dest 127
Mar 27 21:49:17 ender3 DuetControlServer[22051]: === SBC interface ===
Mar 27 21:49:17 ender3 DuetControlServer[22051]: State: 0, failed transfers: 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Last transfer: 10ms ago
Mar 27 21:49:17 ender3 DuetControlServer[22051]: RX/TX seq numbers: 1055/1056
Mar 27 21:49:17 ender3 DuetControlServer[22051]: SPI underruns 0, overruns 0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Number of disconnects: 0, IAP RAM available 0x11458
Mar 27 21:49:17 ender3 DuetControlServer[22051]: Buffer RX/TX: 240/1728-0
Mar 27 21:49:17 ender3 DuetControlServer[22051]: G0/G1: insufficient axes homed
Mar 27 21:49:17 ender3 DuetControlServer[22051]: [warn] VIN under-voltage event (0.0V)

[dc42]

@serbitar thanks, that confirms that the latest change did not help.

[PaulHew]

I am also experiencing this issue.

My machine was not doing anything, Just completed a heater tune. Below are the last 3 entries in the log.

28/03/2021, 19:03:05 	Warning: VIN under-voltage event (0.0V)
28/03/2021, 16:17:33 	Edit the M307 H0 command in config.g to match this. Omit the V parameter if the heater is not powered from VIN.
28/03/2021, 16:17:33 	Auto tuning heater 0 completed after 4 idle and 7 tuning cycles in 1336 seconds. This heater needs the following M307 command:
 M307 H0 R0.853 C250.9 D5.98 S0.60 V23.9

Pi has a dedicated PSU. Mini 5+ is power by a 300w meanwell.

Do you need any other info?

[dc42]

@paulhew, are you running with attached SBC, or in standalone mode?

[PaulHew]

@dc42 With a SBC. Pi4.
Regards, Paul.

[droftarts]

@paulhew the cause of this issue is still a bit of a mystery; I’ve been running air prints all weekend with different firmware from dc42 to try and narrow it down. It shouldn’t be a problem if you run in standalone mode.

Ian

[PaulHew]

@droftarts I have been trying to get back to a working printer after the couple of hiccups I have encountered after moving over to a Duet Mini 5+ and 1LC, the BLT being the main one, Appreciated your guidance on that Ian.

If I can help test, please let me know. I would be more than happy to run some air prints, however that is done.

[PaulHew]

Just had another under voltage.

For info....

[droftarts]

@paulhew Thanks. Please post response to M115, which shows firmware version and date. We're currently testing newer firmware versions internally, but this problem is proving quite elusive.

Ian

[PaulHew]

@droftarts as requested

m115
FIRMWARE_NAME: RepRapFirmware for Duet 3 Mini 5+ FIRMWARE_VERSION: 3.2.2 ELECTRONICS: Duet 3 Mini5plus Ethernet FIRMWARE_DATE: 2021-02-11

[droftarts]

@paulhew Thanks. Is that in SBC or Standalone? I think we haven't seen any 0V events confirmed in Standalone mode.

Ian

[PaulHew]

@droftarts Ian, SBC mode.

[Fletcher]

I'm seeing this too. DuetMini5+ Ethernet in SBC mode (RPi 3B+), M115 for mainboard and toolboard below:

M115
FIRMWARE_NAME: RepRapFirmware for Duet 3 Mini 5+ FIRMWARE_VERSION: 3.3beta1 ELECTRONICS: Duet 3 Mini5plus Ethernet FIRMWARE_DATE: 2021-02-14 16:24:45
M115 B121
Duet TOOL1LC firmware version 3.3beta1 (2021-02-14 16:34:04)

I left my printer sit powered on this week, and see 3 VIN undervolt events while idle. It also died again 3 hours into a 5 hour print last night. I'm happy to do whatever troubleshooting would help folks gather more info to get the root cause nailed down.

[dc42]

@fletcher said in Duet 3 Mini 5+ VIN under-voltage issues in SBC mode:

I left my printer sit powered on this week, and see 3 VIN undervolt events while idle.

That's the first time we have seen this issue when the printer is idle. Perhaps it just wasn't noticed before. What was the voltage reported with the undervoltage events: 0.0, or a higher value?

[Fletcher]

@dc42

Also 0.0v. It's possible I mis-read my logs, but I am pretty sure of what I saw. Also of note: for at least some of that time, my heater was on because I am an idiot (but thankfully I didn't burn my house down). Of course I closed the RRF tab in Firefox this morning because my laptop was dragging -- is there any way to retrieve historical console data from SBC, so I could potentially see exactly when those messages happened and if the heater was on at the time?

[edit]On further thought, I'd say it's more likely that I misread my log, than that this is a new problem you haven't seen before. I'm going to leave my machine on and idle again with a console window up and see if I can capture this again.[/edit]

[edit]I can, in fact, Google, so I've enabled logging with M929 S1 P"vin-debug.log" and we'll see what we get. Is there any harm in adding this to my config.g considering I'm running on a 32gb SD card and there's no way I'm anywhere close to filling that? Would be nice if we could do auto log rollover (ie: save to a new filename after the file reaches a certain size) but c'est la vie! [/edit]

[Serbitar]

@fletcher You can probably get logs from the SBC:

sudo journalctl  --unit duetcontrolserver 

[Fletcher]

Ok, I caught a VIN error while idle! See below: The error at 1:42 AM was the print failing. The error at 4:43 PM was the machine sitting idle.

[PaulHew]

Happened again Mid-Print.

Had to issue a m564 h0 so I could lower the bed and clear the bed.

Again, SBC mode, rrf3.2.2

P.