Duet 3 Mini 5+ VIN under-voltage issues in SBC mode

Phaedrux

@Serbitar Thanks for the report.

In SBC mode you could try enabling more detailed logging.

https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Monitoring_optional

carcamerarig

I also had one this evening why performing home all
3.3b2 sbc mini 5

dc42

@carcamerarig, thanks for the additional data point.

My best guess at this point is that the DMA for the SPI transfers between the Pi and the Duet is somehow interfering with the DMA used by the ADC. Do any of you experiencing this problem see short-lived temperature jumps too?

Serbitar

@dc42 by the time i'm back at the printer, usually several hours have past since it stopped so the temperature charts have gone past.

I have a few prints to complete in standalone mode and then i will enable the extra monitoring. Is there any way to save logs of the temp values too?

dc42

Until I have pinned down the exact cause, it might be worth trying a lower SPI frequency. The frequency is set in the DCS configuration file on the Pi.

dc42

PS - if anyone who has experienced this has their machine idle, please can you run some simulated prints, to see whether they can trigger this issue. The increased SPI traffic rate from the Pi during a simulation might make it happen more often. OTOH it could turn out that step interrupts also need to happen for this issue to occur.

WILD9

@dc42 Happy to try, how would this manifest during a simulation, just console messages showing low voltage detected? In which case I will make sure I have 24Vin active during simulation.

dc42

@WILD9 said in Duet 3 Mini 5+ VIN under-voltage issues in SBC mode:

@dc42 Happy to try, how would this manifest during a simulation, just console messages showing low voltage detected? In which case I will make sure I have 24Vin active during simulation.

I expect the indications of the problem to be:

A console report of an under-voltage event, with the reported VIN being 0.0
If you run M122 afterwards then it will report that one or more under-voltage events occurred and the minimum recorded VIN will be 0.0. Note, a subsequent M122 will only report under-voltage events and the minimum VIN that happened since the previous M122.

You will need to have VIN power applied throughout.

carcamerarig

@dc42 said in Duet 3 Mini 5+ VIN under-voltage issues in SBC mode:

PS - if anyone who has experienced this has their machine idle, please can you run some simulated prints, to see whether they can trigger this issue. The increased SPI traffic rate from the Pi during a simulation might make it happen more often. OTOH it could turn out that step interrupts also need to happen for this issue to occur.

I’ve been spending quite a few hours daily of stop start printing while I’m still tuning and configuring (mostly learning) and have only seen the one which was at a home all, the effect was a crash into the left x carriage carrier, I’m unsure if the error was before during or after the crash.
I’ll be doing more of the same later if you want me to test or log anything, I’ll need some instructions though if I’m to alter some code.

Serbitar

@dc42 Is 4MHz low enough?

lodger

Is it worth running some heaters duing a simulation. To give the PSU some load.
During a print my VIN can jump up and down a few points of a volt. But during a sim . She is bang on 24.2v and dont move.

WILD9

6ish hours of simulation so far with no issues

droftarts

@WILD9 I ran simulations for 8 hours last night, with no issues. Still trying to track this issue down.

Ian

lodger

I have come down this morning to find a stalled print. Here is my M122,

m122

=== Diagnostics ===
RepRapFirmware for Duet 3 Mini 5+ version 3.2.2 running on Duet 3 Mini5plus WiFi (SBC mode)
Board ID: XJDX9-1296U-D65J0-40KMJ-4K03Z-7J6X6
Used output buffers: 1 of 40 (18 max)
=== RTOS ===
Static ram: 98732
Dynamic ram: 95844 of which 64 recycled
Never used RAM 50576, free system stack 122 words
Tasks: Linux(ready,99) HEAT(blocked,297) CanReceiv(blocked,947) CanSender(blocked,358) CanClock(blocked,362) TMC(blocked,75) MAIN(running,476) IDLE(ready,20) AIN(blocked,264)
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 17:19:34 ago, cause: software
Last software reset at 2021-03-19 15:58, reason: User, none spinning, available RAM 50808, slot 1
Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0400f000 BFAR 0xe000ed38 SP 0x00000000 Task Linu Freestk 0 n/a
Error status: 0x00
Aux0 errors 0,0,0
Aux1 errors 0,0,0
Supply voltage: min 0.0, current 24.2, max 24.3, under voltage events: 2, over voltage events: 0, power good: yes
Driver 0: position 16355, standstill, SG min/max 0/428, read errors 0, write errors 1, ifcnt 223, reads 13005, writes 43, timeouts 17110, DMA errors 0, failedOp 0x71
Driver 1: position 2458, standstill, SG min/max 0/410, read errors 0, write errors 1, ifcnt 223, reads 30107, writes 43, timeouts 7, DMA errors 0, failedOp 0x6f
Driver 2: position 63048, standstill, SG min/max 0/44, read errors 0, write errors 1, ifcnt 189, reads 29631, writes 32, timeouts 494, DMA errors 0, failedOp 0x71
Driver 3: position 0, standstill, SG min/max 0/30, read errors 0, write errors 1, ifcnt 188, reads 29889, writes 32, timeouts 235, DMA errors 0, failedOp 0x01
Driver 4: position 0, standstill, SG min/max 0/0, read errors 0, write errors 1, ifcnt 155, reads 30130, writes 27, timeouts 0, DMA errors 0
Driver 5: position 0, assumed not present
Driver 6: position 0, assumed not present
Date/time: 2021-03-20 09:18:05
Cache data hit count 4294967295
Slowest loop: 178.10ms; fastest: 0.08ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 0.0MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 83, maxWait 11368610ms, bed compensation in use: mesh, comp offset 0.000
=== MainDDARing ===
Scheduled moves 190766, completed moves 190766, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1, chamberHeaters = 2 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File* is idle in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger* is idle in state(s) 0
Queue* is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages queued 561364, send timeouts 561361, received 0, lost 0, longest wait 0ms for reply type 0, free buffers 15
=== SBC interface ===
State: 4, failed transfers: 0
Last transfer: 3ms ago
RX/TX seq numbers: 42328/42328
SPI underruns 0, overruns 0
Number of disconnects: 0, IAP RAM available 0x10eec
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.2.2
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 36.40
Maximum length of RX/TX data transfers: 5108/1664.

dc42

@lodger, was that a print or a simulation?

Please can you try the firmware at https://www.dropbox.com/sh/6203euyyobllic5/AABTHIbn7F_maK9p-oy7vq3Ea?dl=0 with the same file to see if it happens again.

lodger

@dc42 Hi,

Yes it was a print. I will just put the kettle on and update and print. It was a 10 hour print so i may not get back to you till tomo.

Edit:: I have updated and run a Similator of the same file. Without problems !
Just running the print now .

lodger

@dc42 Mornings.

The print completed . with no under voltage triggers at all. I will try print something else and report if i get any more problems

dc42

@lodger said in Duet 3 Mini 5+ VIN under-voltage issues in SBC mode:

@dc42 Mornings.

The print completed . with no under voltage triggers at all. I will try print something else and report if i get any more problems

Thanks for the update.

Anyone else having this problem and happy to use beta firmware, feel free to try the firmware that I linked to above. It's compatible with DWC 3.3beta2.

Serbitar

I have put my machine back into SBC mode running the linked beta firmware.

It completed a 12h print over night, print queue has shorter jobs for the next few days. I will update if there are any under voltage detection issues.

dc42

@Serbitar, thanks for the update.