Duet 3 EB 3HC intermittent CAN/boot? failure

[Max 0]

Hello everyone, I just finished the assembling and wiring my new IDEX Printer (RatRig V-Cast) and started configuring the Duet 6HC and 3HC. At this point I should probably point out that this is my first Duet/RepRap firmware system and I may very well be making stupid beginner mistakes. Also I hope this is the right section of the Forum, this seems to be where most other issues with the CAN bus where being posted.

When I first loaded the configuration I created with the configuration tool onto the 6HC through Duet Web Control yesterday , I was able to control the fan connected to the 3HC but the thermistor and heater were not showing up and the diagnostic led was flashingin sync with the 6HC. However after I rebooted it to make a change to the config.g it did not connect ot the 6HC anymore and I started seeing the error messages shown below. Then this morning when I turned the printer on the 3HC worked normally again with the diagnostic LED flashing in sync with the 6HC and I was able to control the heater, fans and motors connected to the 3HC and even get it to spit out an M122 (also below). However, after I turned the machine off and on again, I had the same issue as yesterday and the diagnostic LED didn´t turn on at all again, whilst all the power LEDS are on (Does this mean the board is not booting?).

Here is some troubleshooting I tried after reading some other threads on here about similiar Issues:

• If i set the 3HC to CAN adress 0 by turning of all the switches and remove the CAN cable, it will occasionaly flash the diagnostic LED four times, as would be expected from what I read in other threads, however most times the diagnostic LED will just not come on at all. Meanwhile, the four power LEDs (VIN, 12V, 5V and 3,3V) always come on.

• Adding a G4 delay to the config.g to allow for the 3HC to boot does not seem to have any impact on the problem

• I checked and my RJ11 cable was sold as an ADSL 2+ High speed cable on Amazon, so I think that should not be the issue

Both boards are running Firmware Version 3.2.2 and the 6HC is running in SBC mode with a Raspberry Pi 3.

When I turn the Printer on and the CAN is not working I get the following Error messages followed by some more errors about the components connected to the 3HC not being found:

Error: M569: Response timeout: CAN addr 1, req type 6018, RID=0
Error: M569: Response timeout: CAN addr 1, req type 6018, RID=1
Error: M569: Response timeout: CAN addr 1, req type 6018, RID=2
Error: M584: Response timeout: CAN addr 1, req type 6042, RID=3

Here is an M122 I was able to extract from the 3HC when it was working this morning:

M122 B1
Diagnostics for board 1:
Duet EXP3HC firmware version 3.2.2 (2021-02-11)
Bootloader ID: not available
Never used RAM 154440, free system stack 200 words
HEAT 92 CanAsync 94 CanRecv 87 TMC 64 MAIN 173 AIN 259
Last reset 00:01:45 ago, cause: power up
Last software reset data not available
Driver 0: position 0, 400.0 steps/mm, standstill, reads 26978, writes 16 timeouts 0, SG min/max 0/0
Driver 1: position 0, 1.0 steps/mm, standstill, reads 26982, writes 14 timeouts 0, SG min/max 0/0
Driver 2: position 0, 420.0 steps/mm, standstill, reads 26983, writes 16 timeouts 0, SG min/max 0/0
Moves scheduled 0, completed 0, in progress 0, hiccups 0
No step interrupt scheduled
VIN: 24.2V, V12: 12.3V
MCU temperature: min 41.8C, current 42.0C, max 42.0C
Ticks since heat task active 158, ADC conversions started 105900, completed 105898, timed out 0
Last sensors broadcast 0x00000004 found 1 161 ticks ago, loop time 0
CAN messages queued 1032, send timeouts 0, received 888, lost 0, free buffers 36

and here is an M122 from the 6HC:

m122
=== Diagnostics ===
RepRapFirmware for Duet 3 MB6HC version 3.2.2 running on Duet 3 MB6HC v1.01 or later (SBC mode)
Board ID: 08DJM-956BA-NA3TN-6J1F2-3S46K-9BAAS
Used output buffers: 1 of 40 (13 max)
=== RTOS ===
Static ram: 149788
Dynamic ram: 62976 of which 60 recycled
Never used RAM 146008, free system stack 200 words
Tasks: Linux(ready,101) HEAT(blocked,297) CanReceiv(blocked,927) CanSender(blocked,371) CanClock(blocked,352) TMC(blocked,53) MAIN(running,1203) IDLE(ready,19)
Owned mutexes: HTTP(MAIN)
=== Platform ===
Last reset 00:01:59 ago, cause: software
Last software reset at 2021-04-07 13:44, reason: User, none spinning, available RAM 146008, slot 1
Software reset code 0x0012 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0044a000 BFAR 0x00000000 SP 0x00000000 Task Linu Freestk 0 n/a
Error status: 0x00
Aux0 errors 1,1,1
Aux1 errors 0,0,0
MCU temperature: min 39.6, current 41.3, max 42.8
Supply voltage: min 23.9, current 23.9, max 24.0, under voltage events: 0, over voltage events: 0, power good: yes
12V rail voltage: min 12.0, current 12.1, max 12.1, under voltage events: 0
Driver 0: position 0, standstill, reads 24326, writes 14 timeouts 0, SG min/max 0/0
Driver 1: position 0, standstill, reads 24326, writes 14 timeouts 0, SG min/max 0/0
Driver 2: position 0, standstill, reads 24327, writes 13 timeouts 0, SG min/max 0/0
Driver 3: position 0, standstill, reads 24326, writes 14 timeouts 0, SG min/max 0/0
Driver 4: position 0, standstill, reads 24329, writes 11 timeouts 0, SG min/max 0/0
Driver 5: position 0, standstill, reads 24329, writes 11 timeouts 0, SG min/max 0/0
Date/time: 2021-04-07 13:47:00
Slowest loop: 0.44ms; fastest: 0.04ms
=== Storage ===
Free file entries: 10
SD card 0 not detected, interface speed: 37.5MBytes/sec
SD card longest read time 0.0ms, write time 0.0ms, max retries 0
=== Move ===
DMs created 125, maxWait 0ms, bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 0
Movement lock held by null
HTTP* is doing "M122" in state(s) 0
Telnet is idle in state(s) 0
File is idle in state(s) 0
USB is idle in state(s) 0
Aux is assembling a command in state(s) 0
Trigger* is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
SBC is idle in state(s) 0
Daemon is idle in state(s) 0
Aux2 is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== CAN ===
Messages queued 331, send timeouts 928, received 0, lost 0, longest wait 0ms for reply type 0, free buffers 48
=== SBC interface ===
State: 4, failed transfers: 0
Last transfer: 1ms ago
RX/TX seq numbers: 3946/3946
SPI underruns 0, overruns 0
Number of disconnects: 0, IAP RAM available 0x2c8a8
Buffer RX/TX: 0/0-0
=== Duet Control Server ===
Duet Control Server v3.2.2
Code buffer space: 4096
Configured SPI speed: 8000000 Hz
Full transfers per second: 33.79
Maximum length of RX/TX data transfers: 3072/844

Here is my Config.g file:
config.g

[dc42]

@max-0, I'm sorry you are having problems with that expansion board. I am tempted to arrange to have your board replaced immediately. But first, please can you identify the crystal - which is the small metal can next to one corner of the main processor (which has the serial number label on it) and report the markings on it.

PS - which country are you in?

[Max 0]

@dc42 thank you for the quick response. I believe it it has the same marking as the one in your picture:
120
020a

I am in Germany, but the board was purchased from RatRig in Protugal.

[dc42]

@max-0, thanks. We will replace your board under warranty. Please complete the reselleller warranty form at https://docs.google.com/forms/d/e/1FAIpQLSdXpbRCNxUc2j_OePtuWdwyKPJ9VEg2Ohq_80f-VCZ4SCJJ9w/viewform.

We would like you to send that board direct to me, at our cost. We will be in touch with you by email to arrange that.

[Max 0]

@dc42 thank you for your quick response on this. I have just filled out the warranty form.

[Max 0]

@dc42 my new board has arrived now, so I thought I would give you a quick update on this. I have tested power cycling the machine multiple times and so far the expansion board has worked flawlessly, establishing a connection every time. What I found quite interesting is that the serial number on the new board I received is only two lower than that on the old board and the crystal appears to be the same aswell.

According to the DHL tracking number my old board has left the import sorting facility in the U.K. already, so it should hopefully be with you some time at the beginning of next week.

Thank you once again for your help with solving this and if you need any more information about this please don´t hesitate to contact me.

Max

[JuJuDelta]

Hi Dave & Max,

I have 4 off 3HC boards and one of them has been a bit of a CAN bus nuisance since day 1, but I have persevered.

The board in question has the same crystal markings as the photo above "120 020a"

Since a recent update (3.3.2 and various 3.3 betas) there is no longer any CAN bus activity RED led flashing, it is off all the time.
Now there is a small chance that I may have inadvertently messed up by trying to load the boot loader and consequently there is no LED activity.

so, I have ordered a ST-Link V2 and JST 1.5mm harness so will try to re-install the boot loader at the correct location.

I will report back later during the week.

@dc42:
Dave, if I am unsuccessful, may I return the board back to you for a quick check, happy to paypal you return postage ? Based in Coventry, not a million miles from you.
Or I am happy to re-solder a different batch# XTAL if you can advise or signpost me to Farnell RS etc.

Board was originally purchased by my client from Oozenest.

PCB is v1.0 with a central hole near the processor, orange sticker dot and label "DUET3 EB1 WD01117"

T
Thanks & Regards,

Julian.

[dc42]

Sure, send it to me. I will investigate the problem and fit a new crystal if it needs one.

[JuJuDelta]

@dc42 said in Duet 3 EB 3HC intermittent CAN/boot? failure:

Sure, send it to me. I will investigate the problem and fit a new crystal if it needs one.

Hi Dave,

Thanks for the offer.
Can you PM your postal address please ?

skunkwerkzlabATgmailDOTcom

Regards,

Julian

[dc42]

@jujudelta I have sent you a PM.

[ballanux]

Hi all!

I'm new to the Duet platform so let me know if I'm missing something but I'm quite sure I have this same issue. I was experiencing very low torque on my 3 Z axis steppers, but good torque on the XY ones. The Z steppers are connected to the 3HC board ant the XY ones to the mainboard....

I didn't look into the forum until my "replacement" and more powerful steppers showed up and didn't perform any better

After finding this and other similar post, I tested sending manually the M906 command I got a CAN bus error:

I really would had appreciated if this were also reported during normal boot of the mainboard... It would have reduced the time to find the issue...

Anyway, if this is related to a bad batch of crystals, here is mine:

I bought my board through FilaFarm, please let me know if I'm really affected by this, or how can I confirm it and If I have to manage a replacement either with Filafarm or directly here.

Thanks

[dc42]

@ballanux the key test is: when you power up the 3HC board, does the red LED between the CAN connectors start blinking almost immediately (within half a second)? If it doesn't always start blinking immediately, then it's a startup issue. If that's the case, please send M122 B1 when the board is responding, so that we can check whether you are running the correct firmware version.

If the red LED does always start blinking immediately after power up, then it's probably configuration issue, or perhaps you have incompatible firmware versions on the main board and the expansion board. So please share your config.g file and the results of running M115 and M122 B1.

[ballanux]

@dc42 Thanks for your quick reply and pointing me in the right direction

I had different firmware in the 3HC and the mainboard, I'm quite sure I uploaded both .bin files from the same release, but maybe the 3HC didn't update... anyway, I have now both of them on 3.2.2 version and the M906 command is being received and the stepper has the proper torque.