CAN Failure after bootloader update
After testing several times, I suspect bootloader v2.2.
Any advice would be appreciated.
- Board configuration: MB6HC----------3HC----------1LC
- Frimware: v3.2
- Bootloader: v2.2
- CAN Bus termination: open on 3HC, 120 Ohm termination on 1LC, cables are good condition.
- No CAN-diagnsis-red-LED blicking on 3HC after power-on and communication time-out for'M122 B1' command.
- If CAN cable between 3HC and 1LC is unplugged, CAN-diagnsis-LED(red) on 3HC starts blinking.
- Then operates normally after plugged.
- Before update the bootload, it worked good.
- Before update
M122 B121 Diagnostics for board 121: Board TOOL1LC firmware 3.1.0 (2020-05-15b1) Never used RAM 8.7Kb, max stack 352b HEAT 1184 CanAsync 1460 CanRecv 1416 TMC 204 AIN 524 MAIN 2308 Last reset 00:00:51 ago, cause: power up Driver 0: standstill, SG min/max 0/0, read errors 0, write errors 0, ifcount 9, reads 25502, writes 9, timeouts 1, DMA errors 0, failedOp 0x80 Moves scheduled 0, completed 0, hiccups 0 VIN: 24.1V MCU temperature: min 13.6C, current 17.7C, max 17.7C Ticks since heat task active 58, ADC conversions started 50881, completed 50880, timed out 0 Last sensors broadcast 00000000 found 0 61 ticks ago Free CAN buffers: 36 NVM user row b1d088a9 fffff8bb 20002a18 b TSENS 0006f9 GAIN 015ac2 OFFS 00241e CAL 1f2a
- After update
M122 === Diagnostics === RepRapFirmware for Duet 3 MB6HC version 3.2 running on Duet 3 MB6HC v0.6 or 1.0 (standalone mode) Board ID: 08DJM-956BA-NA3TJ-6JKDG-3S86R-TU9UT Used output buffers: 1 of 40 (22 max) === RTOS === Static ram: 149788 Dynamic ram: 92720 of which 76 recycled Never used RAM 116248, free system stack 188 words Tasks: NETWORK(ready,163) ETHERNET(blocked,118) HEAT(blocked,302) CanReceiv(blocked,834) CanSender(blocked,352) CanClock(blocked,356) TMC(blocked,19) MAIN(running,1095) IDLE(ready,19) Owned mutexes: HTTP(MAIN) === Platform === Last reset 00:01:22 ago, cause: power up Last software reset at 2021-03-03 11:41, reason: User, GCodes spinning, available RAM 116152, slot 2 Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00400000 BFAR 0x00000000 SP 0x00000000 Task MAIN Freestk 0 n/a Error status: 0x00 Aux0 errors 0,0,0 Aux1 errors 0,0,0 MCU temperature: min 17.0, current 17.3, max 17.4 Supply voltage: min 23.8, current 23.8, max 23.9, under voltage events: 0, over voltage events: 0, power good: yes 12V rail voltage: min 12.0, current 12.0, max 12.1, under voltage events: 0 Driver 0: position 8, standstill, reads 15712, writes 0 timeouts 0, SG min/max not available Driver 1: position 8, standstill, reads 15712, writes 0 timeouts 0, SG min/max not available Driver 2: position 20727, standstill, reads 15712, writes 0 timeouts 0, SG min/max not available Driver 3: position 0, standstill, reads 15713, writes 0 timeouts 0, SG min/max not available Driver 4: position 0, standstill, reads 15712, writes 0 timeouts 0, SG min/max not available Driver 5: position 0, standstill, reads 15712, writes 0 timeouts 0, SG min/max not available Date/time: 2021-03-03 14:59:00 Slowest loop: 22.45ms; fastest: 0.05ms === Storage === Free file entries: 10 SD card 0 detected, interface speed: 25.0MBytes/sec SD card longest read time 2.2ms, write time 16.4ms, max retries 0 === Move === DMs created 125, maxWait 0ms, bed compensation in use: none, comp offset 0.000 === MainDDARing === Scheduled moves 1, completed moves 1, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === AuxDDARing === Scheduled moves 0, completed moves 0, hiccups 0, stepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1 === Heat === Bed heaters = 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1 Heater 1 is on, I-accum = 0.0 === GCodes === Segments left: 0 Movement lock held by null HTTP is ready with "M122 " in state(s) 0 Telnet is idle in state(s) 0 File is idle in state(s) 0 USB is idle in state(s) 0 Aux is idle in state(s) 0 Trigger is idle in state(s) 0 Queue is idle in state(s) 0 LCD is idle in state(s) 0 SBC is idle in state(s) 0 Daemon is idle in state(s) 0 Aux2 is idle in state(s) 0 Autopause is idle in state(s) 0 Code queue is empty. === Network === Slowest loop: 28.11ms; fastest: 0.02ms Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions Telnet(0), 0 sessions HTTP sessions: 1 of 8 - Ethernet - State: active Error counts: 0 0 1 0 0 Socket states: 5 2 2 2 2 0 0 0 === CAN === Messages queued 11, send timeouts 0, received 36, lost 0, longest wait 0ms for reply type 0, free buffers 48 M122 B1 Diagnostics for board 1: Duet EXP3HC firmware version 3.2 (2021-01-05) Bootloader ID: SAME5x bootloader version 2.2 (2021-01-16b1) Never used RAM 154896, free system stack 200 words HEAT 102 CanAsync 94 CanRecv 87 TMC 64 MAIN 299 AIN 257 Last reset 00:00:07 ago, cause: software Last software reset data not available Driver 0: position 0, 80.0 steps/mm, standstill, reads 35212, writes 11 timeouts 0, SG min/max 0/0 Driver 1: position 0, 80.0 steps/mm, standstill, reads 35214, writes 11 timeouts 0, SG min/max 0/0 Driver 2: position 0, 80.0 steps/mm, standstill, reads 35217, writes 11 timeouts 0, SG min/max 0/0 Moves scheduled 0, completed 0, in progress 0, hiccups 0 No step interrupt scheduled VIN: 24.3V, V12: 12.3V MCU temperature: min 28.1C, current 28.1C, max 28.1C Ticks since heat task active 30, ADC conversions started 7772, completed 7771, timed out 0 Last sensors broadcast 0x00000000 found 0 33 ticks ago, loop time 0 CAN messages queued 18, send timeouts 0, received 108, lost 0, free buffers 36 M122 B121 Diagnostics for board 121: Duet TOOL1LC firmware version 3.2 (2021-01-05) Bootloader ID: SAMC21 bootloader version 2.2 (2021-01-16b1) Never used RAM 4140, free system stack 96 words HEAT 90 CanAsync 89 CanRecv 83 TMC 60 MAIN 218 AIN 70 Last reset 00:00:45 ago, cause: power up Last software reset data not available Driver 0: position 0, 420.0 steps/mm, standstill, SG min/max 0/0, read errors 0, write errors 0, ifcnt 13, reads 22960, writes 13, timeouts 0, DMA errors 0 Moves scheduled 0, completed 0, in progress 0, hiccups 0 No step interrupt scheduled VIN: 24.1V MCU temperature: min 6.1C, current 10.1C, max 10.1C Ticks since heat task active 227, ADC conversions started 45840, completed 45839, timed out 0 Last sensors broadcast 0x00000002 found 1 230 ticks ago, loop time 0 CAN messages queued 422, send timeouts 0, received 385, lost 0, free buffers 36
This is an interesting one! It could be either the bootloader or the firmware not starting.
If you temporarily downgrade the 3HC firmware to 3.1.1, does it start up and flash the LED with the tool board connected?
If it still doesn't start up with the tool board connected, then I suggest you downgrade the bootloader to version 1.3. When that is working and booting the firmware, put firmware 3.2.2 back on the 3HC.
As a result, The worst has happened. All three HC3 boards have been ICE-state(red diagnostic LED off, No response).
No response for M122 B1. No response to the reset button
I tested several things using the HC3 board I had.
RRF3.1 + Bootloader1.3 (estimated, No ID on diagnostics) -> good
RRF3.2.2 + Bootloader1.3 -> good
RRF3.2.2 + Bootloader2.2 -> Abnormal
RRF3.2.2 + Bootloader1.3 Down grade -> HC3 ICE-state
No problem on 1LC board.
I need the ATMEL ICE tool.
Please try the following:
- Upload EXP3HC firmware version 3.1.1 to the Duet
- Do a factory reset of one of the 3HC boards (set all 4 switches off and then turn on the power).
First of all, thank you very much for the always quick and kind reply.
I've tried following your advice, One of three boards could be made with firmware v3.1.1 + Bootloader v1.3.
I repeated the test with this.
I thought it was OK with firmware v3.1.1 + Bootloader v1.3, but there was a problem.
When 3HC is disconnected from every cable (include CAN) and only power is connected, does not the DIAG red LED flash at all, just off state.
Repeated plugging in and unplugging the CAN cable, the red LED starts blinking.
The other 2 3HC are still ICE"ICE-state" (I don't know what to call it in English. I call it'ICE'. power-on, no response, no blinking, do nothing state)
Disconnected the 3HC board from CAN and set the address to zero (all switches off), does not the DIAG red LED flash at all, just off state.
The 3HC is not connected to CAN in any way.
In my experience, I suspect the bootloader 2.2. the bootloader procedure seems to be stuck in the CAN init loop.
Even when it worked normally in the past, sometimes (30% probability) there was a situation where it was suspected that booting was delayed.
Because I found a case where the contents of config.g were not passed to 3HC.
So, I put "G4 S2" (delay 2 seconds) at the beginning of config.g.
I hope you try to reproduce the same situation as me.
And 1LC looks fine.
@psyblue, which country are you in, and which reseller did you buy the boards from?
I am Korean in Korea, bought at filastruder.com
Isn't it a known problem?
It could be an important source of potential bugs in the future.
I can send you one, if you want. Of course, I want to get it back with the improved one.
However, I have anxiety that seems to work normally in your hand as always
Or, I ordered ATMEL ICE. I will reprogram it and report the result.
It will take about 2 weeks.
And short question,
Is there any changed or added pull-up/ pull-down resistor on 3HC board?
All of the above symptoms are considered a problem with Bootloader 2.2.
After updating to Bootloader 2.3 via ATMEL-ICE, everything works fine.
Thank you for the bootloader 2.3.
@psyblue, I'm glad you solved it with bootloader 2.3. I'm sorry that it was necessary for you to get an ICE to fix it.
psyblue last edited by
@dc42, I am also glad that the problem disappeared with the bootloader 2.3, but I am curious if the above symptoms match the potential problem with Bootloader 2.2
@psyblue the main change between bootloader 2.2 and 2.3 is that in 2.2 if it didn't get a satisfactory frequency reading within a certain time, its reset the processor to start again. In 2.3 it just keeps on taking readings, in effect allowing the crystal oscillator as long as it needs to stabilise.