Duet sometimes really slow? - I2C error or?
-
@deckingman said in Duet sometimes really slow? - I2C error or?:
David,
I just ran "the sequence" as far as I am able (everything apart from homing the upper gantry with mapped endstops). After which I changed (lowered) the temperature threshold for one of the thermostatic fans connected to the Duex5, to force the fan to come on which it did. Then I toggled the E2 end stop whilst observing the status on the machine properties page and that all worked as normal. After which I ran M122 and noted the following:
I2C nak errors 0, send timeouts 1, receive timeouts 0, finishTimeouts 1, resets 0
Is that what you would expect to see?
Cheers
Thanks for testing. I was expecting to see 1 reset as well, because it evidently had one I2C transaction error that it recovered from. I will check the code that counts resets.
EDIT: I omitted to increment the reset count at the proper place. I will fix this in the next RC or final release..
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 Cool. Do you want to carry on testing?
-
It appears I triggered a condition, and it appears to recover.
M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 2.03RC2-YG2 running on Duet Ethernet 1.02 or later + DueX5
Board ID: 08DGM-9T6BU-FG3S0-7JTD4-3S06K-1A4ZD
Used output buffers: 3 of 24 (18 max)
=== RTOS ===
Static ram: 25676
Dynamic ram: 96832 of which 0 recycled
Exception stack ram used: 396
Never used ram: 8168
Tasks: NETWORK(ready,648) HEAT(blocked,1232) DUEX(suspended,164) MAIN(running,1688) IDLE(ready,156)
Owned mutexes:
=== Platform ===
Last reset 00:32:18 ago, cause: power up
Last software reset time unknown, reason: User, spinning module GCodes, available RAM 8196 bytes (slot 0)
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0441f000 BFAR 0xe000ed38 SP 0xffffffff Task 0x4e49414d
Error status: 8
Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 0.0ms, max retries 0
MCU temperature: min 26.6, current 26.7, max 27.7
Supply voltage: min 24.3, current 24.6, max 24.7, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: standstill, SG min/max 0/93
Driver 1: standstill, SG min/max not available
Driver 2: standstill, SG min/max not available
Driver 3: standstill, SG min/max 0/242
Driver 4: standstill, SG min/max not available
Driver 5: standstill, SG min/max not available
Driver 6: standstill, SG min/max 83/432
Driver 7: standstill, SG min/max 39/449
Driver 8: standstill, SG min/max 0/413
Driver 9: standstill, SG min/max 29/422
Date/time: 2019-05-18 16:08:09
Cache data hit count 4294967295
Slowest loop: 30.12ms; fastest: 0.08ms
I2C nak errors 1, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Move ===
Hiccups: 0, FreeDm: 169, MinFreeDm: 163, MaxWait: 70775ms
Bed compensation in use: none
Bed probe heights: -0.004 0.002 -0.006 0.002 -0.044
=== DDARing ===
Scheduled moves: 434, completed moves: 434, StepErrors: 0, LaErrors: 0, Underruns: 0, 0
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1
Heater 0 is on, I-accum = 0.2
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is idle in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 6.51ms; fastest: 0.03ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
HTTP sessions: 1 of 8
Interface state 5, link 100Mbps full duplex -
@dc42 it recovered, thought not right away -- I have similar commands for each tpostx1.g
;M116 P1
G90
G1 S2 V611.7 F8000
G1 S2 V641.7 F8000
G1 S2 V611.7 F8000
G1 S2 V641.7 F8000
G1 S2 V611.7 F8000
G1 S2 V641.7 F8000
G1 S2 V611.7 F8000
G1 S2 V641.7 F8000
G1 S2 V611.7 F8000
M106 R2it does a wipe of a the nozzle across a brush, and this operation appeared to start operating in slow motion -- that is, there was a delay between each movement of the wipe, then after the tool change completed, the rest was fine and even the nak error count reset to 0 after another 30 minutes of configuration.
-
@dc42 back to slow motion.
M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 2.03RC2-YG2 running on Duet Ethernet 1.02 or later + DueX5
Board ID: 08DGM-9T6BU-FG3S0-7JTD4-3S06K-1A4ZD
Used output buffers: 1 of 24 (16 max)
=== RTOS ===
Static ram: 25676
Dynamic ram: 96832 of which 0 recycled
Exception stack ram used: 412
Never used ram: 8152
Tasks: NETWORK(ready,520) HEAT(blocked,1232) DUEX(suspended,164) MAIN(running,1688) IDLE(ready,156)
Owned mutexes:
=== Platform ===
Last reset 00:24:42 ago, cause: software
Last software reset at 2019-05-19 11:57, reason: User, spinning module GCodes, available RAM 8152 bytes (slot 3)
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0441f000 BFAR 0xe000ed38 SP 0xffffffff Task 0x4e49414d
Error status: 8
Free file entries: 9
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 75.7ms, max retries 0
MCU temperature: min 24.7, current 25.2, max 26.6
Supply voltage: min 24.2, current 24.5, max 24.7, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: standstill, SG min/max 0/109
Driver 1: standstill, SG min/max not available
Driver 2: standstill, SG min/max not available
Driver 3: standstill, SG min/max 0/225
Driver 4: standstill, SG min/max not available
Driver 5: standstill, SG min/max not available
Driver 6: standstill, SG min/max 89/428
Driver 7: standstill, SG min/max 9/440
Driver 8: standstill, SG min/max 0/415
Driver 9: standstill, SG min/max 12/433
Date/time: 2019-05-19 12:22:03
Cache data hit count 3661079197
Slowest loop: 121.00ms; fastest: 0.08ms
I2C nak errors 0, send timeouts 24138, receive timeouts 0, finishTimeouts 24138, resets 0
=== Move ===
Hiccups: 0, FreeDm: 169, MinFreeDm: 163, MaxWait: 315729ms
Bed compensation in use: none
Bed probe heights: -0.153 -0.198 0.104 0.111 0.058
=== DDARing ===
Scheduled moves: 359, completed moves: 359, StepErrors: 0, LaErrors: 0, Underruns: 0, 0
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1
Heater 0 is on, I-accum = 0.4
Heater 1 is on, I-accum = 0.0
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 1 in use
Movement lock held by aux
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is idle in state(s) 0
aux is idle in state(s) 42 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 124.37ms; fastest: 0.02ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
HTTP sessions: 2 of 8
Interface state 5, link 100Mbps full duplex -
Stopped being able to sense duex5 end stops entirely, removed the extra resistors, back to testing again.
-
@kazolar said in Duet sometimes really slow? - I2C error or?:
RepRapFirmware for Duet 2 WiFi/Ethernet version 2.03RC2-YG2 running on Duet Ethernet 1.02 or later + DueX5
I presume the -YG2 means you are using your own build of RRF. Are you certain that you have taken all of the 2.03RC2 changes into your build, in particular the changes to the CoreNG project?
-
Positive. It wouldn't compile otherwise. For whatever it's worth. Been running fine since I removed the resistors.
-
Thanks to @wilriker, I now have a version of firmware that incorporates the I2C changes but also restores end stop mapping so that I can use the exact same sequence of events that were proven to consistently provoke the errors. I tried it earlier today with no issues but that was not from an overnight shutdown. I'll try it again first thing tomorrow morning.
-
@dc42 David,
For info, I ran "the sequence" again today after an over night shutdown, with @wilriker 's firmware 2.03RC3-M574C (2019-05-28b1) (based on your firmware but with end stop mapping enabled). No problems encountered with movement, no I2C errors reported, and no I2C resets.
Looking good so far........
-
@deckingman said in Duet sometimes really slow? - I2C error or?:
@dc42 David,
For info, I ran "the sequence" again today after an over night shutdown, with @wilriker 's firmware 2.03RC3-M574C (2019-05-28b1) (based on your firmware but with end stop mapping enabled). No problems encountered with movement, no I2C errors reported, and no I2C resets.
Looking good so far........
Ian, thanks for the feedback. To be honest, I think you were just lucky that you had no I2C errors on this occasion. However, given the reports of behaviour when you and others tested the 2.03 releases, I am confident that the changes I made to the I2C driver have completely or at least partially solved the original issue. These changes (as well as all the other improvements in the 2.03RC) releases are already in the 3.0beta source code.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@dc42 OK. I'll keep testing but only report back if something amiss happens. So no news will be good news.
Errrrr, just had a thought. Since last time when I could provoke errors consistently, I've changed my motor mounts. So when you said, I was just lucky I started to wonder....
These new mounts are aluminium and the old ones were plastic. So the "XYUVWA" motors will now be earthed through the mount and frame whereas before, they were insulated from the frame by the plastic mounts (which was one of the reasons fort changing them). Might it be possible that the reason why I didn't see I2C errors is not luck but more to do with the fact that I've earthed some of the steppers?
I'm not doubting that the firmware changes have fixed the issues but wondering if the root cause was stepper noise, which can be mitigated by earthing the steppers. Thoughts?
I guess it would be interesting to hear from other users if their stepper motors are earthed or not.
-
Ignore comments above above about my changing the motor mounts. Having run "the sequence" again today, subsequent M122 report shows:
I2C nak errors 0, send timeouts 1, receive timeouts 0, finishTimeouts 1, resets 1
My take on that is I2C errors occurred but the firmware caught and fixed any problem. So earthing the motors through the mounts didn't affect behaviour.
-
@deckingman talking of earths have you got the earth spade near the ethernet connection grounded? Doubt this will effect your issue bit worth doing.
Running 3 P3Steel with Duet 2. Duet 3 on the shelf looking for a suitable machine. One first generation Duet in a Logo/Turtle style robot!
-
@deckingman said in Duet sometimes really slow? - I2C error or?:
I'm not doubting that the firmware changes have fixed the issues but wondering if the root cause was stepper noise, which can be mitigated by earthing the steppers. Thoughts?
Un-earthed stepper motors driving rubber belts can build up static charge (think Van de Graaff generator). If the stepper motors are well-insulated from the printer frame via plastic mounts, then the shortest path for the static charge to arc to ground may be to the stepper motor wires. That will in turn cause a ground transient on the Duet or the Duex (whichever one the stepper motor is connected to), and this will affect any I2C transaction that is in progress. So yes, using metal stepper motor mounts may have fixed the root cause.
But I see from your more recent post that you have had another I2C error.
Duet WiFi hardware designer and firmware engineer
Please do not ask me for Duet support via PM or email, use the forum
http://www.escher3d.com, https://miscsolutions.wordpress.com -
@doctrucker said in Duet sometimes really slow? - I2C error or?:
@deckingman talking of earths have you got the earth spade near the ethernet connection grounded? Doubt this will effect your issue bit worth doing.
What earth spade? Mine is a pre-production or at least the very first production Ethernet boards so it probably doesn't have one.
-
@dc42 Yes I'm aware of the Van de Graff issues with belts - can cause all sorts of problems with model helicopters I believe because the tail rotor is often driven by a continuous belt and the resultant static plays havoc with the receiver. Also anti-static belts are often used in conveyor systems.
But I wonder if it's an issue with printers because generally the belts constantly change direction back and forth rather than always rotate in the same direction. It's an awfully long time since since schoolboy physics but if one reverses the direction of a Van de Graff generator, doesn't that reverse the polarity of the charge?
