Duet3D Logo Duet3D
    • Tags
    • Documentation
    • Order
    • Register
    • Login

    Duet sometimes really slow? - I2C error or?

    Scheduled Pinned Locked Moved
    Other control boards
    13
    147
    17.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • kazolarundefined
      kazolar
      last edited by

      @dc42 I use silicone wire for moving axis, so twisting them doesn't work particularly well, that's where I'm seeing issues. Stationary end stops, I can easily isolate and eliminate all interference. I have 2 moving gantries, I have 2 end stops on each, primary x gantry has both end stops connected to the duet, no issues on it, the 2nd gantry has end stops going to the duex5, that's where I've had issues, originally on the A axis, now on the W axis. I've isolated both runs in their own nylon shielding, and I'm hoping that solves the problem. At least it solved it for the mean time and I was able to home. I have been using normally closed end stops for some time. Even with the wire which got all wrapped up around the stepper wire, end stop was reacting, and m119 was reporting it properly, but clearly there was significant enough intereference on the line that with the new firmware it wouldn't home, trigger a full reboot, interestingly enough it homed fine in the previous version, so if you added an error condition, I must have hit it. Just wondering if there is some filtering we can add to the endstop input itself to mitigate the issue.

      dc42undefined 1 Reply Last reply Reply Quote 0
      • dc42undefined
        dc42 administrators @kazolar
        last edited by

        @kazolar said in Duet sometimes really slow? - I2C error or?:

        Just wondering if there is some filtering we can add to the endstop input itself to mitigate the issue.

        You could try a 100 or 200 ohm resistor in series with the wire to the STP pin of the endstop connector, and a 10nF capacitor between the STP and GND pins.

        If I was certain that the problem was interference on the endstop wires, then we could add filtering to the next revision of the DueX. However, I'm fairly sure that some users have reported getting this i2C comms failure even with no endstops connected to the DueX.

        Duet WiFi hardware designer and firmware engineer
        Please do not ask me for Duet support via PM or email, use the forum
        http://www.escher3d.com, https://miscsolutions.wordpress.com

        1 Reply Last reply Reply Quote 0
        • kazolarundefined
          kazolar
          last edited by

          There appears to be 3 vectors, end stops, thermistors and fans. I have not had any issues with fans. May just be related to my fan choices. I have had to isolate every endstop going to the duex5, as well as every thermistor going to the duex5. Obviously big wire bundles which include stepper wires are an interference nightmare. I'll run through a calibration routine tonight, as I level out each tool to each other, previously that was a guaranteed way to trigger i2c timeout issues. I may add the caps and filters proactively even if I things are working fine.

          1 Reply Last reply Reply Quote 0
          • DocTruckerundefined
            DocTrucker
            last edited by

            I had issues with interference and false trips on a Duet Ethernet with Duex5 when I used a common for the switches to save wiring two cores back to the board for each switch. Swapped back to two cores for each switch and the interference went.

            Running 3 P3Steel with Duet 2. Duet 3 on the shelf looking for a suitable machine. One first generation Duet in a Logo/Turtle style robot!

            1 Reply Last reply Reply Quote 0
            • deckingmanundefined
              deckingman @kazolar
              last edited by

              @kazolar said in Duet sometimes really slow? - I2C error or?:

              I am almost positive at this point these issues have to do with end stop wires having interference on them from the stepper wiring. ....................................

              That might be true in your case but for me, I didn't have any I2C issues from when I installed the board in December 2016 until July(ish) 2018. As far as I can recall, I didn't make any hardware or wiring changes at that time. Also, there are no other threads related to I2C errors which pre-date July 2018 and that date is coincidental with the introduction of RTOS 2.0 firmware.

              Ian
              https://somei3deas.wordpress.com/
              https://www.youtube.com/@deckingman

              1 Reply Last reply Reply Quote 1
              • kazolarundefined
                kazolar
                last edited by

                Yes, I didn't have these problems with the 1.x versions, there was an i2c rewrite in 2.0, I couldn't go back to a 2.0 version which predates the rewrite easily as I have some custom changes I need to apply and I need to compile the firmware myself. What appears to happen is the i2c rewrite brought those hardware issues to the surface, because i can't blame the code if replacing the end stop wire fixes a problem. Just because it homed fine before the code change, simply says to me that there was hardware (wiring issue) and the new code is just exposing it. I'd rather shake out the wiring bugs now than have them bite me during a 100 hour print.

                1 Reply Last reply Reply Quote 0
                • deckingmanundefined
                  deckingman @dc42
                  last edited by

                  @dc42 said in Duet sometimes really slow? - I2C error or?:

                  Ian, you can test the status of the endstops on the DueX by creating additional axes and making them visible. If you create 3 axes then the third one will use the E2 endstop input on the Due X. When that axis is visible, the state of the E2 endstop will show up in the Machine Properties page of the old DWC, and in the M119 response.

                  Yes I thought about that but maybe you don't remember the problems I had when I tried it before - this thread https://forum.duet3d.com/topic/6677/corexyuvwa-3rd-gantry-homing-solved/. I could only get it to work with the introduction of 2.03 beta2 when you introduced mapping of end stops to axes which is not possible in this RC candidate.

                  I'll see if a can cobble together some sort of configuration that'll make the end stops visible but as I said before, I won't be able to run the exact same sequence that was proven to provoke the problem unless I can map those additional end stops to the XY axes.

                  Just out of curiosity, why was that facility withdrawn and can you confirm that it'll be available in 3.0?

                  Cheers

                  Ian
                  https://somei3deas.wordpress.com/
                  https://www.youtube.com/@deckingman

                  dc42undefined 1 Reply Last reply Reply Quote 0
                  • dc42undefined
                    dc42 administrators @deckingman
                    last edited by

                    @deckingman said in Duet sometimes really slow? - I2C error or?:

                    Just out of curiosity, why was that facility withdrawn and can you confirm that it'll be available in 3.0?

                    It messed up the M585 command, which is needed by CNC users, and there was no easy fix. Endstop inputs can be mapped in firmware 3.0, see https://duet3d.dozuki.com/Wiki/RepRapFirmware_3_overview.

                    Duet WiFi hardware designer and firmware engineer
                    Please do not ask me for Duet support via PM or email, use the forum
                    http://www.escher3d.com, https://miscsolutions.wordpress.com

                    deckingmanundefined 1 Reply Last reply Reply Quote 0
                    • deckingmanundefined
                      deckingman @dc42
                      last edited by

                      @dc42

                      David,

                      I just ran "the sequence" as far as I am able (everything apart from homing the upper gantry with mapped endstops). After which I changed (lowered) the temperature threshold for one of the thermostatic fans connected to the Duex5, to force the fan to come on which it did. Then I toggled the E2 end stop whilst observing the status on the machine properties page and that all worked as normal. After which I ran M122 and noted the following:

                      I2C nak errors 0, send timeouts 1, receive timeouts 0, finishTimeouts 1, resets 0

                      Is that what you would expect to see?

                      Cheers

                      Ian
                      https://somei3deas.wordpress.com/
                      https://www.youtube.com/@deckingman

                      dc42undefined 1 Reply Last reply Reply Quote 0
                      • dc42undefined
                        dc42 administrators @deckingman
                        last edited by dc42

                        @deckingman said in Duet sometimes really slow? - I2C error or?:

                        @dc42

                        David,

                        I just ran "the sequence" as far as I am able (everything apart from homing the upper gantry with mapped endstops). After which I changed (lowered) the temperature threshold for one of the thermostatic fans connected to the Duex5, to force the fan to come on which it did. Then I toggled the E2 end stop whilst observing the status on the machine properties page and that all worked as normal. After which I ran M122 and noted the following:

                        I2C nak errors 0, send timeouts 1, receive timeouts 0, finishTimeouts 1, resets 0

                        Is that what you would expect to see?

                        Cheers

                        Thanks for testing. I was expecting to see 1 reset as well, because it evidently had one I2C transaction error that it recovered from. I will check the code that counts resets.

                        EDIT: I omitted to increment the reset count at the proper place. I will fix this in the next RC or final release..

                        Duet WiFi hardware designer and firmware engineer
                        Please do not ask me for Duet support via PM or email, use the forum
                        http://www.escher3d.com, https://miscsolutions.wordpress.com

                        deckingmanundefined 1 Reply Last reply Reply Quote 0
                        • deckingmanundefined
                          deckingman @dc42
                          last edited by

                          @dc42 Cool. Do you want to carry on testing?

                          Ian
                          https://somei3deas.wordpress.com/
                          https://www.youtube.com/@deckingman

                          1 Reply Last reply Reply Quote 0
                          • kazolarundefined
                            kazolar
                            last edited by

                            It appears I triggered a condition, and it appears to recover.
                            M122
                            === Diagnostics ===
                            RepRapFirmware for Duet 2 WiFi/Ethernet version 2.03RC2-YG2 running on Duet Ethernet 1.02 or later + DueX5
                            Board ID: 08DGM-9T6BU-FG3S0-7JTD4-3S06K-1A4ZD
                            Used output buffers: 3 of 24 (18 max)
                            === RTOS ===
                            Static ram: 25676
                            Dynamic ram: 96832 of which 0 recycled
                            Exception stack ram used: 396
                            Never used ram: 8168
                            Tasks: NETWORK(ready,648) HEAT(blocked,1232) DUEX(suspended,164) MAIN(running,1688) IDLE(ready,156)
                            Owned mutexes:
                            === Platform ===
                            Last reset 00:32:18 ago, cause: power up
                            Last software reset time unknown, reason: User, spinning module GCodes, available RAM 8196 bytes (slot 0)
                            Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0441f000 BFAR 0xe000ed38 SP 0xffffffff Task 0x4e49414d
                            Error status: 8
                            Free file entries: 10
                            SD card 0 detected, interface speed: 20.0MBytes/sec
                            SD card longest block write time: 0.0ms, max retries 0
                            MCU temperature: min 26.6, current 26.7, max 27.7
                            Supply voltage: min 24.3, current 24.6, max 24.7, under voltage events: 0, over voltage events: 0, power good: yes
                            Driver 0: standstill, SG min/max 0/93
                            Driver 1: standstill, SG min/max not available
                            Driver 2: standstill, SG min/max not available
                            Driver 3: standstill, SG min/max 0/242
                            Driver 4: standstill, SG min/max not available
                            Driver 5: standstill, SG min/max not available
                            Driver 6: standstill, SG min/max 83/432
                            Driver 7: standstill, SG min/max 39/449
                            Driver 8: standstill, SG min/max 0/413
                            Driver 9: standstill, SG min/max 29/422
                            Date/time: 2019-05-18 16:08:09
                            Cache data hit count 4294967295
                            Slowest loop: 30.12ms; fastest: 0.08ms
                            I2C nak errors 1, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
                            === Move ===
                            Hiccups: 0, FreeDm: 169, MinFreeDm: 163, MaxWait: 70775ms
                            Bed compensation in use: none
                            Bed probe heights: -0.004 0.002 -0.006 0.002 -0.044
                            === DDARing ===
                            Scheduled moves: 434, completed moves: 434, StepErrors: 0, LaErrors: 0, Underruns: 0, 0
                            === Heat ===
                            Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1
                            Heater 0 is on, I-accum = 0.2
                            === GCodes ===
                            Segments left: 0
                            Stack records: 2 allocated, 0 in use
                            Movement lock held by null
                            http is idle in state(s) 0
                            telnet is idle in state(s) 0
                            file is idle in state(s) 0
                            serial is idle in state(s) 0
                            aux is idle in state(s) 0
                            daemon is idle in state(s) 0
                            queue is idle in state(s) 0
                            autopause is idle in state(s) 0
                            Code queue is empty.
                            === Network ===
                            Slowest loop: 6.51ms; fastest: 0.03ms
                            Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
                            HTTP sessions: 1 of 8
                            Interface state 5, link 100Mbps full duplex

                            1 Reply Last reply Reply Quote 0
                            • kazolarundefined
                              kazolar
                              last edited by

                              @dc42 it recovered, thought not right away -- I have similar commands for each tpostx1.g

                              ;M116 P1
                              G90
                              G1 S2 V611.7 F8000
                              G1 S2 V641.7 F8000
                              G1 S2 V611.7 F8000
                              G1 S2 V641.7 F8000
                              G1 S2 V611.7 F8000
                              G1 S2 V641.7 F8000
                              G1 S2 V611.7 F8000
                              G1 S2 V641.7 F8000
                              G1 S2 V611.7 F8000
                              M106 R2

                              it does a wipe of a the nozzle across a brush, and this operation appeared to start operating in slow motion -- that is, there was a delay between each movement of the wipe, then after the tool change completed, the rest was fine and even the nak error count reset to 0 after another 30 minutes of configuration.

                              1 Reply Last reply Reply Quote 0
                              • kazolarundefined
                                kazolar
                                last edited by

                                @dc42 back to slow motion.

                                M122
                                === Diagnostics ===
                                RepRapFirmware for Duet 2 WiFi/Ethernet version 2.03RC2-YG2 running on Duet Ethernet 1.02 or later + DueX5
                                Board ID: 08DGM-9T6BU-FG3S0-7JTD4-3S06K-1A4ZD
                                Used output buffers: 1 of 24 (16 max)
                                === RTOS ===
                                Static ram: 25676
                                Dynamic ram: 96832 of which 0 recycled
                                Exception stack ram used: 412
                                Never used ram: 8152
                                Tasks: NETWORK(ready,520) HEAT(blocked,1232) DUEX(suspended,164) MAIN(running,1688) IDLE(ready,156)
                                Owned mutexes:
                                === Platform ===
                                Last reset 00:24:42 ago, cause: software
                                Last software reset at 2019-05-19 11:57, reason: User, spinning module GCodes, available RAM 8152 bytes (slot 3)
                                Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0441f000 BFAR 0xe000ed38 SP 0xffffffff Task 0x4e49414d
                                Error status: 8
                                Free file entries: 9
                                SD card 0 detected, interface speed: 20.0MBytes/sec
                                SD card longest block write time: 75.7ms, max retries 0
                                MCU temperature: min 24.7, current 25.2, max 26.6
                                Supply voltage: min 24.2, current 24.5, max 24.7, under voltage events: 0, over voltage events: 0, power good: yes
                                Driver 0: standstill, SG min/max 0/109
                                Driver 1: standstill, SG min/max not available
                                Driver 2: standstill, SG min/max not available
                                Driver 3: standstill, SG min/max 0/225
                                Driver 4: standstill, SG min/max not available
                                Driver 5: standstill, SG min/max not available
                                Driver 6: standstill, SG min/max 89/428
                                Driver 7: standstill, SG min/max 9/440
                                Driver 8: standstill, SG min/max 0/415
                                Driver 9: standstill, SG min/max 12/433
                                Date/time: 2019-05-19 12:22:03
                                Cache data hit count 3661079197
                                Slowest loop: 121.00ms; fastest: 0.08ms
                                I2C nak errors 0, send timeouts 24138, receive timeouts 0, finishTimeouts 24138, resets 0
                                === Move ===
                                Hiccups: 0, FreeDm: 169, MinFreeDm: 163, MaxWait: 315729ms
                                Bed compensation in use: none
                                Bed probe heights: -0.153 -0.198 0.104 0.111 0.058
                                === DDARing ===
                                Scheduled moves: 359, completed moves: 359, StepErrors: 0, LaErrors: 0, Underruns: 0, 0
                                === Heat ===
                                Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1
                                Heater 0 is on, I-accum = 0.4
                                Heater 1 is on, I-accum = 0.0
                                === GCodes ===
                                Segments left: 0
                                Stack records: 2 allocated, 1 in use
                                Movement lock held by aux
                                http is idle in state(s) 0
                                telnet is idle in state(s) 0
                                file is idle in state(s) 0
                                serial is idle in state(s) 0
                                aux is idle in state(s) 42 0
                                daemon is idle in state(s) 0
                                queue is idle in state(s) 0
                                autopause is idle in state(s) 0
                                Code queue is empty.
                                === Network ===
                                Slowest loop: 124.37ms; fastest: 0.02ms
                                Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
                                HTTP sessions: 2 of 8
                                Interface state 5, link 100Mbps full duplex

                                dc42undefined 1 Reply Last reply Reply Quote 0
                                • kazolarundefined
                                  kazolar
                                  last edited by

                                  Stopped being able to sense duex5 end stops entirely, removed the extra resistors, back to testing again.

                                  1 Reply Last reply Reply Quote 0
                                  • dc42undefined
                                    dc42 administrators @kazolar
                                    last edited by

                                    @kazolar said in Duet sometimes really slow? - I2C error or?:

                                    RepRapFirmware for Duet 2 WiFi/Ethernet version 2.03RC2-YG2 running on Duet Ethernet 1.02 or later + DueX5

                                    I presume the -YG2 means you are using your own build of RRF. Are you certain that you have taken all of the 2.03RC2 changes into your build, in particular the changes to the CoreNG project?

                                    Duet WiFi hardware designer and firmware engineer
                                    Please do not ask me for Duet support via PM or email, use the forum
                                    http://www.escher3d.com, https://miscsolutions.wordpress.com

                                    1 Reply Last reply Reply Quote 0
                                    • kazolarundefined
                                      kazolar
                                      last edited by kazolar

                                      Positive. It wouldn't compile otherwise. For whatever it's worth. Been running fine since I removed the resistors.

                                      1 Reply Last reply Reply Quote 0
                                      • deckingmanundefined
                                        deckingman
                                        last edited by deckingman

                                        Thanks to @wilriker, I now have a version of firmware that incorporates the I2C changes but also restores end stop mapping so that I can use the exact same sequence of events that were proven to consistently provoke the errors. I tried it earlier today with no issues but that was not from an overnight shutdown. I'll try it again first thing tomorrow morning.

                                        Ian
                                        https://somei3deas.wordpress.com/
                                        https://www.youtube.com/@deckingman

                                        1 Reply Last reply Reply Quote 0
                                        • deckingmanundefined
                                          deckingman
                                          last edited by

                                          @dc42 David,

                                          For info, I ran "the sequence" again today after an over night shutdown, with @wilriker 's firmware 2.03RC3-M574C (2019-05-28b1) (based on your firmware but with end stop mapping enabled). No problems encountered with movement, no I2C errors reported, and no I2C resets.

                                          Looking good so far........

                                          Ian
                                          https://somei3deas.wordpress.com/
                                          https://www.youtube.com/@deckingman

                                          dc42undefined 1 Reply Last reply Reply Quote 0
                                          • dc42undefined
                                            dc42 administrators @deckingman
                                            last edited by dc42

                                            @deckingman said in Duet sometimes really slow? - I2C error or?:

                                            @dc42 David,

                                            For info, I ran "the sequence" again today after an over night shutdown, with @wilriker 's firmware 2.03RC3-M574C (2019-05-28b1) (based on your firmware but with end stop mapping enabled). No problems encountered with movement, no I2C errors reported, and no I2C resets.

                                            Looking good so far........

                                            Ian, thanks for the feedback. To be honest, I think you were just lucky that you had no I2C errors on this occasion. However, given the reports of behaviour when you and others tested the 2.03 releases, I am confident that the changes I made to the I2C driver have completely or at least partially solved the original issue. These changes (as well as all the other improvements in the 2.03RC) releases are already in the 3.0beta source code.

                                            Duet WiFi hardware designer and firmware engineer
                                            Please do not ask me for Duet support via PM or email, use the forum
                                            http://www.escher3d.com, https://miscsolutions.wordpress.com

                                            deckingmanundefined 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Unless otherwise noted, all forum content is licensed under CC-BY-SA