Printer pausing between commands

gnydick

I must be blessed (cursed)... Check it out... This just started happening while it was printing...

Then, I cancelled the print and lowered the bed and it happened again. So I recorded raising the bed. I clicked -25 in rapid fire succession and got this result...

https://youtu.be/jSBE7MjouSw

One possibility. I found I have a DC power brick plugged into the same circuit that is SQUEEEEELING, I think a cap is ready to blow. That could be interfering.

dc42

Do you have a DueX in your system? That behaviour can occur if I2C communication between the Duet and the DueX breaks down. The I2C stats in the M122 report will tell you if that has happened.

gnydick

so how do I fix it? The boards are beautifully and rigidly mounted, there's absolutely no movement.

M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 2.02(RTOS) running on Duet WiFi 1.02 or later + DueX5
Board ID: 08DGM-917NK-F23T0-6J1F2-3SD6T-KGBWD
Used output buffers: 1 of 20 (13 max)
=== RTOS ===
Static ram: 25524
Dynamic ram: 99004 of which 0 recycled
Exception stack ram used: 512
Never used ram: 6032
Tasks: NETWORK(ready,544) HEAT(blocked,1232) MAIN(running,3780) IDLE(ready,200)
Owned mutexes:
=== Platform ===
Last reset 08:13:04 ago, cause: software
Last software reset at 2019-04-30 23:26, reason: User, spinning module GCodes, available RAM 6032 bytes (slot 0)
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0441f000 BFAR 0xe000ed38 SP 0xffffffff Task 0x4e49414d
Error status: 0
Free file entries: 9
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 14.2ms, max retries 0
MCU temperature: min 31.0, current 31.9, max 37.4
Supply voltage: min 23.5, current 24.7, max 24.9, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: standstill, SG min/max 0/1023
Driver 1: standstill, SG min/max 0/1023
Driver 2: standstill, SG min/max not available
Driver 3: standstill, SG min/max 0/1023
Driver 4: standstill, SG min/max not available
Driver 5: standstill, SG min/max 0/189
Driver 6: standstill, SG min/max 0/234
Driver 7: standstill, SG min/max 0/215
Driver 8: standstill, SG min/max 0/184
Driver 9: standstill, SG min/max not available
Date/time: 2019-05-01 07:39:33
Cache data hit count 4294967295
Slowest loop: 136.20ms; fastest: 0.07ms
I2C nak errors 0, send timeouts 4073957, receive timeouts 0, finishTimeouts 4073957
=== Move ===
Hiccups: 0, StepErrors: 0, LaErrors: 0, FreeDm: 240, MinFreeDm: 150, MaxWait: 19648172ms, Underruns: 0, 91
Scheduled moves: 5, completed moves: 5
Bed compensation in use: none
Bed probe heights: 0.000 0.000 0.000 0.000 0.000
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1
Heater 0 is on, I-accum = 0.4
Heater 1 is on, I-accum = 0.6
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is idle in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 121.94ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0) Telnet(0)
HTTP sessions: 1 of 8
- WiFi -
Network state is running
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.22
WiFi MAC address b4:e6:2d:53:0c:9a
WiFi Vcc 3.36, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 27816
WiFi IP address 192.168.1.200
WiFi signal strength -37dBm, reconnections 0, sleep mode modem
Socket states: 0 0 0 0 0 0 0 0

Veti

did you cover all the points mentioned here?
https://duet3d.dozuki.com/Wiki/Duex2_and_Duex5_Features#Section_Wiring

deckingman

There are quite a few threads about this. Here is one for example https://forum.duet3d.com/topic/7269/duet-sometimes-really-slow-i2c-error-or

It's an ongoing issue for some of us. Because it is so intermittent and one can print for months without any problems, I can understand why it is proving to be difficult to find the cause, but it is frustrating.

At one point it was thought that fitting some extra resistors would cure it and that is what I did in the thread that I linked to. This was promising and seemed to work but then the problem happened again some 2 1/2 month after fitting the resistors and has recurred a couple of times since. Unfortunately, that is the nature of the beast - everything can run fine for months, then the problem occurs. Cycling the power will always clear it and it will then be fine for another long period of time, but it will happen again and it is frustrating for all concerned.

gnydick

@deckingman yeah, I found that thread. Seems like it's happening at the same exact spot in the print. I'm guessing there is a bug that's being tickled. If you re-run a print that fails in the middle, does it fail at the same time every time?

gnydick

@dc42 I've gone through the other threads, have tried everything listed minus the resistors, I'm not comfortable trying that. I've also removed all functionality from the duex5 except for motors. I'm starting another print and I'll let you know how it goes. FWIW, the only other thing I had running off the duex5 was the nozzle fan.

deckingman

@gnydick said in Printer pausing between commands:

@deckingman .............................If you re-run a print that fails in the middle, does it fail at the same time every time?

No never. I wish it would, then we might be in with a chance of finding the root cause.

In my experience, it has been completely random. It may just be co-incidental but it seems that the problem is more likely to manifest itself after the printer has been switched off and idle for an extended period of time. I have no idea how this could possibly be anything other than coincidence unless something on a gantry tightens up which puts a bit more load on a motor than is normal. I'm clutching at straws here and it's probably just co-incidental. But always a power cycle will clear it and it might not manifest itself for months.

deckingman

@gnydick If you take a gander at the picture I posted in the other thread, I used some hook up wires that had sockets on the end which plug onto the pins on the connector. I just cut the wires and soldered the resistors with bit of heat shrink over the joints. Then just plugged them in. That way, there was no chance of me accidentally shorting something out.

EDIT - but the problem still occurs with the resistors........

2nd Edit. I didn't have any problems with I2C errors for the first year or so of using the Duex5 - it's only been the second year or so.

gnydick

@deckingman my printer is almost never idle, and if the resistors didn't help, why bother Also, I've only had my duex5 for a couple months.

deckingman

@dc42 David, I've been trying to do some detective work on this I2C issue.

AFAIK, I was the first one to report the problem and it was in this thread https://forum.duet3d.com/topic/7513/i-think-my-duet-and-or-duex5-is-are-dying that I started on 1st November 2018. The problem started to occur a short time (say 2 to 4 months or so) prior to me starting that thread. So somewhere around 1st July to 1 September 2018 give or take a month or two.

As far as I have been able to ascertain, nobody reported any I2C problems prior to that date.

Now going back in time, I can see that my Duex 5 was one one of the very first pre-production boards which I acquired around December 2016. So I think it's fair to say that the issue affects all Duex5 boards and not just recent hardware versions. I can also say that I had no I2C issues from Dec 2016 to July/September (ish) 2018.

Trying to ascertain what might have changed by looking back through my blog and design files and so forth, I can see that around the time I started seeing the errors was when I started to implement my 3rd load balancing gantry. So it was at that point when I added 2 extra motors to the Duex5 making 5 in total as opposed to 3 prior to that date. This might be a clue.

However, although that might be a clue as to why the errors started to appear on my machine, it doesn't explain why other people started to get these errors. Since my original thread dated 1st November 2018 there have been a number of other threads reporting similar I2C errors but there are no threads which pre-date my original one. Which would reinforce my theory that the errors did not manifest themselves prior to that date. Of course it may also be that it is only since November 2018 that other people have been buying Duex 5 boards but you could easily check that.

One thing that is common to all all users and which changed around that time is the firmware. I can't off hand find the exact release dates but from the folders that I create when I download firmware, I can see that on 26th July I installed RTOS 2.01 and that is what I was using when I started that thread on 1st November which itself was 2 to 4 months after I first noticed a problem.

Maybe it's just co-incidental but it seems that the I2C errors are something that has started happening since the firmware changed to 2.n RTOS. Far be it for me to suggest that the firmware might be at fault but I'm always suspicious of coincidences. Might it be worth checking if something happened when changing to RTOS that might have an impact on these I2C errors?

dc42

@gnydick said in Printer pausing between commands:

@deckingman yeah, I found that thread. Seems like it's happening at the same exact spot in the print. I'm guessing there is a bug that's being tickled. If you re-run a print that fails in the middle, does it fail at the same time every time?

Are you able to reproduce it in that print? If so, can you still reproduce it if you delete from the GCode file all or most of the layers before the one in which it occurs? If someone can give me a reproducible test case, I can debug it.

gnydick

@dc42 good idea, i'll try

Phaedrux

@deckingman said in Printer pausing between commands:

AFAIK, I was the first one to report the problem

Earliest I can recall is this one about a Voron2 printer with 4 z motors and firmware 2.0 from July 2018, which would line up. https://forum.duet3d.com/topic/6077/inconsistent-delays-during-homing-and-other-movements

gnydick

@gnydick So, to repro, first I'm printing the entire thing again to find the right layer height at which to stop. Well, it's gotten much further than it has before, so I'm flummoxed. Not sure what to do next.

Oh, and, of course it works when I'm using crappy filament, but it fails on me when I'm using my Prusament.

Oh, and based on my failed prints, they were failing at different times, but close.

deckingman

@gnydick I suspect it's just random, like it s for everyone else. Which is a pity because it means that @dc42 won't be able to reproduce it.

deckingman

@phaedrux said in Printer pausing between commands:

@deckingman said in Printer pausing between commands:

AFAIK, I was the first one to report the problem

Earliest I can recall is this one about a Voron2 printer with 4 z motors and firmware 2.0 from July 2018, which would line up. https://forum.duet3d.com/topic/6077/inconsistent-delays-during-homing-and-other-movements

Well done Watson. It would seem to be the same issue and as you say, the timing would coincide with the introduction of firmware 2.0.

I wonder if we have eliminated all other possibilities and are now at the "“When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.” stage.

Sherlock.

samlogan87

@deckingman I have experienced these issues with my Duex2 board as well. It has only happened twice and both times have been intermittent and a power cycle has fixed it.

Sam

gnydick

@samlogan87 mine has just miraculously stopped happening. I'm attributing it to disconnecting the lamp with the squeeeling power supply.

dc42

Those of you who have experienced this issue, did you have anything connected to the endstop inputs on the DueX5, or to the I2C pins on the 10-pin header (other than perhaps the resistor I suggested), or to the GPIO pins on the 10-pin header?