Printer pausing between commands

deckingman

@gnydick said in Printer pausing between commands:

@deckingman .............................If you re-run a print that fails in the middle, does it fail at the same time every time?

No never. I wish it would, then we might be in with a chance of finding the root cause.

In my experience, it has been completely random. It may just be co-incidental but it seems that the problem is more likely to manifest itself after the printer has been switched off and idle for an extended period of time. I have no idea how this could possibly be anything other than coincidence unless something on a gantry tightens up which puts a bit more load on a motor than is normal. I'm clutching at straws here and it's probably just co-incidental. But always a power cycle will clear it and it might not manifest itself for months.

deckingman

@gnydick If you take a gander at the picture I posted in the other thread, I used some hook up wires that had sockets on the end which plug onto the pins on the connector. I just cut the wires and soldered the resistors with bit of heat shrink over the joints. Then just plugged them in. That way, there was no chance of me accidentally shorting something out.

EDIT - but the problem still occurs with the resistors........

2nd Edit. I didn't have any problems with I2C errors for the first year or so of using the Duex5 - it's only been the second year or so.

gnydick

@deckingman my printer is almost never idle, and if the resistors didn't help, why bother Also, I've only had my duex5 for a couple months.

deckingman

@dc42 David, I've been trying to do some detective work on this I2C issue.

AFAIK, I was the first one to report the problem and it was in this thread https://forum.duet3d.com/topic/7513/i-think-my-duet-and-or-duex5-is-are-dying that I started on 1st November 2018. The problem started to occur a short time (say 2 to 4 months or so) prior to me starting that thread. So somewhere around 1st July to 1 September 2018 give or take a month or two.

As far as I have been able to ascertain, nobody reported any I2C problems prior to that date.

Now going back in time, I can see that my Duex 5 was one one of the very first pre-production boards which I acquired around December 2016. So I think it's fair to say that the issue affects all Duex5 boards and not just recent hardware versions. I can also say that I had no I2C issues from Dec 2016 to July/September (ish) 2018.

Trying to ascertain what might have changed by looking back through my blog and design files and so forth, I can see that around the time I started seeing the errors was when I started to implement my 3rd load balancing gantry. So it was at that point when I added 2 extra motors to the Duex5 making 5 in total as opposed to 3 prior to that date. This might be a clue.

However, although that might be a clue as to why the errors started to appear on my machine, it doesn't explain why other people started to get these errors. Since my original thread dated 1st November 2018 there have been a number of other threads reporting similar I2C errors but there are no threads which pre-date my original one. Which would reinforce my theory that the errors did not manifest themselves prior to that date. Of course it may also be that it is only since November 2018 that other people have been buying Duex 5 boards but you could easily check that.

One thing that is common to all all users and which changed around that time is the firmware. I can't off hand find the exact release dates but from the folders that I create when I download firmware, I can see that on 26th July I installed RTOS 2.01 and that is what I was using when I started that thread on 1st November which itself was 2 to 4 months after I first noticed a problem.

Maybe it's just co-incidental but it seems that the I2C errors are something that has started happening since the firmware changed to 2.n RTOS. Far be it for me to suggest that the firmware might be at fault but I'm always suspicious of coincidences. Might it be worth checking if something happened when changing to RTOS that might have an impact on these I2C errors?

dc42

@gnydick said in Printer pausing between commands:

@deckingman yeah, I found that thread. Seems like it's happening at the same exact spot in the print. I'm guessing there is a bug that's being tickled. If you re-run a print that fails in the middle, does it fail at the same time every time?

Are you able to reproduce it in that print? If so, can you still reproduce it if you delete from the GCode file all or most of the layers before the one in which it occurs? If someone can give me a reproducible test case, I can debug it.

gnydick

@dc42 good idea, i'll try

Phaedrux

@deckingman said in Printer pausing between commands:

AFAIK, I was the first one to report the problem

Earliest I can recall is this one about a Voron2 printer with 4 z motors and firmware 2.0 from July 2018, which would line up. https://forum.duet3d.com/topic/6077/inconsistent-delays-during-homing-and-other-movements

gnydick

@gnydick So, to repro, first I'm printing the entire thing again to find the right layer height at which to stop. Well, it's gotten much further than it has before, so I'm flummoxed. Not sure what to do next.

Oh, and, of course it works when I'm using crappy filament, but it fails on me when I'm using my Prusament.

Oh, and based on my failed prints, they were failing at different times, but close.

deckingman

@gnydick I suspect it's just random, like it s for everyone else. Which is a pity because it means that @dc42 won't be able to reproduce it.

deckingman

@phaedrux said in Printer pausing between commands:

@deckingman said in Printer pausing between commands:

AFAIK, I was the first one to report the problem

Earliest I can recall is this one about a Voron2 printer with 4 z motors and firmware 2.0 from July 2018, which would line up. https://forum.duet3d.com/topic/6077/inconsistent-delays-during-homing-and-other-movements

Well done Watson. It would seem to be the same issue and as you say, the timing would coincide with the introduction of firmware 2.0.

I wonder if we have eliminated all other possibilities and are now at the "“When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.” stage.

Sherlock.

samlogan87

@deckingman I have experienced these issues with my Duex2 board as well. It has only happened twice and both times have been intermittent and a power cycle has fixed it.

Sam

gnydick

@samlogan87 mine has just miraculously stopped happening. I'm attributing it to disconnecting the lamp with the squeeeling power supply.

dc42

Those of you who have experienced this issue, did you have anything connected to the endstop inputs on the DueX5, or to the I2C pins on the 10-pin header (other than perhaps the resistor I suggested), or to the GPIO pins on the 10-pin header?

deckingman

@dc42 I'm fairly sure that I am using end stops on the Duex5 but I'll confirm when I get back to the UK in a couple of days time.

gnydick

Nope, just motors and one fan

dc42

Thanks. Next time the problem occurs, please check whether when when you do a software reset of the printer (e.g. M999 or Emergency Stop), the DueX is recognised afterwards (send M115 and look at the Electronics string).

If the DueX is not recognised after a software reset (so you have to power the system down an up again for the DueX to be recognised), then I have a theory about what is causing the problem. If it is correct, then the following should help. Most users run the VIN ground wire from the PSU to the Duet, and then on to the DueX using a short thick wire. The fix would be to run the VIN ground wire from the PSU to the DueX first, and then on to the Duet using a short thick wire. The +Vin wire isn't critical, it can be routed in either order or using separate wires from the PSU to the Duet and DueX.

deckingman

@deckingman said in Printer pausing between commands:

@dc42 I'm fairly sure that I am using end stops on the Duex5 but I'll confirm when I get back to the UK in a couple of days time.

I can now confirm that I'm using 4 of the end stops. Also, 5 motors, 5 fans and 5 thermistors. Nothing connected to the GPIO other than the recommended resistors. Confused as to why a wiring change might be the solution when I had no problems from when I originally started using the board in December 2016 until July(ish) 2018. But when it misbehaves again, I'll do the tests and make the changes.

fulg

@dc42 said in Printer pausing between commands:

[...] Next time the problem occurs, please check whether when when you do a software reset of the printer (e.g. M999 or Emergency Stop), the DueX is recognised afterwards [...].

I can reproduce this problem easily, and every time soft-resetting the Duet via M999 was enough to get the printer back in operation. I did not issue M115 explicitly but my setup uses 4 Z motors on the Duex and it always worked fine after a soft reset (so the Duex was recognized).

My wiring is as per the documentation, VIN/GND to the Duet then short wires to the Duex. I can flip that around if needed.

dc42

@fulg said in Printer pausing between commands:

I can reproduce this problem easily, and every time soft-resetting the Duet via M999 was enough to get the printer back in operation. I did not issue M115 explicitly but my setup uses 4 Z motors on the Duex and it always worked fine after a soft reset (so the Duex was recognized).

If that's the case then there is hope that recovery from the i2C communication breakdown may be possible. But please check with M115 next time you have this issue and do a soft reset.

My wiring is as per the documentation, VIN/GND to the Duet then short wires to the Duex. I can flip that around if needed.

gnydick

@dc42 I found that a loose connection on my main board caused my problem. I don't remember if it was a sensor, a fan, or a thermistor, but I found it, reproduced, snugged it up, and it's been fine.