Extruder motor stops during print…

ofloveandhate

logging from startup gives this:

2018-02-28 21:27:52 Error: Can't open 0:/sys/oem.json to read, error code 4

```is that disconcerting?

dc42

No, that message is normal. DWC tries to open oem.json but that file is only present in a special configuration used by one of our OEMs.

ofloveandhate

another failure, most of the way through tobel60, with interpolation off.

the gcode console does not have a message about the finishing of printing.

here are the outputs of some reports:

M563 P0
Tool 0 - drives: 0; heaters (active/standby temps): 1 (0.0/0.0); xmap: X; ymap: Y; fans: 0; status: selected

M122
=== Diagnostics ===
Used output buffers: 4 of 32 (10 max)
=== Platform ===
RepRapFirmware for Duet WiFi version 1.20 running on Duet WiFi 1.0
Board ID: 08DDM-9FAM2-LW4SD-6JKD0-3SN6N-T2ZVY
Static ram used: 15448
Dynamic ram used: 99288
Recycled dynamic ram: 4048
Stack ram used: 1392 current, 8552 maximum
Never used ram: 3736
Last reset 21:25:40 ago, cause: power up
Last software reset at 2018-02-28 21:32, reason: User, spinning module GCodes, available RAM 11880 bytes (slot 2)
Software reset code 0x0003 HFSR 0x00000000, CFSR 0x00000000, ICSR 0x0441f000, BFAR 0xe000ed38, SP 0xffffffff
Error status: 0
Free file entries: 9
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 2.8ms
MCU temperature: min 29.7, current 43.3, max 60.7
Supply voltage: min 23.8, current 24.3, max 24.9, under voltage events: 0, over voltage events: 0
Driver 0: standstill, SG min/max 0/1023
Driver 1: standstill, SG min/max 0/1023
Driver 2: standstill, SG min/max 0/1023
Driver 3: standstill, SG min/max 0/1023
Driver 4: standstill, SG min/max not available
Date/time: 2018-03-02 09:19:22
Cache data hit count 4294967295
Slowest main loop (seconds): 0.077818; fastest: 0.000044
=== Move ===
MaxReps: 1440, StepErrors: 0, FreeDm: 240, MinFreeDm 120, MaxWait: 1957299979ms, Underruns: 7045, 3
Scheduled moves: 0, completed moves: 0
Bed compensation in use: none
Bed probe heights: 0.000 0.000 0.000 0.000 0.000
=== Heat ===
Bed heaters = 0 -1 -1 -1, chamberHeaters = -1 -1
Heater 0 is on, I-accum = 0.0
Heater 1 is on, I-accum = 0.6
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is idle in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
Network state is running
WiFi module is connected to access point 
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.20
WiFi MAC address 2c:3a:e8:0b:20:8d
WiFi Vcc 3.37, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 16656
WiFi IP address 10.0.1.2
WiFi signal strength -50dBm, reconnections 0, sleep mode modem
HTTP sessions: 1 of 8
Socket states: 2 0 0 0 0 0 0 0
Responder states: HTTP(1) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)

M906
Motor current (mA) - X:1000, Y:1000, Z:1000, E:400:400:400:400:400:400:400:400:400, idle factor 30%

M913
Motor current % of normal - X:100, Y:100, Z:100, E:100:100:100:100:100:100:100:100:100

M350
Microstepping - X:64, Y:64, Z:64, E:16:16(on):16:16:16:16:16:16:16

M408 S4
{"status":"I","coords":{"axesHomed":[0,0,0],"extr":[50225.4],"xyz":[0.000,0.000,430.673]},"currentTool":0,"params":{"atxPower":0,"fanPercent":[0.00,100.00,100.00,0.00,0.00,0.00,0.00,0.00,0.00],"speedFactor":100.00,"extrFactors":[100.00],"babystep":0.000},"sensors":{"probeValue":0,"fanRPM":0},"temps":{"bed":{"current":16.3,"active":0.0,"state":2,"heater":0},"current":[16.3,190.0,2000.0,2000.0,2000.0,2000.0,2000.0,2000.0],"state":[2,2,0,0,0,0,0,0],"heads":{"current":[190.0],"active":[190.0],"standby":[0.0],"state":[2]},"tools":{"active":[[190.0]],"standby":[[0.0]]},"extra":[{"name":"MCU","temp":44.0}]},"time":79008.0,"currentLayer":0,"currentLayerTime":0.0,"extrRaw":[50225.4],"fractionPrinted":0.0,"firstLayerDuration":0.0,"firstLayerHeight":0.50,"printDuration":0.0,"warmUpDuration":0.0,"timesLeft":{"file":0.0,"filament":0.0,"layer":0.0}}

i am able to turn on the heater for the extruder. attempting to extrude yields very slow moves. i can feel the motor running, but it is barely turning. i swapped motors, and it did not solve the problem. i also removed the motor from the machine, and it still is barely moving when i extrude, so it was not physical binding causing the problem.

i will leave the machine errored for a while, hoping that we can try some things together.

dc42

Thanks. Please run M92 to check that the extruder steps/mm is still correct. Also M572 to check the pressure advance setting, and M201 and M203 to check the speed and acceleration settings.

Kezat

In the error state are you able to get the extruder to move faster if you manually send a G1 commands that includes a faster speed or will it only move at one really slow speed for any move command?

ofloveandhate

still in errored state:

M92
Steps/mm: X: 800.000, Y: 800.000, Z: 800.000, E: 700.000:700.000:700.000:700.000:700.000:700.000:700.000:700.000:700.000

M572
Extruder pressure advance: 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000

M201
Accelerations: X: 1000.0, Y: 1000.0, Z: 1000.0, E: 1000.0:1000.0:1000.0:1000.0:1000.0:1000.0:1000.0:1000.0:1000.0

M203
Maximum feedrates: X: 20000.0, Y: 20000.0, Z: 8000.0, E: 20000.0:20000.0:20000.0:20000.0:20000.0:20000.0:20000.0:20000.0:20000.0

ofloveandhate

no, putting in higher feedrates doesn't really make it move faster. this is hard to measure. putting in longer extrudes and higher feedrates doesn't seem to make a difference.

ofloveandhate

no errors in the text log, but it did log the completion of the file. thus, i cannot trust the web interface's gcode console – things show up in the text file that don't in the web interface, when i am not connected to the machine.

Kezat

@ofloveandhate:

no, putting in higher feedrates doesn't really make it move faster. this is hard to measure. putting in longer extrudes and higher feedrates doesn't seem to make a difference.

So strange, all other motors/fans/heater PIDs operate normally correct? Its only the extruder motor that stop/slows mid print yet the printer keeps on "printing" until the end of the gcode? Other then a tool change or full reset have you found any other way to clear the error state? Will another print without a reset start normally i wonder. Perhaps M92 E0 will clear it or even resending all the steps/mm, max speed, acceleration commands. (even though it tells you its correct) Also to test E movement in the error state are you heating the nozzle or enabling cold extrusion to let you test it?
Are you able to tell if the error is happening at truly the exact same spot in the gcode or just around the same layer/print time?

ofloveandhate

@Kezat – these are all good wonders. i still have the machine errored, so will try sending some gcodes to "restore" settings, see what happens.

1.

So strange, all other motors/fans/heater PIDs operate normally correct? Its only the extruder motor that stop/slows mid print yet the printer keeps on "printing" until the end of the gcode?

yep

2.

Other then a tool change or full reset have you found any other way to clear the error state? Will another print without a reset start normally i wonder

no, it's errored and nothing seems to fix it. the only way i know to get it working again is to M999 or power cycle. additional testing as you suggest it merited (re-sending machine parameters, etc). i will do that now.

3.

Also to test E movement in the error state are you heating the nozzle or enabling cold extrusion to let you test it?

A: heating it.

4.

Are you able to tell if the error is happening at truly the exact same spot in the gcode or just around the same layer/print time?

it is not the same spot every time. it varies wildly. i wish it was the same spot every time, we probably would've solved this by now, ha. repeatability / replicability is highly desirable. this is the major reason i doubt it is a firmware bug – otherwise, it would be repeatable. unless it's an intermittent problem caused by something like dynamic memory allocation, that stuff can be intermittent. so far, no-one's tried my gcode files on a duet delta machine, so i have no information on replicability.

garyd9

@ofloveandhate:

so far, no-one's tried my gcode files on a duet delta machine, so i have no information on replicability.

Not even the folks at UltiBots? That doesn't speak well for their support…

ofloveandhate

update: running

M350 X64 Y64 Z64 E16 I0

```allows me to extrude
i do not know how to return it to errored state without printing another object.

tell me, what do you think of this idea: i insert this statement into my layer change gcode, and see if i can make it through a whole file? there will be gaps where the machine went error.

ofloveandhate

@garyd9:

@ofloveandhate:

so far, no-one's tried my gcode files on a duet delta machine, so i have no information on replicability.

Not even the folks at UltiBots? That doesn't speak well for their support…

to their credit, my gcode files are long. some are 16-24 hour prints, with the error occurring anywhere in there, and not guaranteed to occur… brad has been very helpful to me so far, with a thread on the ultibots forum with > 70 posts, and several long phone calls. forum.ultibots.com/viewtopic.php?f=59&t=297

i would like someone to try my code, though… will you, dear reader? links above

Kezat

@ofloveandhate:

update: running

M350 X64 Y64 Z64 E16 I0

```allows me to extrude
i do not know how to return it to errored state without printing another object.

tell me, what do you think of this idea: i insert this statement into my layer change gcode, and see if i can make it through a whole file? there will be gaps where the machine went error.

Well that's something, did you try and have fail any of the other settings before you got the microstepping command to fix things? I wonder what bits and flags this command bumps that would cause it to start working.

Really sucks that it take so long to get back to the error state:( i would love to help test the Gcode but i only have 1 printer with the duet board and its the one I'm actively working on atm.

Not sure about the M350 resend each layer, what is that going to tell us we did not just find out, its a long time to tie up the printer and no error state to play around in at the end of it.

ofloveandhate

i think i need to know the union of the things that are set by this command (M350…), and the things that can set those things. then, i can search in the gcode file, or insert debug statements into the firmware, and re-test.

to github i go. i am so glad the firmware is written in c++, cuz i can read that.

anyone know how i can insert my own print-to-file statements into the firmware, and how i go about compiling it, so i can run my custom firmware?

ofloveandhate

Well that's something, did you try and have fail any of the other settings before you got the microstepping command to fix things? I wonder what bits and flags this command bumps that would cause it to start working.

i tried many reports, G92 E0 before extruding, and various speeds of extruding. the M350 command was the first i tried that should have modified any internally stored parameters.

Kezat

A way to test things faster could be to unplug XYZ motors from the bot, remove E motor, allow cold extrusion, tune accel and jerk up really high and increase print speed as high as you can, Or anyone with a spare board and motors. It would not prove anything if it works but if it still fails it might be a faster way to test things.

As a last resort you can swapping out the power supply and move the bot to another breaker/room that may help deduce if this is an interference issue.

When you get the error state again it might be worth attempting to sort out the true value that the motor is rotating at, maybe connect it to one of the bot axis so its easy to measure the movement.

After you test other things (like changing accel/max speed) in the error state it could be worth sending only one part of the M350 command at a time such as X Y and Z then I0 then last E16, to narrow down what really is fixing things. I wonder if the controller thinks its still at one microstepping/setting but the stepper drivers thinks it's in another.

ofloveandhate

unplug XYZ motors from the bot, remove E motor, allow cold extrusion, tune accel and jerk up really high and increase print speed as high as you can

good idea. doing that now.

i have requested an UPS to borrow to eliminate power line fluctuations… tech support is slow at my uni, tho, so that'll be a while.

the idea of swapping the supply out is interesting. i have a spare 24v supply from another printer, so i'll see what it will take to wire it in. i also have a big-ole variable power supply i can try, which would be simpler than re-wiring the alt-printer supply.

dc42

I'm glad you managed to track it down to the microstepping. Do you think it is possible that when the error occurs, it is running with x256 microstepping? That would cause extrusion to be 1/16 of the correct amount.

If the stepper motors reset themselves then the microstepping will default to x256. But that should only happen if the supply voltage drops below about 9.5V, and if that happened then an under-voltage event should show up in the M122 report.

ofloveandhate

how often does the system poll the voltage?

i experienced a power-related failure on my taz6 overnight (printer stopped mid-print), so now am thinking a UPS might solve this problem. argh.