CONSTANT AJAX disconnect errors

dc42 · 24 Oct 2017, 14:46

Yes, I am thinking that EMI from the stepper cables, or the PSU, or the Duet itself could be generating interference.

I haven't ruled out the possibility that the WiFi modules on your Duet is faulty. But as I believe you reported that the connection is much less reliable when you are printing, I don't think that is very likely - unless for some reason the 3.3V power to the WiFi module is dropping for some reason when you are printing.

Interference generated by the Duet or the PSU might also explain why some WiFi channels seem to be more reliable than others.

KeeganB · 24 Oct 2017, 18:40

David,

Aside from maybe the the stepper idle currents, is there ANYTHING in the config.g that could be related to my issues?

dc42 · 25 Oct 2017, 07:21

@KeeganB:

David,

Aside from maybe the the stepper idle currents, is there ANYTHING in the config.g that could be related to my issues?

As long as M122 shows a low MaxReps value after printing, nothing that I can think of.

KeeganB · 25 Oct 2017, 12:57

Well,

I had a really good run yesterday afternoon. I was printing, sitting at idle with and without the steppers powers, and editing configs and etc though DWC for hours and hours yesterday with not issues. I was even able to move all of my network hardware back to its rightful place in my office and move my printer back to its normal spot as well. I powered it all up and it all connected just fine, no usb needed!

and then…. as I'm going to start a print before bed DISCONNECT, and it would not reconnect. I didn't want to mess with pronterface, so I just let it print. As usual about an hour later or so I was able to access DWC again.

dc42 · 25 Oct 2017, 13:39

Did the disconnection happen soon after the stepper motors were powered up?

Could the behaviour be related to temperature? i.e. it works OK when the ambient temperature is low and the machine is not printing, but the connection fails when the wifi module gets a bit warmer due to higher ambient temperature or heating due to the stepper drivers passing current?

I don't suppose you have a radio taxi station, ATC beacon or other high-power transmission source in your vicinity?

KeeganB · 25 Oct 2017, 17:29

Hey David,

To start, I did your previously requested test in relation to the motor idle current and didn't get any disconnects. This was done during that really long stretch of good connection.

Yes, the disconnect last night happened probably 30 seconds or so after the print started. I do a G32 as part of the print, and after that completed, while working on the first layer it disconnected.
I have been keeping an eye on the MCU temp via DWC and it stays in the mid 30s Celsius.
I could add another small cooling fan to blow direction onto the wifi module if you think the is the problem. I can test that tonight. honestly though, if that is the problem I would have to question the quality of the module, or its ability to withstand heat. I don't think it is working under any extreme conditions at all.

No, no high power transmission sources nearby.

dc42 · 25 Oct 2017, 22:47

Keegan, please try firmware 1.20beta2 and the DuetWiFiServer that goes with it. It stops the WiFi module going into sleep mode and the M122 report includes additional network diagnostics.

I'm discussing with my colleagues the possibility of loaning you another Duet WiFi to test, in case the WiFi module on your Duet is faulty. Meanwhile, if you can temporarily add a fan to cool the WiFi module, that would test the temperature theory.

When you did that test with the motors energised but idle, what idle current factor were you using? Please re-test it at 100% idle current if you haven't already.

KeeganB · 25 Oct 2017, 23:49

Hey David,

I will test with a fan on the module tonight as well as upload the newest firmwares.

For the idle test, i tested with a print paused with both a factor of 60 and 100.

Thank You

dc42 · 25 Oct 2017, 23:57

Thanks. Can you confirm that you set the idle current to 100% before the motors went into idle hold? I'm not sure whether using M906 to adjust the idle current affects motors that have already been switched to idle.

Jarery · 26 Oct 2017, 00:02

Ive installed the new firmware.
confused by this statement :

Duet WiFi user can use either DuetWiFiServer 1.20beta2 (which disables WiFi module sleep) or 1.20alpha1 (which uses the default "modem sleep" mode)

My M122 has the following, but I'm using 1.20beta2, where above states modem sleep is only available with 1.20alpha1

WiFi signal strength -47dBm, reconnections 0, sleep mode modem

KeeganB · 26 Oct 2017, 02:04

@dc42:

Thanks. Can you confirm that you set the idle current to 100% before the motors went into idle hold? I'm not sure whether using M906 to adjust the idle current affects motors that have already been switched to idle.

I can't confirm that. I used M906 I100 to set it to 100. I could do the test again with the idles set to 100 in the config.g before starting a print.

I did a little temp testing by setting my bed heat to 100C, closing up all panels in the base of the machine (rostock max v2) and also setting the hot end to 200C with it about .5mm above the bed. I wanted to get the base of the machine as warm as possible. It sat like that for over an hour with no issue, and the MCU temp never exceeded 35C.

Also, I updated to the latest firmwares and afterwards I was homing the machine again, and again, back to back to back, and on the probably 4th or 5th time, it disconnected and would not reconnect via the browser. I plugged in the USB and connected to pronterface to run M122. here are the results:

[c]>>> M122
SENDING:M122
=== Diagnostics ===
Used output buffers: 1 of 32 (14 max)
=== Platform ===
RepRapFirmware for Duet WiFi version 1.20beta2 running on Duet WiFi 1.0
Board ID: 08DDM-9FAM2-LW4SD-6JTF0-3S86N-TLWZX
Static ram used: 15136
Dynamic ram used: 96560
Recycled dynamic ram: 2992
Stack ram used: 4000 current, 4808 maximum
Never used ram: 11576
Last reset 00:04:50 ago, cause: software
Last software reset reason: User, spinning module GCodes, available RAM 11496 bytes (slot 3)
Software reset code 0x0003, HFSR 0x00000000, CFSR 0x00000000, ICSR 0x00400000, BFAR 0xe000ed38, SP 0xffffffff
Error status: 0
[ERROR] Error status: 0

Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 41.9ms
MCU temperature: min 32.4, current 32.6, max 36.5
Supply voltage: min 12.0, current 12.1, max 12.2, under voltage events: 0, over voltage events: 0
Driver 0: stalled standstill
Driver 1: stalled standstill
Driver 2: stalled standstill
Driver 3: standstill
Driver 4: standstill
Date/time: 2017-10-25 22:00:43
Slowest main loop (seconds): 2.390603; fastest: 0.000032
=== Move ===
MaxReps: 3, StepErrors: 0, FreeDm: 240, MinFreeDm 234, MaxWait: 3783856217ms, Underruns: 0, 0
Scheduled moves: 30, completed moves: 30
Bed compensation in use: none
Bed probe heights: 0.000 0.000 0.000 0.000 0.000
=== Heat ===
Bed heater = 0, chamber heater = -1
Heater 0 is on, I-accum = 0.0
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is ready with "M122" in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
Network state is running
WiFi module is connected to access point
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.20beta2
WiFi MAC address 5c:cf:7f:ef:51:6f
WiFi Vcc 3.10, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 39160
WiFi IP address 10.0.1.158
WiFi signal strength -52dBm, reconnections 0, sleep mode modem
HTTP sessions: 1 of 8
Socket states: 0 0 0 0 0 0 0 0
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)[/c]

KeeganB · 26 Oct 2017, 02:11

got it reconnected, and it disconnected before i could home it even once.

ran through M552 S-1/S1 and it connected again, immediate disconnect again. It did this pattern 3 times. on the third time it stayed connected for about 45 seconds and then disconnected.

Here is the M122 from IMMEDIATELY after the last disconnect.

[c]>>> M122
SENDING:M122
=== Diagnostics ===
Used output buffers: 1 of 32 (14 max)
=== Platform ===
RepRapFirmware for Duet WiFi version 1.20beta2 running on Duet WiFi 1.0
Board ID: 08DDM-9FAM2-LW4SD-6JTF0-3S86N-TLWZX
Static ram used: 15136
Dynamic ram used: 96560
Recycled dynamic ram: 2992
Stack ram used: 4000 current, 5172 maximum
Never used ram: 11212
Last reset 00:13:49 ago, cause: software
Last software reset reason: User, spinning module GCodes, available RAM 11496 bytes (slot 3)
Software reset code 0x0003, HFSR 0x00000000, CFSR 0x00000000, ICSR 0x00400000, BFAR 0xe000ed38, SP 0xffffffff
Error status: 0
[ERROR] Error status: 0

Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 0.0ms
MCU temperature: min 30.9, current 31.2, max 31.5
Supply voltage: min 12.0, current 12.1, max 12.2, under voltage events: 0, over voltage events: 0
Driver 0: stalled standstill
Driver 1: stalled standstill
Driver 2: stalled standstill
Driver 3: standstill
Driver 4: standstill
Date/time: 2017-10-25 22:09:42
Slowest main loop (seconds): 0.006131; fastest: 0.000098
=== Move ===
MaxReps: 0, StepErrors: 0, FreeDm: 240, MinFreeDm 240, MaxWait: 0ms, Underruns: 0, 0
Scheduled moves: 107, completed moves: 107
Bed compensation in use: none
Bed probe heights: 0.000 0.000 0.000 0.000 0.000
=== Heat ===
Bed heater = 0, chamber heater = -1
Heater 0 is on, I-accum = 0.0
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is ready with "M122" in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
Network state is running
WiFi module is connected to access point
Failed messages: pending 0, notready 0, noresp 0
WiFi firmware version 1.20beta2
WiFi MAC address 5c:cf:7f:ef:51:6f
WiFi Vcc 3.10, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 38240
WiFi IP address 10.0.1.158
WiFi signal strength -50dBm, reconnections 0, sleep mode modem
HTTP sessions: 1 of 8
Socket states: 0 0 0 0 0 0 0 0
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)[/c]

EDIT: all of that occurred with the front panel off the romax, and a small fan blowing directly on the wifi module.

KeeganB · 26 Oct 2017, 03:02

A video….

https://youtu.be/0VvTF9d91_w

dc42 · 26 Oct 2017, 10:48

Keegan, thanks for your patience. We've decided to swap your board to see if that fixes the problem. Please email the following to info at duet3d dot com:

1. Board PCB revision (1.0, 1.01 or 1.02)
2. Board serial number (on the sticker on top of the microcontroller)
3. Your mailing addresss

KeeganB · 26 Oct 2017, 11:48

@dc42:

Keegan, thanks for your patience. We've decided to swap your board to see if that fixes the problem. Please email the following to info at duet3d dot com:

1. Board PCB revision (1.0, 1.01 or 1.02)
2. Board serial number (on the sticker on top of the microcontroller)
3. Your mailing addresss

Thank You David. Email sent.

KeeganB · 27 Oct 2017, 02:11

Hey David,

Just wanted to check to make sure you received me email?

Thanks!

dc42 · 27 Oct 2017, 07:32

Emails to that address don't come to me, but I do know that it was received.

Jarery · 27 Oct 2017, 17:01

Will be interested to see if it goes away with the new board he receives.

My wifi disconnection issues which seem identical to KeeganB’s are present on both boards I have.
I’ll see if I can downgrade my one system back to 18 firmware to see if issues go away this weekend.

KeeganB · 27 Oct 2017, 17:19

Thanks David,

I got the emails from Roland.

Jarery,

It'll be kind of rough if the new board doesn't resolve the issue, as I am running out of ideas. Yesterday via USB power, and no power from the printer's PSU, the duet stayed connected sitting idle for over 14hrs! Next I powered the printer up, and homed it to energize the steppers, it stayed for about 3 hours like that. Still being connected, i went ahead and started running G32's with everything still cold. Still connected about an hour later, i preheated everything and started a print. It stayed connected for about 30 mins after that and then disconnected. About 30 mins later I was able to reconnect via DWC without using USB to restart the wifi card. It stayed connected after that for over 6 hours to finish the print and was still connected when I left this morning.

That was a wild series of events, given the night before it was disconnecting like you see if the video I posted. I haven't figured out how to trigger the disconnects.

someuniqname · 27 Oct 2017, 17:38

i hate to join this thread but i started to see Ajax disconnects in the recent versions. today i'm getting just disconnects within minutes. switching printer on (heat bed+heat hotted) end disconnect. the board is alive via usb and can be pinged. i attach the m122 logs

M122
SENDING:M122
=== Diagnostics ===
Used output buffers: 1 of 32 (20 max)
=== Platform ===
RepRapFirmware for Duet WiFi version 1.19 running on Duet WiFi 1.0
Board ID: 08DAM-999TL-MQ4S4-6J1FG-3S06T-T5H3Z
Static ram used: 21176
Dynamic ram used: 96064
Recycled dynamic ram: 1544
Stack ram used: 4008 current, 4840 maximum
Never used ram: 7448
Last reset 00:03:38 ago, cause: power up
Last software reset reason: User, spinning module GCodes, available RAM 7448 bytes (slot 0)
Software reset code 0x0003, HFSR 0x00000000, CFSR 0x00000000, ICSR 0x00400000, BFAR 0xe000ed38, SP 0xffffffff
Error status: 0
[ERROR] Error status: 0

Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 0.0ms
MCU temperature: min 14.3, current 18.9, max 19.5
Supply voltage: min 16.2, current 18.7, max 19.1, under voltage events: 0, over voltage events: 0
Driver 0: stalled standstill
Driver 1: stalled standstill
Driver 2: stalled standstill
Driver 3: standstill
Driver 4: standstill
Date/time: 2017-10-27 19:28:11
Slowest main loop (seconds): 0.015995; fastest: 0.000034
=== Move ===
MaxReps: 3, StepErrors: 0, FreeDm: 240, MinFreeDm 234, MaxWait: 717ms, Underruns: 0, 0
Scheduled moves: 5, completed moves: 5
Bed compensation in use: none
Bed probe heights: 0.000 0.000 0.000 0.000 0.000
=== Heat ===
Bed heater = 0, chamber heater = -1
Heater 0 is on, I-accum = 0.1
Heater 1 is on, I-accum = 0.1
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is ready with "M122" in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
Network state is running
WiFi module is connected to access point
WiFi firmware version 1.19
WiFi MAC address 5c:cf:7f:2b:e7:ae
WiFi Vcc 3.12, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 40232
WiFi IP address 192.168.178.36
WiFi signal strength -50dBm
HTTP sessions: 1 of 8
Socket states: 0 0 0 0 0 0 0 0
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)

M122
SENDING:M122
=== Diagnostics ===
Used output buffers: 1 of 32 (24 max)
=== Platform ===
RepRapFirmware for Duet WiFi version 1.19 running on Duet WiFi 1.0
Board ID: 08DAM-999TL-MQ4S4-6J1FG-3S06T-T5H3Z
Static ram used: 21176
Dynamic ram used: 96128
Recycled dynamic ram: 1480
Stack ram used: 4008 current, 6156 maximum
Never used ram: 6132
Last reset 00:02:27 ago, cause: software
Last software reset reason: User, spinning module GCodes, available RAM 7208 bytes (slot 1)
Software reset code 0x0003, HFSR 0x00000000, CFSR 0x00000000, ICSR 0x00400000, BFAR 0xe000ed38, SP 0xffffffff
Error status: 0
[ERROR] Error status: 0

Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 456.1ms
MCU temperature: min 18.5, current 19.4, max 21.2
Supply voltage: min 18.0, current 18.5, max 19.2, under voltage events: 0, over voltage events: 0
Driver 0: stalled standstill
Driver 1: stalled standstill
Driver 2: stalled standstill
Driver 3: standstill
Driver 4: standstill
Date/time: 2017-10-27 19:33:47
Slowest main loop (seconds): 0.457424; fastest: 0.000035
=== Move ===
MaxReps: 0, StepErrors: 0, FreeDm: 240, MinFreeDm 240, MaxWait: 0ms, Underruns: 0, 0
Scheduled moves: 0, completed moves: 0
Bed compensation in use: none
Bed probe heights: 0.000 0.000 0.000 0.000 0.000
=== Heat ===
Bed heater = 0, chamber heater = -1
Heater 0 is on, I-accum = 0.2
Heater 1 is on, I-accum = 0.1
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is ready with "M122" in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
Network state is running
WiFi module is connected to access point
WiFi firmware version 1.19
WiFi MAC address 5c:cf:7f:2b:e7:ae
WiFi Vcc 3.12, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 39352
WiFi IP address 192.168.178.36
WiFi signal strength -51dBm
HTTP sessions: 1 of 8
Socket states: 0 0 0 0 0 0 0 0
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)