CONSTANT AJAX disconnect errors

KeeganB · 26 Oct 2017, 11:48

Keegan, thanks for your patience. We've decided to swap your board to see if that fixes the problem. Please email the following to info at duet3d dot com:

1. Board PCB revision (1.0, 1.01 or 1.02)
2. Board serial number (on the sticker on top of the microcontroller)
3. Your mailing addresss

Thank You David. Email sent.

KeeganB · 27 Oct 2017, 02:11

Hey David,

Just wanted to check to make sure you received me email?

Thanks!

dc42 · 27 Oct 2017, 07:32

Emails to that address don't come to me, but I do know that it was received.

Jarery · 27 Oct 2017, 17:01

Will be interested to see if it goes away with the new board he receives.

My wifi disconnection issues which seem identical to KeeganB’s are present on both boards I have.
I’ll see if I can downgrade my one system back to 18 firmware to see if issues go away this weekend.

KeeganB · 27 Oct 2017, 17:19

Thanks David,

I got the emails from Roland.

Jarery,

It'll be kind of rough if the new board doesn't resolve the issue, as I am running out of ideas. Yesterday via USB power, and no power from the printer's PSU, the duet stayed connected sitting idle for over 14hrs! Next I powered the printer up, and homed it to energize the steppers, it stayed for about 3 hours like that. Still being connected, i went ahead and started running G32's with everything still cold. Still connected about an hour later, i preheated everything and started a print. It stayed connected for about 30 mins after that and then disconnected. About 30 mins later I was able to reconnect via DWC without using USB to restart the wifi card. It stayed connected after that for over 6 hours to finish the print and was still connected when I left this morning.

That was a wild series of events, given the night before it was disconnecting like you see if the video I posted. I haven't figured out how to trigger the disconnects.

someuniqname · 27 Oct 2017, 17:38

i hate to join this thread but i started to see Ajax disconnects in the recent versions. today i'm getting just disconnects within minutes. switching printer on (heat bed+heat hotted) end disconnect. the board is alive via usb and can be pinged. i attach the m122 logs

M122
SENDING:M122
=== Diagnostics ===
Used output buffers: 1 of 32 (20 max)
=== Platform ===
RepRapFirmware for Duet WiFi version 1.19 running on Duet WiFi 1.0
Board ID: 08DAM-999TL-MQ4S4-6J1FG-3S06T-T5H3Z
Static ram used: 21176
Dynamic ram used: 96064
Recycled dynamic ram: 1544
Stack ram used: 4008 current, 4840 maximum
Never used ram: 7448
Last reset 00:03:38 ago, cause: power up
Last software reset reason: User, spinning module GCodes, available RAM 7448 bytes (slot 0)
Software reset code 0x0003, HFSR 0x00000000, CFSR 0x00000000, ICSR 0x00400000, BFAR 0xe000ed38, SP 0xffffffff
Error status: 0
[ERROR] Error status: 0

Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 0.0ms
MCU temperature: min 14.3, current 18.9, max 19.5
Supply voltage: min 16.2, current 18.7, max 19.1, under voltage events: 0, over voltage events: 0
Driver 0: stalled standstill
Driver 1: stalled standstill
Driver 2: stalled standstill
Driver 3: standstill
Driver 4: standstill
Date/time: 2017-10-27 19:28:11
Slowest main loop (seconds): 0.015995; fastest: 0.000034
=== Move ===
MaxReps: 3, StepErrors: 0, FreeDm: 240, MinFreeDm 234, MaxWait: 717ms, Underruns: 0, 0
Scheduled moves: 5, completed moves: 5
Bed compensation in use: none
Bed probe heights: 0.000 0.000 0.000 0.000 0.000
=== Heat ===
Bed heater = 0, chamber heater = -1
Heater 0 is on, I-accum = 0.1
Heater 1 is on, I-accum = 0.1
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is ready with "M122" in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
Network state is running
WiFi module is connected to access point
WiFi firmware version 1.19
WiFi MAC address 5c:cf:7f:2b:e7:ae
WiFi Vcc 3.12, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 40232
WiFi IP address 192.168.178.36
WiFi signal strength -50dBm
HTTP sessions: 1 of 8
Socket states: 0 0 0 0 0 0 0 0
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)

M122
SENDING:M122
=== Diagnostics ===
Used output buffers: 1 of 32 (24 max)
=== Platform ===
RepRapFirmware for Duet WiFi version 1.19 running on Duet WiFi 1.0
Board ID: 08DAM-999TL-MQ4S4-6J1FG-3S06T-T5H3Z
Static ram used: 21176
Dynamic ram used: 96128
Recycled dynamic ram: 1480
Stack ram used: 4008 current, 6156 maximum
Never used ram: 6132
Last reset 00:02:27 ago, cause: software
Last software reset reason: User, spinning module GCodes, available RAM 7208 bytes (slot 1)
Software reset code 0x0003, HFSR 0x00000000, CFSR 0x00000000, ICSR 0x00400000, BFAR 0xe000ed38, SP 0xffffffff
Error status: 0
[ERROR] Error status: 0

Free file entries: 10
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest block write time: 456.1ms
MCU temperature: min 18.5, current 19.4, max 21.2
Supply voltage: min 18.0, current 18.5, max 19.2, under voltage events: 0, over voltage events: 0
Driver 0: stalled standstill
Driver 1: stalled standstill
Driver 2: stalled standstill
Driver 3: standstill
Driver 4: standstill
Date/time: 2017-10-27 19:33:47
Slowest main loop (seconds): 0.457424; fastest: 0.000035
=== Move ===
MaxReps: 0, StepErrors: 0, FreeDm: 240, MinFreeDm 240, MaxWait: 0ms, Underruns: 0, 0
Scheduled moves: 0, completed moves: 0
Bed compensation in use: none
Bed probe heights: 0.000 0.000 0.000 0.000 0.000
=== Heat ===
Bed heater = 0, chamber heater = -1
Heater 0 is on, I-accum = 0.2
Heater 1 is on, I-accum = 0.1
=== GCodes ===
Segments left: 0
Stack records: 2 allocated, 0 in use
Movement lock held by null
http is idle in state(s) 0
telnet is idle in state(s) 0
file is idle in state(s) 0
serial is ready with "M122" in state(s) 0
aux is idle in state(s) 0
daemon is idle in state(s) 0
queue is idle in state(s) 0
autopause is idle in state(s) 0
Code queue is empty.
Network state is running
WiFi module is connected to access point
WiFi firmware version 1.19
WiFi MAC address 5c:cf:7f:2b:e7:ae
WiFi Vcc 3.12, reset reason Turned on by main processor
WiFi flash size 4194304, free heap 39352
WiFi IP address 192.168.178.36
WiFi signal strength -51dBm
HTTP sessions: 1 of 8
Socket states: 0 0 0 0 0 0 0 0
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0)

someuniqname · 27 Oct 2017, 17:54

the server is still in the network and http seems to be there but a request instantly dies

Uwes-MBP-3:Edge uwe$ nmap 192.168.178.36

Starting Nmap 7.40 ( https://nmap.org ) at 2017-10-27 19:52 CEST
Nmap scan report for duet.fritz.box (192.168.178.36)
Host is up (0.056s latency).
Not shown: 999 closed ports
PORT STATE SERVICE
80/tcp open http

Nmap done: 1 IP address (1 host up) scanned in 1.31 seconds
Uwes-MBP-3:Edge uwe$ nmap 192.168.178.36

Starting Nmap 7.40 ( https://nmap.org ) at 2017-10-27 19:52 CEST
Nmap scan report for duet.fritz.box (192.168.178.36)
Host is up (0.047s latency).
Not shown: 999 closed ports
PORT STATE SERVICE
80/tcp open http

Nmap done: 1 IP address (1 host up) scanned in 2.66 seconds
Uwes-MBP-3:Edge uwe$ wget 192.168.178.36
–2017-10-27 19:53:04-- http://192.168.178.36/
Connecting to 192.168.178.36:80… connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

KeeganB · 30 Oct 2017, 23:44

new board is in, updating firmwares and connecting to wifi now. fingers crossed

[7:39]
updated and connected. now we wait

[7:42]
Well, about an hour of work to swap the boards, and it only took 2mins and 2seconds for it to disconnect.

[7:43]
I am going to go eat some pizza and walk away from this for a minute.

number40fan · 31 Oct 2017, 01:10

Damn!!!

KeeganB · 31 Oct 2017, 01:23

@number40fan:

Damn!!!

I used a different word, but yup…

KeeganB · 31 Oct 2017, 13:45

Jarery,

Any luck going back to 1.18?

Jarery · 31 Oct 2017, 15:42

Keegan, sorry i've been busy changing careers in the last week and have not had a chance yet.
Hopefully later this week I can downgrade my other printer.

dc42 · 31 Oct 2017, 17:05

As the replacement board didn't fix the problem, that suggests that it is something about the environment of your Duet that is triggering the problem.

Can you post a few photos of your printer, to show how you have the Duet mounted and any cables that are in the vicinity of the WiFi module?

KeeganB · 31 Oct 2017, 17:27

David,

I will try to get some pictures tonight. Ill warn in advance, that the wiring is pretty sloppy right now. I didn't give it much care when swapping the board because I wanted to get it in there as quickly as possible to test it. My next idea was to maybe just disconnect everything I could from the board except for power and try testing it that way.

Comparing to a Ultibots DV300s that comes with the duet, the main difference I could see is that my PSU (meanwell) is in the base of the Rostock with the Duet, where it is external for the DV300s. Could the PSU cause issue?

On the idea of the PSU too, is there any benefit in turning up the voltage potentiometer?

Mentaluproar · 31 Oct 2017, 17:59

I had this problem too. It only went away once I used Firefox developers edition to access the printer. I know there's a lot of hate for Safari, but I like it. It just can't access the duet properly.

KeeganB · 31 Oct 2017, 18:11

@Mentaluproar:

I had this problem too. It only went away once I used Firefox developers edition to access the printer. I know there's a lot of hate for Safari, but I like it. It just can't access the duet properly.

Thanks, I will test this tonight too. So far I've tried Safari and Chrome.

dc42 · 31 Oct 2017, 18:11

@KeeganB:

Comparing to a Ultibots DV300s that comes with the duet, the main difference I could see is that my PSU (meanwell) is in the base of the Rostock with the Duet, where it is external for the DV300s. Could the PSU cause issue?

I have my PSU under the bed with the Duet too. The bed is AC mains powered, so it's quite a small PSU. The edge of the Duet with the external connectors is between two of the 2020 base extrusions, so that the WiFi antenna protrudes outside them. See https://www.thingiverse.com/thing:965396.

KeeganB · 31 Oct 2017, 18:52

David,
Thanks for that link. Those pictures are a nice reference. My PSU is no closer than that to my duet board. I noticed that you have flex cable around all of your stepper wiring. Does this help with EMI at all, or is it just to clean up the wiring?

MiR · 31 Oct 2017, 21:33

I today tested latest firmware 1.20beta4+WifiServer1.20beta2, unfortunately the test ended with an Ajax error when I started printing a real file.

But, to start at the beginning, I first simply let autocalibration run for quite a while (30++ minutes) and all was fine.
Then I edited a gcode file stored on the printer, saved it and started printing the file.
Shortly after that I saw the Ajax Disconnect Error message.

RSSI Values and Noise Floor are very good, Powersave is disabled (see flags, no PS in flags) and the WiFi Card is still associated and answering fine to sta_info but the TCP-Stack is dead, cannot ping the Duet

in network 2897 seconds
flags 0x1e03a: WME N_CAP AMPDU AMSDU
per antenna average rssi of rx data frames: -39 -43 -45 0
per antenna noise floor: -95 -96 -95 0

What I see in sta_info is that after the error happend I have rx decrypt errors and one tx error, before the error decrypt failures were 0:

tx failures: 1
rx decrypt succeeds: 30153
rx decrypt failures: 144

Not sure if this means anything, at least it looks suspicious.

I have now rebooted the board and have directly started the print job, right now it runs for 30 minutes, decrypt errors are still 0 and tx failures are also 0.

Michael

MiR · 31 Oct 2017, 21:43

@David: Could it help when I provide remote debugging for you? I could set up a pc with windows on it that is also connected to duet's usb