Constant 'connection interrupted'

[Foden]

@bearer Only thing having issues is my Duet, everything else Wi-Fi related is fine. I have a wireless camera on the printer using the same SSID and it's OK. Definitely the Duet, likely the wifi module as Phaedrux suggested.

Nothing changed on the network environment and life was hunky dory until firmware and Duet web interface upgrade. If i get chance over the weekend I will look to go back to an earlier setup

[mrehorstdmd]

I couldn't roll back to the old firmware, but I put my machine in AP mode and found that it does not drop the connection like it does in client mode. It does do something weird every hour- I get messages about a possible virus attack on the console. Here's the console text including results of M122:

11/22/2020, 6:13:42 PM: Connected to 192.168.0.254
11/22/2020, 6:14:07 PM: M32 "0:/gcodes/11-22-20-001.gcode": File 0:/gcodes/11-22-20-001.gcode selected for printing
11/22/2020, 6:14:42 PM: Connection interrupted, attempting to reconnect...: Network error
11/22/2020, 6:15:18 PM: Connection established
11/22/2020, 6:21:26 PM: : Finished printing file 0:/gcodes/11-22-20-001.gcode, print time was 0h 7m
11/22/2020, 6:35:53 PM: : Warning: IP 192.168.0.154 requested file 'rootr1/ME4wTDBKMEgwR...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'rootr3/ME8wTTBLMEkwR...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'gsextendcodesignsha2...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'gstimestampingsha2g2...' from HTTP server, possibly a virus attack
11/22/2020, 7:36:13 PM: : Warning: IP 192.168.0.154 requested file 'rootr1/ME4wTDBKMEgwR...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'rootr3/ME8wTTBLMEkwR...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'gsextendcodesignsha2...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'gstimestampingsha2g2...' from HTTP server, possibly a virus attack
11/22/2020, 7:58:32 PM: M32 "0:/gcodes/11-19-20-002.gcode": File 0:/gcodes/11-19-20-002.gcode selected for printing
11/22/2020, 8:12:02 PM: M25: Resume state saved
11/22/2020, 8:12:07 PM: : Printing paused at X600.0 Y990.0 Z0.0
11/22/2020, 8:13:13 PM: M0 H1: Cancelled printing file 0:/gcodes/11-19-20-002.gcode, print time was 0h 13m
11/22/2020, 8:13:28 PM: m84
11/22/2020, 8:21:47 PM: M32 "0:/gcodes/test03.gcode": File 0:/gcodes/test03.gcode selected for printing
11/22/2020, 8:25:24 PM: M122: === Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.1.1 running on Duet WiFi 1.02 or later
Board ID: 08DGM-917DA-G4MSJ-6JTF2-3SD6J-KTNHA
Used output buffers: 3 of 24 (13 max)
=== RTOS ===
Static ram: 27980
Dynamic ram: 92564 of which 932 recycled
Exception stack ram used: 544
Never used ram: 9052
Tasks: NETWORK(ready,412) HEAT(blocked,1456) MAIN(running,1732) IDLE(ready,80)
Owned mutexes: WiFi(NETWORK)
=== Platform ===
Last reset 02:14:41 ago, cause: software
Last software reset time unknown, reason: User, spinning module GCodes, available RAM 9292 bytes (slot 3)
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x0041f000 BFAR 0xe000ed38 SP 0xffffffff Task MAIN
Error status: 0
MCU temperature: min 33.5, current 39.4, max 39.6
Supply voltage: min 24.1, current 24.3, max 24.6, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: standstill, SG min/max not available
Driver 1: standstill, SG min/max not available
Driver 2: standstill, SG min/max not available
Driver 3: standstill, SG min/max not available
Driver 4: standstill, SG min/max not available
Date/time: 2020-11-22 20:25:23
Cache data hit count 4294967295
Slowest loop: 266.72ms; fastest: 0.13ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 9
SD card 0 detected, interface speed: 20.0MBytes/sec
SD card longest read time 2.7ms, write time 104.3ms, max retries 0
=== Move ===
Hiccups: 2101(0), FreeDm: 167, MinFreeDm: 159, MaxWait: 5826231ms
Bed compensation in use: none, comp offset 0.000
=== MainDDARing ===
Scheduled moves: 233, completed moves: 224, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: 3
=== AuxDDARing ===
Scheduled moves: 0, completed moves: 0, StepErrors: 0, LaErrors: 0, Underruns: 0, 0  CDDA state: -1
=== Heat ===
Bed heaters = -1 -1 -1 -1, chamberHeaters = -1 -1 -1 -1
=== GCodes ===
Segments left: 1
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is doing "G01 X2.673 Y859.133" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 209.83ms; fastest: 0.00ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
- WiFi -
Network state is active
WiFi module is providing access point 
Failed messages: pending 0, notready 81, noresp 1
WiFi firmware version 1.23
WiFi MAC address ee:fa:bc:01:df:36
WiFi Vcc 3.35, reset reason Exception
WiFi flash size 4194304, free heap 24728
WiFi IP address 192.168.0.254
Connected clients 1
Socket states: 0 0 0 0 0 0 0 0
11/22/2020, 8:36:33 PM: : Warning: IP 192.168.0.154 requested file 'rootr1/ME4wTDBKMEgwR...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'rootr3/ME8wTTBLMEkwR...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'gsextendcodesignsha2...' from HTTP server, possibly a virus attack
Warning: IP 192.168.0.154 requested file 'gstimestampingsha2g2...' from HTTP server, possibly a virus attack
 
 

I was also running wireshark and grabbed all the traffic for about 5 minutes. I stopped capture shortly after the last "virus attack" messages were generated.

[Phaedrux]

@mrehorstdmd
Do you have any anti virus software on your PC?

https://forum.duet3d.com/topic/18054/virus-warning-message/2?_=1606098311359

[A Former User]

@mrehorstdmd what is the 192.168.0.154 host? it certainly seems like its got some sort of malware looking for exploits

[mrehorstdmd]

192.168.0.154 is the laptop that is connected to the Duet. Windows defender and Malwarebytes didn't find anything.

[Phaedrux]

@mrehorstdmd well in some cases it's the av software that's sending some packets that are making it puke. Not sure if that's the case here though. Try disabling malwarebytes.

[A Former User]

didn't see packets like that in the first wireshark capture where it did go wonky. (not sure if non-unicast traffic would be captured though)

netstat -o -m | find "192.68.0.254:80" should list he PID for the process behind the request and tasklist /FI "PID eq 1234" should find the executable name (replacing 1234 with the actual process id from netstat)

downside is you have to run the commands when the its being sent so tricky to do on once in a while stuff without getting more creative.

[mrehorstdmd]

@Phaedrux said in Constant 'connection interrupted':

@mrehorstdmd well in some cases it's the av software that's sending some packets that are making it puke. Not sure if that's the case here though. Try disabling malwarebytes.

Malwarebytes wasn't running when those messages were generated. Windows might have been doing it...

[Phaedrux]

@mrehorstdmd said in Constant 'connection interrupted':

I couldn't roll back to the old firmware

Probably missing the IAP files from RRF2. Try grabbing iap4e and iap4s from the 2.03 release and putting them in sys. Then try uploading the full 2.05.1 release zip.

I put my machine in AP mode and found that it does not drop the connection like it does in client mode.

Well at least that means the module is probably OK.

Malwarebytes wasn't running when those messages were generated. Windows might have been doing it...

Maybe. That would seem aggressive for Windows Defender.

[A Former User]

@Phaedrux said in Constant 'connection interrupted':

Probably missing the IAP files from RRF2. Try grabbing iap4e and iap4s from the 2.03 release and putting them in sys. Then try uploading the full 2.05.1 release zip.

if hes running 3.1.1 then he would need the IAP files from 3.1.1 to downgrade to 2.05.1 if thats the issue.

[Foden]

Replacement Wi-Fi module has arrived, will look to get it installed in the next couple of weeks (hopefully without frying the Duet)

Took 2 weeks to arrive from Germany, roughly $6CDN

https://www.ebay.ca/itm/254138264408

[dc42]

@Foden said in Constant 'connection interrupted':

Able to sign into in on my laptop, but still get drops

I noticed in the diagnostics that i do NOT see what I should, I see

WiFi Vcc 3.45, reset reason Unknown

rather than

WiFi Vcc 3.42, reset reason Turned on by main processor

"reason unknown" is normal for DuetWiFiServer 1.23. You should see "turned on by main processor" if running 1.24.

Did you try changing WiFi channel on your router?

[Foden]

Hi dc42

Yes I changed Wi-Fi channels and installed a Wi-Fi extender/booster next to the Duet. Sadly neither made any difference.

I was getting reset reason unknown on the previous version of DuetWiFiServer 1.21 (I think)

[taglia]

Hi all,

I see the same problem on a wired connection (Duet 2 Maestro, static IP, connected to my main switch). I have, of course, tried different cables, so this is not the issue.

I saw this problem on 2.05 (possibly on 2.04 though I don't remember); now I am on 3.2beta4, and it keeps happening. I could not find any triggering factor; sometimes, everything works fine for hours, while other times I can't keep a stable connection for more than 10 seconds. When the HTTP connection is not responsive, the same happens to FTP, but I can still ping the board with no losses and normal ping times.

My network is very stable, and the printer is on a subnet with very few devices.

I can run Wireshark as well if helpful, but I wanted to point out that the issue is not limited to WiFi.

I can live with this issue (I have for quite a while), but it is annoying, so if there's a solution I'd gladly take it!

M115
FIRMWARE_NAME: RepRapFirmware for Duet 2 Maestro FIRMWARE_VERSION: 3.2-beta4 ELECTRONICS: Duet Maestro 1.0 FIRMWARE_DATE: 2020-11-26

M122
=== Diagnostics ===
RepRapFirmware for Duet 2 Maestro version 3.2-beta4 running on Duet Maestro 1.0
Board ID: 08DJM-956DU-LL3T0-6JTD8-3S86Q-1U2AP
Used output buffers: 1 of 24 (24 max)
=== RTOS ===
Static ram: 22980
Dynamic ram: 91876 of which 60 recycled
Never used RAM 15132, free system stack 144 words
Tasks: NETWORK(ready,201) HEAT(blocked,329) TMC(blocked,113) MAIN(running,404) IDLE(ready,21)
Owned mutexes:
=== Platform ===
Last reset 01:18:57 ago, cause: software
Last software reset at 2020-12-01 11:28, reason: User, GCodes spinning, available RAM 15132, slot 2
Software reset code 0x0003 HFSR 0x00000000 CFSR 0x00000000 ICSR 0x00000000 BFAR 0xe000ed38 SP 0xffffffff Task MAIN
Error status: 0x04
MCU temperature: min 55.3, current 59.3, max 59.9
Supply voltage: min 23.3, current 24.0, max 24.1, under voltage events: 0, over voltage events: 0, power good: yes
Driver 0: position 8094, standstill, read errors 0, write errors 0, ifcnt 48, reads 17804, writes 0, timeouts 0, DMA errors 0
Driver 1: position 10545, ok, read errors 0, write errors 0, ifcnt 47, reads 17804, writes 0, timeouts 0, DMA errors 0
Driver 2: position 3385, ok, read errors 0, write errors 0, ifcnt 48, reads 17804, writes 0, timeouts 0, DMA errors 0
Driver 3: position 0, ok, read errors 0, write errors 0, ifcnt 40, reads 17804, writes 0, timeouts 0, DMA errors 0
Driver 4: position 0, ok, read errors 0, write errors 0, ifcnt 48, reads 17803, writes 0, timeouts 0, DMA errors 0
Driver 5: position 0, assumed not present
Driver 6: position 0, assumed not present
Date/time: 2020-12-01 12:47:55
Slowest loop: 14.37ms; fastest: 0.10ms
I2C nak errors 0, send timeouts 0, receive timeouts 0, finishTimeouts 0, resets 0
=== Storage ===
Free file entries: 9
SD card 0 detected, interface speed: 15.0MBytes/sec
SD card longest read time 2.0ms, write time 4.1ms, max retries 0
=== Move ===
Hiccups: 0(0), FreeDm: 166, MinFreeDm: 151, MaxWait: 0ms
Bed compensation in use: mesh, comp offset 0.000
=== MainDDARing ===
Scheduled moves 11733, completed moves 11728, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state 3
=== AuxDDARing ===
Scheduled moves 0, completed moves 0, StepErrors 0, LaErrors 0, Underruns [0, 0, 0], CDDA state -1
=== Heat ===
Bed heaters = 0 -1, chamberHeaters = -1 -1
Heater 0 is on, I-accum = 0.6
Heater 1 is on, I-accum = 0.5
=== GCodes ===
Segments left: 1
Movement lock held by null
HTTP is idle in state(s) 0
Telnet is idle in state(s) 0
File is doing "G1 X100.690 Y105.428 E1.0722" in state(s) 0
USB is idle in state(s) 0
Aux is idle in state(s) 0
Trigger is idle in state(s) 0
Queue is idle in state(s) 0
LCD is idle in state(s) 0
Daemon is idle in state(s) 0
Autopause is idle in state(s) 0
Code queue is empty.
=== Network ===
Slowest loop: 48.65ms; fastest: 0.03ms
Responder states: HTTP(0) HTTP(0) HTTP(0) HTTP(0) FTP(0) Telnet(0), 0 sessions
HTTP sessions: 1 of 8
Interface state active, link 100Mbps full duplex
=== Filament sensors ===
Extruder 0: pos 91.41, errs: frame 16 parity 0 ovrun 0 pol 14 ovdue 0
 

[A Former User]

@taglia said in Constant 'connection interrupted':

When the HTTP connection is not responsive, the same happens to FTP, but I can still ping the board with no losses and normal ping times.

it'd be interesting to see the wireshark log; if its a common issue it needs to be in RRF and not in the Wifi Firmware (separate fimwares) .. but it could also be different issue but same symptoms however unlikely it may be.

[taglia]

@bearer said in Constant 'connection interrupted':

it'd be interesting to see the wireshark log; if its a common issue it needs to be in RRF and not in the Wifi Firmware (separate fimwares) .. but it could also be different issue but same symptoms however unlikely it may be.

It looks very similar to the screenshots @Foden sent earlier. Here is the WS capture: https://www.dropbox.com/t/tIMSR7APIe63zAsL

113 is the Duet, 25 is my laptop

[Foden]

In my ignorance - I'm not sure the Wireshark logs show much useful information. Other than traffic relating to equipment disappearing/disconnecting from the network, which is why our logs look similar. The Duet webserver is basically saying 'Hello...HELLO...H E L L O O O ....etc"
I could be wrong though

[taglia]

I agree, and I would add that it might be something related to our network configuration, or more people would be raising it.

I am not sure if this is helpful in any way, but I have noticed that the connection is usually reliable for a while, right after switching the board on. Then, it degrades progressively, and after some time, it's hard to do anything.

Also, on the status page during a print, often the web interface keeps trying to reconnect. Still, I can see data updating behind the overlay: is the JS code too aggressive to detect disconnections? I have set the number of AJAX retries to 4 and the update interval to 500ms.

[jens55]

@taglia said in Constant 'connection interrupted':

I am not sure if this is helpful in any way, but I have noticed that the connection is usually reliable for a while, right after switching the board on. Then, it degrades progressively, and after some time, it's hard to do anything.

When I had disconnect issues, they were completely unrelated to board "on" time. My printer is on 24/7 and there was no connection between connection issues and length of Duet board being on.
Of course there is nothing saying that my issues (which I haven't seen in a while) are the same as your issues. Oh, I have a Duet wifi.

[Foden]

@taglia Nothing has changed on my network at all is the only issue I have. I have a Wyze v2 cam on the printer which uses the same SSID and it doesn't drop.

I agree that it normally starts off ok and then degrades - I was thinking that was maybe temperature related? The printer will still carry on printing regardless of whether I can connect to it via WIFI or not.

My I have my AJAX retries set to 4 also