DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1

A Former User

@Garfield said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

Terminated immediately ...

t~~hats interesting; it means the cpu is able to relatively cleanly terminate the session as opposed to just freezing; although it doesn't help you.~~ see below for correction.

Garfield

I wonder if something didn't uninstall or get overwritten in the 'downgrade' process. I never used RC9, I came straight from RC6

A Former User

@ChrisP said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

So using screen isn't going to help.

mostly a precaution to avoid terminating the process if the session is interrupted for other reasons.

Garfield

@bearer I should say that there is no disconnect - just zero repsonse - no messages, it just stops ... I use a commercial tool (Secure CRT 8.7) and it still thinks it is connected but hitting enter just causes an on screen line feed.

A Former User

@Garfield said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

@bearer I should say that there is no disconnect - just zero repsonse - no messages, it just stops ... I use a commercial tool (Secure CRT 8.7) and it still thinks it is connected but hitting enter just causes an on screen line feed.

ah, that is more what i was expecting. it would terminate after 30-60 seconds or so as a timeout; in turn meaning the pi forze or was too busy to close the connection. still good info one way or the other.

Garfield

@bearer said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

it would terminate after 30-60 seconds or so as a timeout

Yup it does ...

A Former User

(i wonder if setting process affinity could isolate the hanging to leave a core running ssh etc, if possible in raspbian - anyways thats it for me today)

Garfield

I found this in the duet web server log

Apr 26 19:19:11 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
Apr 26 19:19:11 duet3 DuetWebServer[1106]:       Route matched with {action = "Get", controller = "WebSocket"}. Executing controller action with signature System.Threading.Tasks.Task Get() on controller DuetWebServer.Controllers.WebSocketController (DuetWe
Apr 26 19:19:11 duet3 DuetWebServer[1106]: fail: DuetWebServer.Controllers.WebSocketController[0]
Apr 26 19:19:11 duet3 DuetWebServer[1106]:       [WebSocketController] DCS is not started
Apr 26 19:19:11 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
Apr 26 19:19:11 duet3 DuetWebServer[1106]:       Executed action DuetWebServer.Controllers.WebSocketController.Get (DuetWebServer) in 339302.5643ms
Apr 26 19:19:11 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Apr 26 19:19:11 duet3 DuetWebServer[1106]:       Executed endpoint 'DuetWebServer.Controllers.WebSocketController.Get (DuetWebServer)'
Apr 26 19:19:11 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Apr 26 19:19:11 duet3 DuetWebServer[1106]:       Request finished in 339446.6255ms 101
Apr 26 19:19:11 duet3 DuetWebServer[1106]: warn: DuetWebServer.Services.ModelObserver[0]
Apr 26 19:19:11 duet3 DuetWebServer[1106]:       Failed to synchronize machine model
Apr 26 19:19:11 duet3 DuetWebServer[1106]: System.Net.Sockets.SocketException (107): Transport endpoint is not connected
Apr 26 19:19:11 duet3 DuetWebServer[1106]:    at DuetAPI.Utility.JsonHelper.ReceiveUtf8Json(Socket socket, CancellationToken cancellationToken) in /home/christian/duet/DuetSoftwareFramework/src/DuetAPI/Utility/JsonHelper.cs:line 154
Apr 26 19:19:11 duet3 DuetWebServer[1106]:    at DuetAPIClient.BaseConnection.ReceiveJson(CancellationToken cancellationToken) in /home/christian/duet/DuetSoftwareFramework/src/DuetAPIClient/BaseConnection.cs:line 294
Apr 26 19:19:11 duet3 DuetWebServer[1106]:    at DuetAPIClient.SubscribeConnection.GetMachineModelPatch(CancellationToken cancellationToken) in /home/christian/duet/DuetSoftwareFramework/src/DuetAPIClient/SubscribeConnection.cs:line 100
Apr 26 19:19:11 duet3 DuetWebServer[1106]:    at DuetWebServer.Services.ModelObserver.Execute() in /home/christian/duet/DuetSoftwareFramework/src/DuetWebServer/Services/ModelObserver.cs:line 156
Apr 26 19:19:11 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]

Apr 26 19:59:29 duet3 DuetWebServer[1106]: warn: DuetWebServer.Services.ModelObserver[0]
Apr 26 19:59:29 duet3 DuetWebServer[1106]:       Failed to synchronize machine model
Apr 26 19:59:29 duet3 DuetWebServer[1106]: System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (99): Cannot assign requested address /var/run/dsf/dcs.sock
Apr 26 19:59:29 duet3 DuetWebServer[1106]:    at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
Apr 26 19:59:29 duet3 DuetWebServer[1106]:    at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
Apr 26 19:59:29 duet3 DuetWebServer[1106]:    at DuetAPIClient.BaseConnection.Connect(ClientInitMessage initMessage, String socketPath, CancellationToken cancellationToken) in /home/christian/duet/DuetSoftwareFramework/src/DuetAPIClient/BaseConnection.cs:l
Apr 26 19:59:29 duet3 DuetWebServer[1106]:    at DuetWebServer.Services.ModelObserver.Execute() in /home/christian/duet/DuetSoftwareFramework/src/DuetWebServer/Services/ModelObserver.cs:line 131
Apr 26 19:59:30 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Apr 26 19:59:30 duet3 DuetWebServer[1106]:       Request starting HTTP/1.1 GET http://10.100.2.225/machine
Apr 26 19:59:30 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Apr 26 19:59:30 duet3 DuetWebServer[1106]:       Executing endpoint 'DuetWebServer.Controllers.WebSocketController.Get (DuetWebServer)'
Apr 26 19:59:30 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
Apr 26 19:59:30 duet3 DuetWebServer[1106]:       Route matched with {action = "Get", controller = "WebSocket"}. Executing controller action with signature System.Threading.Tasks.Task Get() on controller DuetWebServer.Controllers.WebSocketController (DuetWe
Apr 26 19:59:30 duet3 DuetWebServer[1106]: fail: DuetWebServer.Controllers.WebSocketController[0]
Apr 26 19:59:30 duet3 DuetWebServer[1106]:       [WebSocketController] DCS is not started
Apr 26 19:59:30 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
Apr 26 19:59:30 duet3 DuetWebServer[1106]:       Executed action DuetWebServer.Controllers.WebSocketController.Get (DuetWebServer) in 6.6056ms
Apr 26 19:59:30 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Apr 26 19:59:30 duet3 DuetWebServer[1106]:       Executed endpoint 'DuetWebServer.Controllers.WebSocketController.Get (DuetWebServer)'
Apr 26 19:59:30 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Apr 26 19:59:30 duet3 DuetWebServer[1106]:       Request finished in 7.2117ms 101
Apr 26 19:59:32 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Apr 26 19:59:32 duet3 DuetWebServer[1106]:       Request starting HTTP/1.1 GET http://10.100.2.225/machine
Apr 26 19:59:32 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Apr 26 19:59:32 duet3 DuetWebServer[1106]:       Executing endpoint 'DuetWebServer.Controllers.WebSocketController.Get (DuetWebServer)'
Apr 26 19:59:32 duet3 DuetWebServer[1106]: info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
Apr 26 19:59:32 duet3 DuetWebServer[1106]:       Route matched with {action = "Get", controller = "WebSocket"}. Executing controller action with signature System.Threading.Tasks.Task Get() on controller DuetWebServer.Controllers.WebSocketController (DuetWe
Apr 26 19:59:32 duet3 DuetWebServer[1106]: fail: DuetWebServer.Controllers.WebSocketController[0]
Apr 26 19:59:32 duet3 DuetWebServer[1106]:       [WebSocketController] DCS is not started

Same error even in RC9

Danal

@Garfield said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

I feel the need for a compatibility matrix for the 3 main components - which versions of RRF work wich versions of DWC.

Interesting. Duet3 + Pi 4B, 4 gig. I've been having random hangs that take a power cycle to clear. I am also on RC10, as of mid evening yesterday. I was not certain this was happening, nor certain that it started at RC whatever, so I have not reported anything, yet.

Now that I think about it, it came on hard when I switched to RC10. I had to power cycle at least eight or ten times last night.

I typically have a DWC, a VNC and a SSH running. They all just hang. Attempting to start a new SSH also hangs (note, not refused, connects and never gets a password prompt).

I will see what data I can gather.

gtj0

Can I ask all you guys with issues, if the DWC is NOT connected, does it still lock up?

Garfield

I've never run without it, only ever connected via WiFi, would take me a while to set up if the way to test is to put the SD card into the duet itself.

Danal

I believe he's saying, "start a job, and then close DWC".

Yes, I tried that. SSH only, no VNC, no DWC. Still locked within a few minutes.

gtj0

@Danal said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

I believe he's saying, "start a job, and then close DWC".

Yes, I tried that. SSH only, no VNC, no DWC. Still locked within a few minutes.

Yeah that was it. I wanted to make sure it wasn't DWC related. I've been running RC10 + 2.1.1 and printing fine but I don't use the DWC.

Something to try... The systemd service file for the DCS was changed to set

CPUSchedulingPolicy=fifo
CPUSchedulingPriority=20

which may be contributing to the problem.

Edit /lib/systemd/system/duetcontrolserver.service and remove those 2 lines, then reboot and see if that helps.

Danal

@gtj0 said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

Edit /lib/systemd/system/duetcontrolserver.service and remove those 2 lines, then reboot and see if that helps.

Will do. I've made a bunch of other changes, so let me re-verify the hangs are real, then I will try that. THANKS!

Wally

I'm a little late to the reporting; to help confirm, I too have seen full system lock ups and print connection losses too with RRF 3.01-R10, DWC 2.1.5 and DSF 2.1.1.

I was able to overcome and get working with:

sudo system duetcontrolserver restart

No system hardware changes, ribbons, or otherwise, just the new Beta install.

The duetcontrolserver would go to 400% CPU usage, and painstakingly getting to the terminal (I have screen direct on my Pi) was able to get a terminal open and issue the fix. SSH & web access were dead - couldn't remote in.

After the duetcontrolserver restart, system will work but still get random disconnects. I will try the modification above and report back too.

Danal

Well... just finished a two hour print, no changes, completed OK.

I will keep data points coming, in either direction.

chas2706

Another fantastic build.

It is like playing the lottery. Sometimes works fine, other times loads of disconnections from DWC and/or error messages like "homing failed" or heaters take forever to turn on!

Back to RRF3.01_RC3!!!!

gtj0

@chas2706 Are you experiencing this specific issue?

chas2706

@gtj0 said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

Are you experiencing this specific issue?

Which issue are you refering to because there are many.
Has anyone tried to upload a new slicer gcode with this version because when I do it the DWC only gives me access to the folders on the SBC!!

A Former User

@chas2706 said in DCS Crash with 3.01-R10 / DWC 2.1.5 / DSF 2.1.1:

Which issue are you refering to because there are many.

i suspect he was, in a slightly more diplomatic tone, trying to say unless you have something constructive to add to the issue in this particular forum topic you might be well advised to go elsewhere looking for some cheese to go with that whine.

yes there are a bunch of issues, and you're not the only one thats frustrated, but how you deal with it is only up to you.

i think you should take a page or two out of your own book..