Control server needs to be restarted on emergency shut down

kuon

At this point, the web ui works without asking for a password and remote api works, and if I reload the web ui, I get this logs:

Nov 07 01:06:12 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[1] Request starting HTTP/1.1 GET http://3d.goyman.com/ - -
Nov 07 01:06:12 lili DuetWebServer[543]: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6] The file /index.html was not modified
Nov 07 01:06:12 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[2] Request finished HTTP/1.1 GET http://3d.goyman.com/ - - - 304 - text/html 41.1221ms
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[1] Request starting HTTP/1.1 GET http://3d.goyman.com/rr_connect?password=reprap&time=2022-11-07T01%3A06%3A13 - -
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[2] Request finished HTTP/1.1 GET http://3d.goyman.com/rr_connect?password=reprap&time=2022-11-07T01%3A06%3A13 - - - 404 0 - 10.3325ms
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[1] Request starting HTTP/1.1 GET http://3d.goyman.com/machine/connect?password=reprap - -
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Routing.EndpointMiddleware[0] Executing endpoint 'DuetWebServer.Controllers.MachineController.Connect (DuetWebServer)'
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3] Route matched with {action = "Connect", controller = "Machine"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Connect(System.String, DuetWebServer.Singletons.ISessionStorage) on controller DuetWebServer.Controllers.MachineController (DuetWebServer).
Nov 07 01:06:13 lili DuetWebServer[543]: DuetWebServer.Singletons.SessionStorage[0] Session b91df835fcf048668f2c833d5da928e4 added (readWrite)
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ContentResultExecutor[1] Executing ContentResult with HTTP Response ContentType of application/json
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2] Executed action DuetWebServer.Controllers.MachineController.Connect (DuetWebServer) in 64.2328ms
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Routing.EndpointMiddleware[1] Executed endpoint 'DuetWebServer.Controllers.MachineController.Connect (DuetWebServer)'
Nov 07 01:06:13 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[2] Request finished HTTP/1.1 GET http://3d.goyman.com/machine/connect?password=reprap - - - 200 49 application/json 74.5523ms
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[1] Request starting HTTP/1.1 GET http://3d.goyman.com/machine?sessionKey=b91df835fcf048668f2c833d5da928e4 - -
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Cors.Infrastructure.CorsService[5] CORS policy execution failed.
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Cors.Infrastructure.CorsService[6] Request origin http://3d.goyman.com does not have permission to access the resource.
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Routing.EndpointMiddleware[0] Executing endpoint 'DuetWebServer.Controllers.WebSocketController.Get (DuetWebServer)'
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3] Route matched with {action = "Get", controller = "WebSocket"}. Executing controller action with signature System.Threading.Tasks.Task Get(System.String, DuetWebServer.Singletons.ISessionStorage) on controller DuetWebServer.Controllers.WebSocketController (DuetWebServer).
Nov 07 01:06:18 lili DuetWebServer[543]: DuetWebServer.Singletons.SessionStorage[0] Session b91df835fcf048668f2c833d5da928e4 registered a WebSocket connection
Nov 07 01:06:18 lili DuetWebServer[543]: DuetWebServer.Controllers.WebSocketController[0] WebSocket connected from 2a02:aa08:e000:abab::100:1:47034
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[1] Request starting HTTP/1.1 GET http://3d.goyman.com/machine/directory/0:%2Fmacros%2F - -
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Routing.EndpointMiddleware[0] Executing endpoint 'DuetWebServer.Controllers.MachineController.GetFileList (DuetWebServer)'
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3] Route matched with {action = "GetFileList", controller = "Machine"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] GetFileList(System.String) on controller DuetWebServer.Controllers.MachineController (DuetWebServer).
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[1] Request starting HTTP/1.1 GET http://3d.goyman.com/machine/file/0:%2Fsys%2Fdwc-settings.json - -
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Routing.EndpointMiddleware[0] Executing endpoint 'DuetWebServer.Controllers.MachineController.DownloadFile (DuetWebServer)'
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ContentResultExecutor[1] Executing ContentResult with HTTP Response ContentType of application/json
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3] Route matched with {action = "DownloadFile", controller = "Machine"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] DownloadFile(System.String) on controller DuetWebServer.Controllers.MachineController (DuetWebServer).
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2] Executed action DuetWebServer.Controllers.MachineController.GetFileList (DuetWebServer) in 12.5238ms
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Routing.EndpointMiddleware[1] Executed endpoint 'DuetWebServer.Controllers.MachineController.GetFileList (DuetWebServer)'
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[2] Request finished HTTP/1.1 GET http://3d.goyman.com/machine/directory/0:%2Fmacros%2F - - - 200 787 application/json 13.1273ms
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.FileStreamResultExecutor[2] Executing FileStreamResult, sending file with download name '' ...
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2] Executed action DuetWebServer.Controllers.MachineController.DownloadFile (DuetWebServer) in 22.5037ms
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Routing.EndpointMiddleware[1] Executed endpoint 'DuetWebServer.Controllers.MachineController.DownloadFile (DuetWebServer)'
Nov 07 01:06:18 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[2] Request finished HTTP/1.1 GET http://3d.goyman.com/machine/file/0:%2Fsys%2Fdwc-settings.json - - - 200 1417 application/octet-stream 23.1026ms
Nov 07 01:06:19 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[1] Request starting HTTP/1.1 GET http://3d.goyman.com/js/ObjectModelBrowser.a6a80e4b.js - -
Nov 07 01:06:19 lili DuetWebServer[543]: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2] Sending file. Request path: '/js/ObjectModelBrowser.a6a80e4b.js'. Physical path: '/opt/dsf/sd/www/js/ObjectModelBrowser.a6a80e4b.js'
Nov 07 01:06:19 lili DuetWebServer[543]: Microsoft.AspNetCore.Hosting.Diagnostics[2] Request finished HTTP/1.1 GET http://3d.goyman.com/js/ObjectModelBrowser.a6a80e4b.js - - - 0 31351 application/javascript 10.6610ms

kuon

@dc42 Do you have any pointer on how I can debug the issue? Having to restart the controlserver everytime I turn the printer on or press the emergency restart is a bit annoying.

T3P3Tony

@kuon one for @chrishamm

kuon

@chrishamm Any idea? I can provide more debug info if needed.

chrishamm

@kuon I am sorry it took a while to get back to you but I just had a chance to attempt to reproduce it and I didn't manage to provoke this problem. I set the password right at the top of my config.g with v3.4.4 and when I press E-STOP, DWC reconnects with the previously entered password even when I have two independent DWC sessions running in two different tabs.

From your logs it appears you expose the web server to the internet, which is a scenario I have never set up (yet it should not make a difference). Can you confirm the same problem persists when you access your machine from your LAN?

kuon

@chrishamm The hostname is local LAN address. If I try with the IP directly I get the same result.

I noticed something, when the error is present, I get the following logs:

Nov 22 01:17:03 lili systemd[1]: Starting Duet Control Server...
Nov 22 01:17:06 lili DuetControlServer[301]: Duet Control Server v3.4.4
Nov 22 01:17:06 lili DuetControlServer[301]: Written by Christian Hammacher for Duet3D
Nov 22 01:17:06 lili DuetControlServer[301]: Licensed under the terms of the GNU Public License Version 3
Nov 22 01:17:07 lili DuetControlServer[301]: [info] Settings loaded
Nov 22 01:17:07 lili DuetControlServer[301]: [info] Environment initialized
Nov 22 01:17:07 lili DuetControlServer[301]: [info] Connection to Duet established
Nov 22 01:17:08 lili DuetControlServer[301]: [info] IPC socket created at /run/dsf/dcs.sock
Nov 22 01:17:08 lili systemd[1]: Started Duet Control Server.
Nov 22 01:17:08 lili DuetControlServer[301]: [info] Starting macro file config.g on channel Trigger
Nov 22 01:17:11 lili DuetControlServer[301]: [info] Finished macro file config.g

When I restart the server manually, I get this output:

Nov 22 14:32:00 lili systemd[1]: Starting Duet Control Server...
Nov 22 14:32:00 lili DuetControlServer[905]: Duet Control Server v3.4.4
Nov 22 14:32:00 lili DuetControlServer[905]: Written by Christian Hammacher for Duet3D
Nov 22 14:32:00 lili DuetControlServer[905]: Licensed under the terms of the GNU Public License Version 3
Nov 22 14:32:01 lili DuetControlServer[905]: [info] Settings loaded
Nov 22 14:32:01 lili DuetControlServer[905]: [info] Environment initialized
Nov 22 14:32:01 lili DuetControlServer[905]: [info] Connection to Duet established
Nov 22 14:32:01 lili DuetControlServer[905]: [info] IPC socket created at /run/dsf/dcs.sock
Nov 22 14:32:01 lili systemd[1]: Started Duet Control Server.

The two lines about config.g are missing, but in this case, it works and I can access the web UI.

If I empty my config.g and only leave the password and hostname the issue persists.

chrishamm

@kuon Right, config.g is not executed when you restart the DSF service and DSF will keep using the default (unset) password, so actually you should put your M551 into dsf-config.g which is always executed when you restart the Pi or the DCS service in general.

Please share your current config.g file, the only reason I can think of why it isn't working is an error somehwere in your configuration file. M551 works properly on my SBC setup no matter what I do. Note that M551 and HTTP password protection wasn't supported in earlier DSF versions, so that might have just concealed the problem.

kuon

@chrishamm As I said, I tried with a config.g containing only those two lines:

M551 P"xxxx"                                  ; printer password
M550 P"lili"                                       ; printer name

chrishamm

@kuon That works for me, I can repeatedly connect with "xxxx" as the password upon E-STOP using Firefox and MS Edge.

kuon

I realized something, if I restart the server with sudo systemctl restart duetcontrolserver.service then load the web UI (not prompted for password, so so far, so good), then I issue the M551 gcode using the UI, and after that hit F5 to reload the page, I am prompted for password, and if I enter the password it works, but if Ireload the page again, I still get prompted. It seems the session is not sticking.

Is that intended behavior? Because maybe my M551 was not being set and I was actually using the machine without a password. And also that prusa slicer is not supporting the proper way to use Duet API with a password.

chrishamm

@kuon Yes, it is, see my previous comment. config.g is only executed when RRF starts but not when DCS is restarted, so you should put M550/M551 in dsf-config.g instead. The machine password is stored by DCS and not RRF, so it is cleared upon restart if it is only in config.g

kuon

@chrishamm said in Control server needs to be restarted on emergency shut down:

@kuon Yes, it is, see my previous comment. config.g is only executed when RRF starts but not when DCS is restarted, so you should put M550/M551 in dsf-config.g instead. The machine password is stored by DCS and not RRF, so it is cleared upon restart if it is only in config.g

dsf-config.g must be activated in some way? Because I just tried and it has no effect. I added the M551 and M550 in it. (which I removed from config.g) The name in the UI defaults to the hostname of the PI instead of the M550 value I set.

Also, is it normal that the UI prompt for password on each reload and that it does not set a persistent session?

jay_s_uk

@kuon is this a fresh install or have you been upgrading the SBC side for a while?

kuon

@jay_s_uk I have been upgrading for a long time, maybe that's what is causing the issue. I can try a fresh install.

jay_s_uk

@kuon i would try a fresh install because the duetpimanagement plugin wasn't around until RRF3.3

chrishamm

@kuon no, dsf-config.g is automatically called by DCS and does not require the management plugin. Neither do M550 or M551. In order to apply it, ESTOP or a service restart is needed.

kuon

@chrishamm I did restart the service with systemctl and it does not apply dsf-config.g. To test it, I have no M550 in config.g and M550 P"Hello" in dsf-config.g. If I restart the service or emergency stop and then reload the page, the hostname is still used on the top left, and running M550 returns the hostname as well. (instead of Hello)

chrishamm

@kuon I have no good explanation why dsf-config.g isn't executed on your system, perhaps a permission thing? Make sure it is present in your sys directory (also available as /opt/dsf/sd/sys on Linux) and that it is readable by the dsf user/group.

When I restart DCS, I can see

Nov 22 18:41:55 Ender3Pro systemd[1]: Stopped Duet Control Server.
Nov 22 18:41:55 Ender3Pro systemd[1]: duetcontrolserver.service: Consumed 6.445s CPU time.
Nov 22 18:41:55 Ender3Pro systemd[1]: Starting Duet Control Server...
Nov 22 18:41:56 Ender3Pro DuetControlServer[1360]: Duet Control Server v3.4.4
Nov 22 18:41:56 Ender3Pro DuetControlServer[1360]: Written by Christian Hammacher for Duet3D
Nov 22 18:41:56 Ender3Pro DuetControlServer[1360]: Licensed under the terms of the GNU Public License Version 3
Nov 22 18:41:56 Ender3Pro DuetControlServer[1360]: [info] Settings loaded
Nov 22 18:41:56 Ender3Pro DuetControlServer[1360]: [warn] Support for third-party root plugins is enabled
Nov 22 18:41:56 Ender3Pro DuetControlServer[1360]: [info] Environment initialized
Nov 22 18:41:56 Ender3Pro DuetControlServer[1360]: [info] Connection to Duet established
Nov 22 18:41:56 Ender3Pro DuetControlServer[1360]: [info] IPC socket created at /run/dsf/dcs.sock
Nov 22 18:41:56 Ender3Pro systemd[1]: Started Duet Control Server.
Nov 22 18:41:59 Ender3Pro DuetControlServer[1360]: [info] Starting macro file dsf-config.g on channel SBC
Nov 22 18:42:00 Ender3Pro DuetControlServer[1360]: [info] Finished macro file dsf-config.g

which proves it is executed correctly.

Also note that in SBC mode M550 may be used without the DuetPiManagementPlugin only to decorate the hostname, e.g. to change the caption from "ender3pro" to "Ender 3 Pro" (i.e. to change casing and to add spaces). For system-related changes on Linux, the DuetPiManagementPlugin is mandatory.

Since I cannot reproduce any of your problems here, it may actually be a good idea to backup your config, obtain our latest DuetPi or DuetPi Lite image again, and then see if the problems persist once your SBC is up and running again.

kuon

@chrishamm I did some more testing, and I think I "might" have something.

I realized that prusa slicer is using GET /rr_connect but this call returns 404, I have a feeling this is not normal.

I tried with

curl  'http://3d.e42.ch/rr_connect?password=1234' -v

and sure enough I get a 404.

Prusa seems to ignore the error and proceed to the rest of the requests (/machine/status ...), which works if no password is set, but doesn't if it is set.

chrishamm

@kuon It is normal. DSF v3.5 will get an emulation layer for rr_ calls but DSF v3.4 only supports the REST HTTP API, see https://github.com/Duet3D/DuetSoftwareFramework/wiki/REST-API-(v3.4.4-or-newer)