Error trying to start a plugin

stuartofmt

I am trying to develop a plugin which, in part, runs a python script.
Any help would be appreciated.

The plugin loads but gives the following error when trying to start. Its almost as if its getting confused about its working directory ? More likely though, I have something wrong in the plugin.json file ....

I get this same error with:
"sbcExecutable": "/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py"
and
"sbcExecutable": "DuetLapse3.py"

The python script gets installed as follows:

ls -l  /opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py
-rw-rw-r-- 1 dsf dsf 136333 Mar 23 16:52 /opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py

The zip file structure is:

plugin.json
    |  dwc
          -css
          -js
    | dsf
          - DuetLapse3.py

plugin.json is:

{
    "id": "DuetLapse",
    "name": "DuetLapse",
    "author": "Stuartofmt",
    "version": "0.0.0",
    "license": "GPL-2.0-or-later",
    "homepage": "https://github.com/stuartofmt/DuetLapse3",
    "dwcVersion": "3.4.5",
	"sbcRequired": true,
	"sbcDsfVersion": "3.4",
	"sbcExecutable": "/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py",
	"sbcExecutableArguments": "-file /opt/dsf/sd/sys/DuetLapse/DuetLapse.config",
	"sbcOutputRedirected": false,
	"sbcPermissions": ["networkAccess", "webcamAccess", "readSystem", "fileSystemAccess"]
 }

Services are installed and running

pi@srsender:~ $ sudo systemctl list-units | grep duet
  duetcontrolserver.service                                                                    loaded active running   Duet Control Server
  duetpluginservice-root.service                                                               loaded active running   Duet Plugin Service (root)
  duetpluginservice.service                                                                    loaded active running   Duet Plugin Service
  duetwebserver.service                                                                        loaded active running   Duet Web Server

stuartofmt

I've been trying variations on the theme but cannot get past the above error.

This, from the console, may add more insight.

vue.runtime.esm.js:1897 Error: Operation failed (Reason: InternalServerException in StartPlugin:
Win32Exception in StartPlugin: An error occurred trying to start process
'/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py'
with working directory '/opt/dsf/plugins/DuetLapse/dsf'. No such file or directory)
    at new ExtendableBuiltin (app.df8f8eab.js:42:673288)
    at OperationFailedError._createSuperInternal (createSuper.js:12:24)
    at new NetworkError (errors.js:16:19)
    at OperationFailedError._createSuperInternal (createSuper.js:12:24)
    at new OperationFailedError (errors.js:42:3)
    at B.onload (RestConnector.js:132:13)

stuartofmt

Further info that may help.

pi@srsender:~ $ tail -f /var/log/syslog | grep plugin
Mar 25 10:31:50 srsender DuetWebServer[655]: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 PUT http://192.168.30.30/machine/plugin application/zip 46476

Mar 25 10:31:51 srsender kernel: [571738.844867] audit: type=1400 audit(1679761911.289:129):
apparmor="STATUS" operation="profile_load" profile="unconfined" name="/opt/dsf/plugins/DuetLapse/**" pid=14305
comm="apparmor_parser"
Mar 25 10:31:51 srsender DuetWebServer[655]: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished HTTP/1.1 PUT http://192.168.30.30/machine/plugin application/zip 46476 - 204 - - 965.0586ms
Mar 25 10:31:53 srsender DuetPluginService[929]:    System.ComponentModel.Win32Exception

(2): An error occurred trying to start process '/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py'
with working directory '/opt/dsf/plugins/DuetLapse/dsf'. No such file or directory

Mar 25 10:31:53 srsender DuetControlServer[421]:    DuetAPIClient.InternalServerException:
Win32Exception in StartPlugin: An error occurred trying to start process '/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py'
with working directory '/opt/dsf/plugins/DuetLapse/dsf'. No such file or directory

Mar 25 10:31:53 srsender DuetWebServer[655]: DuetWebServer.Controllers.MachineController[0]
[StartPlugin Failed to start plugin DuetAPIClient.InternalServerException: InternalServerException in StartPlugin:
Win32Exception in StartPlugin: An error occurred trying to start process '/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py'
with working directory '/opt/dsf/plugins/DuetLapse/dsf'. No such file or directory
at DuetAPIClient.BaseConnection.PerformCommand(BaseCommand command, CancellationToken cancellationToken)
in /home/christian/Duet3D/DuetSoftwareFramework/src/DuetAPIClient/BaseConnection.cs:line 155
at DuetWebServer.Controllers.MachineController.StartPlugin() in
/home/christian/Duet3D/DuetSoftwareFramework/src/DuetWebServer/Controllers/MachineController.cs:line 1065

Falcounet

"sbcExecutable": "/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py", shoud be "sbcExecutable": "DuetLapse3.py",

You can use https://github.com/LoicGRENON/DSF_ExecOnMcode_Plugin/blob/main/src/plugin.json as example

Edit :
I also think "sbcExecutableArguments": "-file /opt/dsf/sd/sys/DuetLapse/DuetLapse.config" would be better be "sbcExecutableArguments": "-file ../DuetLapse.config" as the base path is configurable in DSF config

stuartofmt

@Falcounet said in Error trying to start a plugin:

"sbcExecutable": "/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py", shoud be "sbcExecutable": "DuetLapse3.py",

You can use https://github.com/LoicGRENON/DSF_ExecOnMcode_Plugin/blob/main/src/plugin.json as example

I had already tried this and noted it in the original post. It gives the same error.

@Falcounet said in Error trying to start a plugin:

Edit :
I also think "sbcExecutableArguments": "-file /opt/dsf/sd/sys/DuetLapse/DuetLapse.config" would be better be "sbcExecutableArguments": "-file ../DuetLapse.config" as the base path is configurable in DSF config

I may be missing something here, but the config file is user created and needs to be able to be changed easily. This is why I placed it in a location that is directly accessible (and editable) using DWC.
The intention is that a user will create a new folder from system, using DWC, (e.g. DuetLapse) and then create their custom config file.

Are you saying that ../DuetLapse.config is equivalent to /opt/dsf/sd/sys/DuetLapse/DuetLapse.config

In any case, I just tried again and got the same error.

{
    "id": "DuetLapse",
    "name": "DuetLapse",
    "author": "Stuartofmt",
    "version": "0.0.0",
    "license": "GPL-2.0-or-later",
    "homepage": "https://github.com/stuartofmt/DuetLapse3",
    "dwcVersion": "3.4.5",
	"sbcRequired": true,
	"sbcDsfVersion": "3.4.5",
	"sbcExecutable": "DuetLapse3.py",
	"sbcExecutableArguments": "-file ../DuetLapse.config",
	"sbcOutputRedirected": false,
	"sbcPermissions": ["commandExecution", "networkAccess", "webcamAccess", "readSystem", "fileSystemAccess"]
 }

Falcounet

@stuartofmt said in Error trying to start a plugin:

Are you saying that ../DuetLapse.config is equivalent to /opt/dsf/sd/sys/DuetLapse/DuetLapse.config

Yes but using a relative path so it doesn't depends on the /opt/dsf/sd/ path which is configurable on /opt/dsf/config/config.json

Falcounet

@stuartofmt said in Error trying to start a plugin:

In any case, I just tried again and got the same error.

Is the error still a permission denied error ?
Did you tried to execute the python script from the CLI as dsf user ?
sudo -u dsf /opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py -file /opt/dsf/sd/sys/DuetLapse/DuetLapse.config

I remember I had some similar issue and it was because of the line termination being incorrect in the script. Using dos2unix solved the issue.

stuartofmt

@Falcounet said in Error trying to start a plugin:

sudo -u dsf /opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py -file /opt/dsf/sd/sys/DuetLapse/DuetLapse.config

Yes, I was trying that when I saw your post. It does indeed give an error.

sudo -u dsf /opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py -file /opt/dsf/sd/sys/DuetLapse/DuetLapse.config
sudo: unable to execute /opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py: No such file or directory

If I try using straight sudo it complains that it cannot find a python module, but at least tries to start. I likely installed all the required modules as my regular use (Pi).

sudo python3 /opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py -file /opt/dsf/sd/sys/DuetLapse/DuetLapse.config
Traceback (most recent call last):
  File "/opt/dsf/plugins/DuetLapse/dsf/DuetLapse3.py", line 84, in <module>
    import psutil

Although the permissions look OK for dsf

pi@srsender:/opt/dsf/plugins/DuetLapse/dsf $ ls -l
total 136
-rwxrwx--- 1 dsf dsf 136333 Mar 25 13:53 DuetLapse3.py

Falcounet

@stuartofmt Do you have the shebang #!/usr/bin/python3 at the top of your python script ?

For the module missing, look at "sbcPythonDependencies" option of the plugin manifest (and also look at DSF_ExecOnMcode_Plugin example).

stuartofmt

@Falcounet said in Error trying to start a plugin:

@stuartofmt Do you have the shebang #!/usr/bin/python3 at the top of your python script ?

For the module missing, look at "sbcPythonDependencies" option of the plugin manifest (and also look at DSF_ExecOnMcode_Plugin example).

I double checked the EOL for all the files and they are using linux convention.

I previously had #!python3 . I changed it to #!/usr/bin/python3 but no difference. Same error message.

I have multiple python modules (all installed) and the script runs fine as the pi user. I was going to adjust that aspect next, but with the script not even starting .... well, I figured first things first :-).

Anyway, since psutil is used for filesystem actions, and with the possibility that the error message wa shappening while the script was actually starting, I added "sbcPythonDependencies": ["psutil"]

Now I get this error. So maybe something is wrong with my dsf configuration ? I installed dsf, using the official guide, over an already existing Debian Bullseye install.

Falcounet

@stuartofmt said in Error trying to start a plugin:

@Falcounet said in Error trying to start a plugin:

@stuartofmt Do you have the shebang #!/usr/bin/python3 at the top of your python script ?

For the module missing, look at "sbcPythonDependencies" option of the plugin manifest (and also look at DSF_ExecOnMcode_Plugin example).

I double checked the EOL for all the files and they are using linux convention.

As long you can run your script from the CLI, it is fine.

I previously had #!python3 . I changed it to #!/usr/bin/python3 but no difference. Same error message.

I have multiple python modules (all installed) and the script runs fine as the pi user. I was going to adjust that aspect next, but with the script not even starting .... well, I figured first things first :-).

Then it's probably because you installed these modules only for the pi user and not as a system wide installation.

Anyway, since psutil is used for filesystem actions, and with the possibility that the error message wa shappening while the script was actually starting, I added "sbcPythonDependencies": ["psutil"]

Now I get this error. So maybe something is wrong with my dsf configuration ? I installed dsf, using the official guide, over an already existing Debian Bullseye install.

python3-pip which provides pip3 isn't installed on your system.
I would suggest using the latest DuetPi image. @chrishamm fixed this lately.

stuartofmt

@Falcounet said in Error trying to start a plugin:

I would suggest using the latest DuetPi image. @chrishamm fixed this lately.

Thanks for hanging in there with me. I was reluctant to go the route of the Duet Pi image for all the obvious reasons. As the old saying goes "shortcuts seldom are".

I do actually have pip3 install but (as you pointed out) likely just for the pi user.
Given the issues around playing with Python system vs user; I'll start again from a DuetPi image. Hopefully I have backed up all my files and getting everything back will be straightforward.

I'll report back when I'm done - hopefully error free

Falcounet

@stuartofmt said in Error trying to start a plugin:

Thanks for hanging in there with me. I was reluctant to go the route of the Duet Pi image for all the obvious reasons. As the old saying goes "shortcuts seldom are".

You are welcome.
You can use another SD card to keep your current image or also make a backup of your current image, it take only few minutes

That's also good to test on a fresh DuetPi image before releasing a version so you are sure you didn't miss something you had to install during development and so.

stuartofmt

@Falcounet

A couple of small challenges along the way - but I have a working plugin !!

Many thanks for all your help.