DuetLapse3

stuartofmt

@johnocfii said in DuetLapse3:

I'm running 3.4.2. I was on 3.4.1 before that.

Do you mind trying 3.4.3. Looks like it was a silly mistake - but I do not have time to rigorously test at the moment.

JohnOCFII

@stuartofmt sure - I'll try that soon.

JohnOCFII

@stuartofmt Worked great! https://youtu.be/vvtMgD2nNE0

war4peace

@stuartofmt
No rush, I am still tuning and polishing a lot of things with my printer. Take your time

stuartofmt

@war4peace

Hi - I think the simplest approach (for now) is to run DuetLapse3.py using the linux systemctl capability.

I updated the instructions here:
https://github.com/stuartofmt/DuetLapse3/blob/main/systemctl setup.md
and added a sample unit file (.service extension) here:
https://github.com/stuartofmt/DuetLapse3/blob/main/DuetLapse3.service

Note that you want to make sure the options you have set are perfected by running DuetLapse3 from the command console, before you replicate these in the unit file.

Essentially - using this approach, DuetLapse3 will terminate normally at the end of each print job and then restart.

In particular - you likely (definitely) want to have -detect layer set. Otherwise - on restart of DuetLapse3, even if there is no print job running, it may start capturing images the entire time

war4peace

@stuartofmt

Thank you for your answer.

Here's my command line (tested to be working, located in a bash file, see below):

/usr/bin/python3 $PROCESS -duet localhost -basedir /home/pi/duetlapse/output -camera1 stream -weburl1 http://localhost:8181/stream -detect layer -seconds 10  -extratime 10 -instances oneip -deletepics -port 8282 &

I am trying to follow the instructions listed, but I have a couple questions.
In the document it says "This needs to be the directory for BOTH startDuetLapse3.py and DuetLapse3.py" - there is no "startDuetLapse3.py" file to be found.
Here's the full list of files in the DuetLapse folder:

pi@duet3:~/duetlapse $ find "$(pwd)"
/home/pi/duetlapse
/home/pi/duetlapse/duetlapse.sh
/home/pi/duetlapse/DuetLapse3.py
/home/pi/duetlapse/output
/home/pi/duetlapse/output/duet3
/home/pi/duetlapse/output/duet3/localhost
/home/pi/duetlapse/startDuetLapse3.service

As for starting DUetLapse, I am using a bash file with the following contents:

#! /bin/bash
#Example bash script for running DuetLapse3.py

PROCESS=/home/pi/duetlapse/DuetLapse3.py

case "$1" in
   start)
#Edit and uncomment for your own needs
#One camera example
      /usr/bin/python3 $PROCESS -duet localhost -basedir /home/pi/duetlapse/output -camera1 stream -weburl1 http://localhost:8181/stream -detect layer -seconds 10  -extratime 10 -instances oneip -deletepics -port 8282 &
#  Two camera example
#/usr/bin/python3 $PROCESS -duet 192.168.86.235 -basedir /home/pi/Lapse -instances oneip -dontwait -seconds 3 -camera1 stream -weburl1 http://192.168.86.230:8081/stream.mjpg  -camera2 other -weburl2 http://192.168.86.230:8081/st$
   ;;
   stop)
      if ["$2" == '']
      then
          PROCESS_PID=`/bin/ps -ef | grep "$PROCESS" | grep -v "grep" | awk '{print $2}'`
          kill -2 $PROCESS_PID
      else
          PROCESS_PID=`/bin/ps -ef | grep "$PROCESS" | grep "$2" | grep -v "grep" | awk '{print $2}'`
          kill -2 $PROCESS_PID
     fi
   ;;
esac

Can I use that bash script instead of startDuetLapse3.py?

I am a newbie in linux-world, sorry for the arguably stupid questions...

JohnOCFII

@war4peace The file startDuetLapse3.py can be found along with the rest of the DuetLapse files here: https://github.com/stuartofmt/DuetLapse3

It is fairly straightforward to replicate if needed.

John

stuartofmt

@war4peace said in DuetLapse3:

In the document it says "This needs to be the directory for BOTH startDuetLapse3.py and DuetLapse3.py" - there is no "startDuetLapse3.py" file to be found.
.

I'll fix that - It should have read This needs to be the directory in which you have DuetLapse3.py installed. If you are using startDuetLapse3.py it will be in the same directory as DuetLapse3.

Can I use that bash script instead of startDuetLapse3.py?

For your use-case, I would not use a bash script at all. The bash script was there to support a much earlier version, when the documentation was less complete. Since then the inbuilt capability together with startDuetLapse3 makes the bash script redundant (in most cases) . I will remove it as it no longer serves a purpose.

Can I use that bash script instead of startDuetLapse3.py?

startDuetLapse3 is an administrative tool. Its use is optional but has many useful features - way beyond a bash script. It's accessible from a browser - so you can administer from more-or-less any local machine. I also use VPN to access it remotely

If you want a single instance of DuetLapse3 running all the time - then you are better off using systemctl for that instance. The key lines in you unit file (given the info you provided) would look like this.

<snip>
WorkingDirectory=/home/pi/duetlapse
<snip>
ExecStart=python3 /home/pi/duetlapse/DuetLapse3.py -duet localhost -basedir /home/pi/duetlapse/output -camera1 stream -weburl1 http://localhost:8181/stream -detect layer -seconds 10 -extratime 10 -instances oneip -deletepics -port 8282
<snip>

Of course you can run additional ad-hoc instances if you like (e.g. through startDuetLapse3).

With that in mind - you may find -instances oneip too restrictive or redundant. The very first instance of DuetLapse essentially ignores this directive because there can be no conflict. Subsequent instances check already running instances for possible conflict based on their directive only. In other words these options are not global in their scope. With -instances oneip - you cannot have 2 or more instances connected to the same Duet. With -instances many, you can have many instances connected to the same (or different) Duets.

war4peace

@johnocfii

Thank you. I have followed the instructions, but receiving an error, see below:

pi@duet3:~/duetlapse $ sudo systemctl status startDuetLapse3.service
● startDuetLapse3.service - startDuetLapse3 Service
   Loaded: loaded (/lib/systemd/system/startDuetLapse3.service; disabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2021-05-21 00:16:32 EEST; 6s ago
  Process: 15508 ExecStart=/usr/bin/python3 startDuetLapse3.py -duet localhost -basedir /home/pi/duetlapse/output -camera1 stream -weburl1 http://localhost:8181/stream -detect layer -seconds 10 -extratime 10 -instances oneip -deletepics -port 8282 & (code=exited, status=2)
 Main PID: 15508 (code=exited, status=2)

May 21 00:16:32 duet3 systemd[1]: startDuetLapse3.service: Service RestartSec=100ms expired, scheduling restart.
May 21 00:16:32 duet3 systemd[1]: startDuetLapse3.service: Scheduled restart job, restart counter is at 5.
May 21 00:16:32 duet3 systemd[1]: Stopped startDuetLapse3 Service.
May 21 00:16:32 duet3 systemd[1]: startDuetLapse3.service: Start request repeated too quickly.
May 21 00:16:32 duet3 systemd[1]: startDuetLapse3.service: Failed with result 'exit-code'.
May 21 00:16:32 duet3 systemd[1]: Failed to start startDuetLapse3 Service.

stuartofmt

@war4peace said in DuetLapse3:

Thank you. I have followed the instructions, but receiving an error, see below:

Just to be clear: Are you intending to start startDuetLapse3.py or DuetLapse3.py?
They are two separate programs startDuetLapse3 provides an administrative, browser-based UI for starting and managing DuetLapse3.py.

In any case:

You did not provide the full path name in the ExecStart command.
It looks like you want to start DuetLapse3.py. If that's the case - you are calling the wrong program.
If you intended to start startDuetLapse3.py then the command options have a different syntax due to some vagaries of how such things are passed by the operating system. See the documentation on startDuetLapse3 here (in particular the -args syntax):
https://github.com/stuartofmt/DuetLapse3/blob/main/startDuetLapse3.md

war4peace

@stuartofmt

Thank you again for your patience.

Updated command line:

ExecStart=/usr/bin/python3 /home/pi/duetlapse/startDuetLapse3.py -duet localhost -basedir /home/pi/duetlapse/output -camera1 stream -weburl1 http://localhost:8181/stream -detect layer -seconds 10  -extratime 10 -instances oneip -deletepics -port 8282

Current status: service starts, then ends after a short while. Running "sudo systemctl status startDuetLapse3.service" immediately after starting the service shows it as running, but after a few seconds it exits. Must be something in the command line.

I am now going through the manual for StartDuetLapse and will adjust accordingly.
Will post updates once done.

EDIT: It is working now.

Thank you very much for your help, much appreciated!

stuartofmt

@war4peace said in DuetLapse3:

I am now going through the manual for StartDuetLapse and will adjust accordingly.

You need to have firmly in your mind which program you are dealing with Many of the options (because of the way you specified them) are incorrect for startDuetLapse3. (which is the program you are trying to start). You are using some options that only apply to DuetLapse3. This is wrong.

If you stop startDuetLapse3

sudo systemctl stop startDuetLapse3.service

and then run the command you are trying to use

/usr/bin/python3 /home/pi/duetlapse/startDuetLapse3.py -duet localhost -basedir /home/pi/duetlapse/output -camera1 stream -weburl1 http://localhost:8181/stream -detect layer -seconds 10 -extratime 10 -instances oneip -deletepics -port 8282

you will see what I mean - you will get errors about the options that you are trying to use.

war4peace

@stuartofmt
Understood.

Confusion cleared.

Later Edit I apologize, I was busy at the moment and now realized my answer was too brief.
As I had said, „I am now going through the manual for StartDuetLapse and will adjust accordingly.” Which I did, used the "args=" parameter as specified in the manual and everything is in order now.

stuartofmt

@war4peace said in DuetLapse3:

everything is in order now

Excellent !! I do have a newer version of start DuetLapse3 coming out in a couple of days. Minor tweaks and corrections. Nothing that will cause a change in arguments or setup. A simple file replacement.

stuartofmt

Its been a while -- I've made updates to both DuetLapse3 and startDuetLapse3.
Included in the documentation are instructions on using DuetLapse3 with the Pi camera on Buster.

Version 3.5.0

Remember to keep DuetLapse3 and startDuetLapse3 at the same release levels

Changes to DuelLApse3

[1] Changed some system calls to allow for better error handling.
[2] Fixed an issue with sending gcodes to SBC
[3] Improved the handling of -pause yes and - movehead
[4] Added an optional argument -rest that delays image capture after a pause. This is because the camera feed can be delayed with respect to the actual head position.
[5] Updated documentation with additional notes
[6] Deprecated the use of -camera pi due to changes in Raspberry Pi (see notes in the section on -camera).
[7] Added a new argument -nolog and deprecated -logtype. Logging will always use the console unless the program is running in the background.
[8] If -verbose is used, much more detail is created. Should usually only be used for debugging.
[9] Added new argument -fps. Sets the default frames-per-seconds
[10] Added the ability to change the default frames-per-second (fps) from the main menu.
[11] Added a new argument -hidebuttons. Hides menu buttons that are currently invalid. Otherwise, invalid buttons are greyed out.
[12] General UI improvements.
[13] After Snapshot, returns to the previous logical state either 'start' or 'pause'

Changes to startDuetLapse3

[1] Changed some system calls to allow for better error handling
[2] Updated documentation with additional notes
[3] Added a default logfile (startDuetLapse3.log)
[4] Added a new argument -nolog. Logging will always use the console unless the program is running in the background.
[5] Added a new argument -verbose. Creates detailed debugging information.
[6] Failure to start DuetLapse3 (from the UI) is reported with reasons. Successful starts are reported after 30 seconds.
[7] Shutdown, from the UI, requires confirmation.
[8] The use of http option nohup=yes is deprecated.
[9] Added new argument -fps. Sets the default frames-per-seconds
[10] Added the ability to change the default frames-per-second (fps) when creating a video from the files menu.
[11] General UI improvements.

stuartofmt

I am please to announce the release of version 4.0.0

It has a number of changes, the most significant being the ability to control DuetLapse3 with gcode.

Controlling DuetLapse3 with gcode

The program, among other things, monitors messages generated with the gcode M117 command.
Two message forms will cause the program to react.

[1] M117 DuetLapse3.(x) where (x) is one of the allowed settings.
[2] M117 (execkey) (command) where (execkey) is a prefix specified by the -execkey option and (command) is an arbitrary command that will be sent to the OS.

M117 gcode messages can be embedded in the print file, placed in a macro, or sent from the DWC console.

Note that as a practical matter, This functionality assumes a single instance of DuetLapse3 connected to a single printer.

DuetLapse3 uses a polling method and checks the printer every 5 seconds for M117 messages.<br>
To prevent a message being missed, IT IS MANDATORY that a delay (G4) be added after every M117 message intended for this program.
The strong recommendation is that the delay is 10 seconds i.e. G4 S10

M117 DuetLapse3.(x)

In the first form, the following settings are available:
start, standby, pause, continue, restart, snapshot, completed, graceful, forced

These correspond to the same actions in the UI.<br>
Note that terminate is not supported instead use graceful or forced depending on your need.

There is also a special variant which can be used to change options on-the-fly.
This is equivalent to setting an option in the command line at startup.

M117 DuetLapse3.change.(variable)=(value)
G4 S10

The following variables are supported:<br>
verbose, seconds, poll, detect, dontwait, pause, restart, novideo, keepfiles, minvideo, extratime, fps, rest , execkey

Examples

M117 DuetLapse3.standby # Will place the program into standby
G4 S10

M117 DuetLapse3.start # Will start capturing image
G4 S10

M117 DuetLapse3.change.verbose=False #  will turn of verbose output
G4 S10

M117 DuetLapse3.change.seconds=20 # Will capture an image every 20 seconds
G4 S10

M117 (execkey) (command)

This form allows an arbitrary command to be executed by the operating system.
The character sequence specified by -execkey is used to identify the command.

For example if -execkey was :do: the following message will attempt to run test.sh

M117 DuetLapse3 :do: ./test.sh "hello world"
G4 S10

Note: There are no additional single or double quotes used in the M117 gcode.
The command portion is presented as it would be from the command line of the relevant OS.

Test Example

This example demonstrates controlling DuetLapse3 using M117 messages.
It does not print anything but simulates a small print job.
DuetLapse3. messages are both inline (in the gcode) and called inside a macro.

Copy the following file to your printer job folder:

M117Test.gcode

and this file to the macro folder:

test_job_settings.g

Optionally (Linux only) create a file test.sh in the DuetLapse3 directory.

#!/bin/bash
echo "-----------"
echo "$1 $2"
echo "----------"

Don't forget to make the file executable:

chmod + x ./test.sh

Start DuetLapse3 with the following suggested options, in addition to those needed for -duet and -camera:
-restart -verbose -standby -keepfiles

Note

[1] The use of M117 DuetLapse3.standby early in the print job (in the macro) to prepare for capture.
[2] The use of M117 DuetLapse3.start to control when capture will start.
[3] The use of M117 DuetLapse3.complated to indicate when capture will stop.

Placement of these options allows fine control over the timelapse. This is especially useful if -restart is used and DuetLapse3 is running continuously.

Macros can be especially useful if running DuetLapse3 continuously.
You could (for example) having a standard set of options that are called at the end of each print job and specific macros (layer only, layer and time, time only, different -second settings etc.) for certain type of timelapse.
Embedding these macro calls from your slicer makes easy use of this functionality.

stuartofmt

###VERSION 4.0.0
[1] Added the ability to control the program with gcode M117 messages
[2] Added the ability to execute an arbitrary program by sending M117 gcode with configurable prefix
[3] Added -restart option to allow a program to continue running at end of print
[4] Significantly improved ability to continue running after extended network interruptions.
[5] Can continue to run even if printer is turned off
[6] Changed - extratime handling so as not to require tpad support in ffmpeg
[7] Added -minvideo option to set minimum video length (seconds default 5)
[8] Changed default -poll to minimum of 12 seconds.
[9] Changed default -seconds to minimum of 12 seconds.
[10] Suppressed a lot of output unless -verbose is used
[11] Changed logfile naming convention to better support - restart.
Initially the logfile is created with a timestamp. When the print starts the logfile is renamed to reflect the name of the print job

The latest versions can be downloaded from here:

DuetLapse3.py

startDuetLapse3.py

JohnOCFII

@stuartofmt Glad to see work continue on this very useful tool!

stuartofmt

@johnocfii Hope you like the new additions. My intent was to allow fairly precise control and at the same time provide the option to have DuetLapse3 run continuously between print jobs.
Any and all feedback greatly appreciated.

stuartofmt

This post is just to illustrate the sort of flexibility and control that DuetLapse3 V4.0 provides.

Usually, when I print, I like to use a combination of layer changes and time to capture images. This makes the position of the printhead somewhat ambiguous at the end of the print job.
I also lower the print bed at the end of the print job.
For timelapse, I want the print head out of the way for the last image capture and I do not want any of the lowering etc.

Here is what I want it to look like:
Benchy.mp4

The following skeletal gcode illustrates one way of achieving this:

; This is the Start G-code
;  Call macro that puts DuetLapse3 into standby and sets the desired parameters
;  Among which seconds=20 and detect=layer and extratime=4
;
;M98 P"/macros/timelapse_layer_20sec.g"
;
;<snip> Pre-print gcode
;
M117 DuetLapse3.start  ; starts both layer and seconds capture just as the real print starts
G4 S10
;
; End of start G-code
;
;<snip>  The actual print gcode
;
; This is the End-gcode
;
G91           ; Use relative mode
G0 Z1         ;E Move the bed away a little note use of E after comment for correct layer count
G90           ;Back to absolute mode
G0 X0 Y0      ;Get the print head out of the way
              ;I don't do this in pause.g
;
;  This next bit is a bit sneaky - change DuetLapse to detect=pause will capture a final image when the pause is sent
;  Because we previously used G0 X0 Y0 the print head will not be in the way when the extratime images are generated
;
M117 DuetLapse3.change.detect=pause
G4 S10
;
M25          ; Don't use M226 here. DuetLapse3 sends unpause when its ready
G4 S10       ; Give pause unpause etc time to catch up
;
M117 DuetLapse3.completed  ; Stop the DuetLapse3 timelapse here
G4 S10					   ; None of the movements that may follow will be captured in the timelapse
;
; <snip>
;
M98 P"/macros/timelapse_defaults.g"  ; Reset to my normal defaults for DuetLapse3
;
M0            ; stops the print job and calls stop.g which in my case has a move to the bottom that I don't want to see
; End of End-gcode