DuetLapse available for Alpha testing

stuartofmt

EDIT: These changes can be downloaded from here:
https://github.com/stuartofmt/DuetLapse

@Danal
Another small suggested mod.

I start DuetLapse.py remotely using a bash script and run it in background. Ctl+C cannot stop it if needed.

I added the following (just above the main routine) so that I could send a SIGINT to stop the process.

#Allows process running in background or foreground to be gracefully
# Shutdown will SIGINt (Kill -2 <pid>
import signal

def quit_gracefully(*args):
    print('Stopped by SIGINT - Post Processing')
    postProcess()
    exit(0);

if __name__ == "__main__":
    signal.signal(signal.SIGINT, quit_gracefully)

The bash script to control this is inelegant at this time but here it is to illustrate the use:

#! /bin/bash

PROCESS=/home/pi/Lapse/DuetLapse.py

case "$1" in
	start)	
       /usr/bin/python3 $PROCESS -duet 192.168.86.235 -camera ffmpeg -weburl http://192.168.86.230:8081/stream.mjpg -detect layer &
       ;;
    stop)
	   PROCESS_PID=`/bin/ps -ef | grep "$PROCESS" | grep -v "grep" | awk '{print $2}'`
	   kill -2 $PROCESS_PID
	   echo stopped
	   ;;
esac

Phaedrux

@stuartofmt said in DuetLapse available for Alpha testing:

@Danal
Another small suggested mod.

Sorry to tell you this, but Danal passed away several months ago now.

stuartofmt

@Phaedrux
Very, very sorry to hear that. Thanks for letting me know.

Do you know if anyone is taking over this work? I can do a little but I'm far from a professional programmer. More a programming by (google) example type.

I'll post my modifications in full, here, later today. I have some more slight mods coming but likely not for a couple of weeks

Phaedrux

@stuartofmt starting a fork of the project on Github might be the best way forward. Then you can tinker away and it's available for others to use as well. I know Danal would have approved.

stuartofmt

I have created a fork of my changes so far and will update as I make additional tweaks.

These changes can be downloaded from here:
https://github.com/stuartofmt/DuetLapse

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

See my post above
If you have two applications both trying to connect to the same Pi camera you will get these ENOSPC errors. This does not mean this is your exact problem - but is a good place to start.

Thanks.

I have a solution in place that is working for me. I have ustreamer controlling the camera, and I have DuetLapse grabbing the images from ustreamer. I have both started by systemctl on my CentOS system. I have a script running that'll start capturing with DuetLapse by layer whenever a print starts.

The only modification I'd like to do is an ffmpeg input to add 2 seconds of the last jpeg image to the timelapse. I know it is possible - just need to figure it out.

John

stuartofmt

@JohnOCFII

Good to hear. When you say add 2 seconds - do you mean continue recording for 2 second after Duet reports that it is no longer printing ?

stuartofmt

@JohnOCFII

Maybe a small change like this at the end ?

        elif (printerState == 2):
            onePhoto()
            time.sleep(2)
            onePhoto()
            postProcess()

stuartofmt

@JohnOCFII

Seeing I'm on a (UNTESTED) role. This may be better than the crude suggestion above. Note that it requires ffmpeg from Nov 2018 or newer.

I'll try it later today as I have a print to get done.

def postProcess():
    print()
    print("Now making {0:d} frames into a video at 10 frames per second.".format(int(np.around(frame))))
    if (250 < frame): print("This can take a while...")
#    fn = basedir+'DuetLapse'+time.strftime('%m%d%y%H%M',time.localtime())+'.mp4'
    fn = basedir+'DuetLapse'+time.strftime('%a-%H:%M',time.localtime())+'.mp4'
    if (vidparms == ''):
        cmd  = 'ffmpeg -r 10 -i /tmp/DuetLapse/IMG%08d.jpeg -vcodec libx264 -y -v 8 tpad=stop_mode=clone:stop_duration=2 '+fn
#        cmd  = 'ffmpeg -r 10 -i /tmp/DuetLapse/IMG%08d.jpeg -vcodec libx264 -y -v 8 '+fn
    else:
        cmd  = 'ffmpeg '+vidparms+' -i /tmp/DuetLapse/IMG%08d.jpeg tpad=stop_mode=clone:stop_duration=2 '+fn
#        cmd  = 'ffmpeg '+vidparms+' -i /tmp/DuetLapse/IMG%08d.jpeg '+fn        
    subprocess.call(cmd, shell=True)
    print('Video processing complete.')
    print('Video file is in '+basedir+' directory, named '+fn)
    exit()

JohnOCFII

@stuartofmt Yes -- something like that. Basically hold the last image for 2 seconds.

Here's how OctoPrint refers to it:

Screen Shot 2020-12-22 at 12.49.28 PM.png

stuartofmt

Seems like the tpad filter is missing from my version (4.1.6) of ffmpeg Debian Buster on Pi.

Also the syntax above may (or not) work ....

I may have to compile ffmpeg from the latest source - will let you know.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

Seems like the tpad filter is missing from my version (4.1.6) of ffmpeg Debian Buster on Pi.

I seem to have the same version of ffmpeg:

ffmpeg version 4.1.6-1~deb10u1+rpt1 Copyright (c) 2000-2020 the FFmpeg developers

I should troll through the OctoPrint/OctoPi code to see what they do.

oozeBot

In case it went overlooked, I just wanted to toss out there that I came up with a solution for timelapse video when using a Duet 3 with attached Raspberry Pi..

https://forum.duet3d.com/topic/19225/timelapse-video-through-execonmcode

stuartofmt

@oozeBot

Thanks - a nice piece of work. I looked at it at one time but did not want to fiddle with generating the additional slicer code (likely an irrational reaction).

In my case I could replace the wget with ffmpeg for the stream output from the app I am using and also the modified aggregation ffmpeg.

I may end up back at your script - but for now I am wrestling with recompiling ffmpeg to get some additional filters that are not present in the somewhat old Pi version.

oozeBot

@stuartofmt I get it, but maybe there is something there that could be built upon. It would also be possible to put the trigger in daemon.g but I wanted to control the trigger within the gCode at precisely the same point, etc..

stuartofmt

@oozeBot

Just so you know - DuetLapse is working fine and has some nice features. I'm just embellishing around the edges.

Being unfamiliar with Python - some of it is trial and error. Also some of it (e.g. ffmpeg on Pi being quite old) would be the same in either approach.

I've written quite a few bash scripts in the past and it would be a bit of a slog to achieve the same flexibility as DuetLapse with bash - although not impossible by any means.

As of this afternoon - I have another embellishment done (required newer ffmpeg) and and will update my git repository shortly.

stuartofmt

@JohnOCFII

I'm pretty well there - doing some final testing now. I'll post the new DuetLapse.py later today / tomorrow. It has additional parameters to control the output directory and the extra time at the end. The defaults are the same as before.

I'm PRETTY sure that the standard Pi ffmpeg does not have tpad - but you can test with the following command to see (required a different syntax than I originally tried) .... I updated ffmpeg before I found this working syntax ....

ffmpeg -r 10 -i /tmp/DuetLapse/IMG%08d.jpeg -c:v libx264 -vf tpad=stop_mode=clone:stop_duration=2,fps=10 ./out1.mp4

Here are the steps I went through to build a more up-to-date ffmpeg (well - the straight line version that avoids the various dead ends and failures to compile the optional packages). They were taken from here but shortened to omit the optional packages in the post and using a build that omits calling those packages.

https://pimylifeup.com/compiling-ffmpeg-raspberry-pi/

Remove existing ffmpeg

sudo apt remove ffmpeg

Make sure OS is up to date

sudo apt update
sudo apt upgrade

Install dependencies

sudo apt -y install autoconf automake build-essential cmake doxygen git graphviz imagemagick libasound2-dev libass-dev libavcodec-dev libavdevice-dev libavfilter-dev libavformat-dev libavutil-dev libfreetype6-dev libgmp-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libopus-dev librtmp-dev libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev libsdl2-net-dev libsdl2-ttf-dev libsnappy-dev libsoxr-dev libssh-dev libssl-dev libtool libv4l-dev libva-dev libvdpau-dev libvo-amrwbenc-dev libvorbis-dev libwebp-dev libx264-dev libx265-dev libxcb-shape0-dev libxcb-shm0-dev libxcb-xfixes0-dev libxcb1-dev libxml2-dev lzma-dev meson nasm pkg-config python3-dev python3-pip texinfo wget yasm zlib1g-dev libdrm-dev

Build ffmpeg (takes a while)

git clone --depth 1 https://github.com/FFmpeg/FFmpeg.git ~/FFmpeg \
  && cd ~/FFmpeg \
  && ./configure \
    --extra-cflags="-I/usr/local/include" \
    --extra-ldflags="-L/usr/local/lib" \
    --extra-libs="-lpthread -lm -latomic" \
    --arch=armel \
    --enable-gmp \
    --enable-gpl \
    --enable-libass \
    --enable-libdrm \
    --enable-libfreetype \
    --enable-libmp3lame \
    --enable-libopencore-amrnb \
    --enable-libopencore-amrwb \
    --enable-libopus \
    --enable-librtmp \
    --enable-libsnappy \
    --enable-libsoxr \
    --enable-libssh \
    --enable-libvorbis \
    --enable-libwebp \
    --enable-libx264 \
    --enable-libx265 \
    --enable-libxml2 \
    --enable-mmal \
    --enable-nonfree \
    --enable-omx \
    --enable-omx-rpi \
    --enable-version3 \
    --target-os=linux \
    --enable-pthreads \
    --enable-openssl \
    --enable-hardcoded-tables \
  && make -j$(nproc) \
  && sudo make install

Check the ffmpeg version

pi@srsenderpi:~ $ ffmpeg -version
ffmpeg version git-2020-12-22-a7f9b3b Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/lib --extra-libs='-lpthread -lm -latomic' --arch=armel --enable-gmp --enable-gpl --enable-libass --enable-libdrm --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libssh --enable-libvorbis --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-mmal --enable-nonfree --enable-omx --enable-omx-rpi --enable-version3 --target-os=linux --enable-pthreads --enable-openssl --enable-hardcoded-tables
libavutil      56. 62.100 / 56. 62.100
libavcodec     58.115.102 / 58.115.102
libavformat    58. 65.100 / 58. 65.100
libavdevice    58. 11.103 / 58. 11.103
libavfilter     7. 94.100 /  7. 94.100
libswscale      5.  8.100 /  5.  8.100
libswresample   3.  8.100 /  3.  8.100
libpostproc    55.  8.100 / 55.  8.100

ffmpeg - filters shows tpad being available.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

I'm pretty well there - doing some final testing now.

I'm PRETTY sure that the standard Pi ffmpeg does not have tpad - but you can test with the following command to see (required a different syntax than I originally tried) .... I updated ffmpeg before I found this working syntax ....

I'll try this tomorrow (US Central time - GMT-6).

stuartofmt

@JohnOCFII

Updated here:

https://github.com/stuartofmt/DuetLapse

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

Updated here:

https://github.com/stuartofmt/DuetLapse

Greetings!

I was hoping that simply replacing ffmpeg and the new DuetLapse.py would "just work," but sadly, not the case.

I confirmed that my previous ffmpeg didn't have support for the pad filter, as you suggested.

I rebuilt ffmpeg according to your instructions. That all went well.

I then tried to execute the DuetLapse script but you addition of -psutil- wasn't working, so I Googled how to build that, and did so.

I then went to run DuetLapse, but received the message:

Process is already running...

Probably because I have DuetLapse starting from -systemd-

I'll have to see what else is different.

John