DuetLapse available for Alpha testing

stuartofmt

I'm pretty well there - doing some final testing now. I'll post the new DuetLapse.py later today / tomorrow. It has additional parameters to control the output directory and the extra time at the end. The defaults are the same as before.

I'm PRETTY sure that the standard Pi ffmpeg does not have tpad - but you can test with the following command to see (required a different syntax than I originally tried) .... I updated ffmpeg before I found this working syntax ....

ffmpeg -r 10 -i /tmp/DuetLapse/IMG%08d.jpeg -c:v libx264 -vf tpad=stop_mode=clone:stop_duration=2,fps=10 ./out1.mp4

Here are the steps I went through to build a more up-to-date ffmpeg (well - the straight line version that avoids the various dead ends and failures to compile the optional packages). They were taken from here but shortened to omit the optional packages in the post and using a build that omits calling those packages.

https://pimylifeup.com/compiling-ffmpeg-raspberry-pi/

Remove existing ffmpeg

sudo apt remove ffmpeg

Make sure OS is up to date

sudo apt update
sudo apt upgrade

Install dependencies

sudo apt -y install autoconf automake build-essential cmake doxygen git graphviz imagemagick libasound2-dev libass-dev libavcodec-dev libavdevice-dev libavfilter-dev libavformat-dev libavutil-dev libfreetype6-dev libgmp-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libopus-dev librtmp-dev libsdl2-dev libsdl2-image-dev libsdl2-mixer-dev libsdl2-net-dev libsdl2-ttf-dev libsnappy-dev libsoxr-dev libssh-dev libssl-dev libtool libv4l-dev libva-dev libvdpau-dev libvo-amrwbenc-dev libvorbis-dev libwebp-dev libx264-dev libx265-dev libxcb-shape0-dev libxcb-shm0-dev libxcb-xfixes0-dev libxcb1-dev libxml2-dev lzma-dev meson nasm pkg-config python3-dev python3-pip texinfo wget yasm zlib1g-dev libdrm-dev

Build ffmpeg (takes a while)

git clone --depth 1 https://github.com/FFmpeg/FFmpeg.git ~/FFmpeg \
  && cd ~/FFmpeg \
  && ./configure \
    --extra-cflags="-I/usr/local/include" \
    --extra-ldflags="-L/usr/local/lib" \
    --extra-libs="-lpthread -lm -latomic" \
    --arch=armel \
    --enable-gmp \
    --enable-gpl \
    --enable-libass \
    --enable-libdrm \
    --enable-libfreetype \
    --enable-libmp3lame \
    --enable-libopencore-amrnb \
    --enable-libopencore-amrwb \
    --enable-libopus \
    --enable-librtmp \
    --enable-libsnappy \
    --enable-libsoxr \
    --enable-libssh \
    --enable-libvorbis \
    --enable-libwebp \
    --enable-libx264 \
    --enable-libx265 \
    --enable-libxml2 \
    --enable-mmal \
    --enable-nonfree \
    --enable-omx \
    --enable-omx-rpi \
    --enable-version3 \
    --target-os=linux \
    --enable-pthreads \
    --enable-openssl \
    --enable-hardcoded-tables \
  && make -j$(nproc) \
  && sudo make install

Check the ffmpeg version

pi@srsenderpi:~ $ ffmpeg -version
ffmpeg version git-2020-12-22-a7f9b3b Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/lib --extra-libs='-lpthread -lm -latomic' --arch=armel --enable-gmp --enable-gpl --enable-libass --enable-libdrm --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libssh --enable-libvorbis --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-mmal --enable-nonfree --enable-omx --enable-omx-rpi --enable-version3 --target-os=linux --enable-pthreads --enable-openssl --enable-hardcoded-tables
libavutil      56. 62.100 / 56. 62.100
libavcodec     58.115.102 / 58.115.102
libavformat    58. 65.100 / 58. 65.100
libavdevice    58. 11.103 / 58. 11.103
libavfilter     7. 94.100 /  7. 94.100
libswscale      5.  8.100 /  5.  8.100
libswresample   3.  8.100 /  3.  8.100
libpostproc    55.  8.100 / 55.  8.100

ffmpeg - filters shows tpad being available.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

I'm pretty well there - doing some final testing now.

I'm PRETTY sure that the standard Pi ffmpeg does not have tpad - but you can test with the following command to see (required a different syntax than I originally tried) .... I updated ffmpeg before I found this working syntax ....

I'll try this tomorrow (US Central time - GMT-6).

stuartofmt

@JohnOCFII

Updated here:

https://github.com/stuartofmt/DuetLapse

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

Updated here:

https://github.com/stuartofmt/DuetLapse

Greetings!

I was hoping that simply replacing ffmpeg and the new DuetLapse.py would "just work," but sadly, not the case.

I confirmed that my previous ffmpeg didn't have support for the pad filter, as you suggested.

I rebuilt ffmpeg according to your instructions. That all went well.

I then tried to execute the DuetLapse script but you addition of -psutil- wasn't working, so I Googled how to build that, and did so.

I then went to run DuetLapse, but received the message:

Process is already running...

Probably because I have DuetLapse starting from -systemd-

I'll have to see what else is different.

John

stuartofmt

@JohnOCFII

If you are using my DuetLapse.py then it will prevent starting twice - so you need to make sure it's stopped before starting it again
kill -s <pid> if you are running in background else CTL+C works.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

If you are using my DuetLapse.py then it will prevent starting twice - so you need to make sure it's stopped before starting it again
kill -s <pid> if you are running in background else CTL+C works.

Heh - yeah, I might switch back to my DuetLapse, but take your ffmpeg command line. I have mine set to run and restart continuously, that way, I'll get a timelapse created every time a print starts.

John

stuartofmt

@JohnOCFII

If you are using systemd and it’s set up correctly then you should be fine with my version. All my code does on startup is look to see if there is an instance already running and if so - not create an additional instance (which would case “confusion”). Similarly systemd should not try to start a new instance if there is one already running (if you have it set up that way).
Bottom line, they should be compatible with each other.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

If you are using systemd and it’s set up correctly then you should be fine with my version. All my code does on startup is look to see if there is an instance already running and if so - not create an additional instance (which would case “confusion”). Similarly systemd should not try to start a new instance if there is one already running (if you have it set up that way).
Bottom line, they should be compatible with each other.

Yes, you are correct -- but I also have a little test script that attempts to launch an additional copy of DuetLapse that I use when testing new features. And it was attempting to launch THAT that caused my issues.

So, yes, I stuck with your version but did comment out the process related bits for testing.

stuartofmt

I’ll add in a switch so you can control with / without the check. May be useful ...

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

I’ll add in a switch so you can control with / without the check. May be useful ...

Lovely idea!

JohnOCFII

@stuartofmt My short test runs succeed, but longer runs fail. Looking at syslog, the output from ffmpeg is a bit hard to follow. I'm curious if you've tried any tests with more than 50 layers.

I've attached a trimmed section of my Raspberry Pi syslog in case you are interested.

There is one point (line 73) where it mentioned it can't find the file that (I think) it should be writing? Then perhaps it gets confused after that... Not sure if my use of a subdirectory is causing an issue there -- I'm taking advantage of one of your new flags.

DLVideos/DuetLapse1227201756.mp4: No such file or directory

This is the command line I'm using to launch DuetLapse:

/home/pi/DuetLapse/DuetLapse.py -duet railcore.localdomain -camera web -weburl http://octocore.localdomain:8081/snapshot -detect layer -basedir DLVideos -extratime 4

Thanks!

John
faillog.txt [0_1609120917817_faillog](Uploading 100%)

stuartofmt

@JohnOCFII

I'll take a look in the morning but -- the basedir should be an absolute path without the trailing /

e.g /home/pi/DLVideos

stuartofmt

Made some more changes:

Here https://github.com/stuartofmt/DuetLapse

Changed from print commands to logging option -logtype [console, file, both] default is both. If -logtype console then only written to the console. If -logtype file then a logfile is written to basedir with the name DuelLapse.log If -logtype both then written to the console and the logfile. Messages are prefixed by the ip address set by -duet. This is to distinguish between printers if multiple are used (see also -instances (below)

added a switch for inhibiting multiple instances -instances [single, oneip, many] default is single. If -instances single then the highlander principle applies. If -instances oneip then only one instance per duet ip address (set by -duet ip address). If -instances many then it's the wild west.

stuartofmt

@JohnOCFII

I took a look at the log - I think it's just the path. So try it with -basedir set as an full path i.e. with leading / and no trailing / as I suggested above.

Also - as a side note - I saw your systemctl restarting the script. I'm not entirely sure why it would not have worked with the single instance being enforced but I changed the exit(x) commands (apparently more applicable for interactive use) to sys.exit(x). The documentation suggests the latter is better form ...

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

I took a look at the log - I think it's just the path. So try it with -basedir set as an full path i.e. with leading / and no trailing / as I suggested above.

Also - as a side note - I saw your systemctl restarting the script. I'm not entirely sure why it would not have worked with the single instance being enforced but I changed the exit(x) commands (apparently more applicable for interactive use) to sys.exit(x). The documentation suggests the latter is better form ...

I'll give the new version a shot. I'll change -basedir to /home/pi/DLVideos

Thanks!

stuartofmt

@JohnOCFII

I just tried with 264 images - no problems creating the Video file.

JohnOCFII

@stuartofmt said in DuetLapse available for Alpha testing:

@JohnOCFII

I just tried with 264 images - no problems creating the Video file.

Correcting the -basedir to full path fixed my issues. The "held frame" is working great.

Not the fanciest timelapse, but shows the hold: https://youtu.be/LW6hV-MECns

Thanks!

stuartofmt

@JohnOCFII

Looks like you have a nice setup.

I'm looking at a couple of other tweaks (e.g. updating the Duet API calls to the newest style) and making the package a single python script rather than two. I am looking at incorporate a capability or two from the other forks.

When that is done - I think we can start another thread without the "alpha" description (DuetLapse2 ?). I suspect the "alpha' bit turns a few folks off.

Will post the changes when I am done.

P.S. Unrelated and from a small side project - I noticed that the mp4 output from ffmpeg (depending on settings) may not be compatible with iphone/ipad e.g if sent as a text or email attachment. Not sure if the output from DueatLapse has the same problem but will check. If it does I'll tweak accordingly.

PCR

@stuartofmt please add this https://github.com/jaysuk/DuetWebAPI to the Readme. The old one of Danal does not work anymore with Duet 3 + SBC

stuartofmt

@PCR

In the next few days (maybe a week) I will be working on this comment "I'm looking at a couple of other tweaks (e.g. updating the Duet API calls to the newest style) and making the package a single python script rather than two"

Specifically - updating the API calls to the newest Duet standard and doing away with the separate DuetWebAPI. It (the separate API has unneeded calls etc. and use the old API versions).

When done - it should be good to go for Duet # + SBC. As they say in the classics -- "Stay tuned"