DuetLapse available for Alpha testing

arhi

hm and just in case wget messed something up .. git clone of both repos

root@octopi:~/DuetLapse/DuetLapse# PYTHONPATH=/root/DuetLapse/DuetWebAPI python3 DuetLapse.py  -h
Traceback (most recent call last):
  File "DuetLapse.py", line 21, in <module>
    import DuetWebAPI as DWA
  File "/root/DuetLapse/DuetWebAPI/DuetWebAPI.py", line 27
    URL=(f'{self._base_url}'+'/rr_status?type=1')
                           ^
SyntaxError: invalid syntax
root@octopi:~/DuetLapse/DuetLapse#

I don't get it, I don't see that this line changed since forever?!

looks like F-Strings are in from 3.6 only raspbian on pi3b+ latest comes with 3.5.3

Danal

@arhi said in DuetLapse available for Alpha testing:

URL=(f'{self._base_url}'+'/rr_status?type=1')

@arhi Very interesting, I don't get that, and that line is not a syntax error. Hmmm... Let me think of a way to pin this down.

arhi

@Danal said in DuetLapse available for Alpha testing:

@arhi said in DuetLapse available for Alpha testing:

URL=(f'{self._base_url}'+'/rr_status?type=1')

@arhi Very interesting, I don't get that, and that line is not a syntax error. Hmmm... Let me think of a way to pin this down.

F-strings are 3.6 python from what I read online, and stretch has max python 3.5.3 ... I'm now going to move to buster and see if it uses newer python

Danal

@arhi said in DuetLapse available for Alpha testing:

@Danal said in DuetLapse available for Alpha testing:

@arhi said in DuetLapse available for Alpha testing:

URL=(f'{self._base_url}'+'/rr_status?type=1')

@arhi Very interesting, I don't get that, and that line is not a syntax error. Hmmm... Let me think of a way to pin this down.

F-strings are 3.6 python from what I read online, and stretch has max python 3.5.3 ... I'm now going to move to buster and see if it uses newer python

Excellent detective work. I had no idea f was that recent.

arhi

@Danal I don't use python at all so I don't have anything but what's normally there with the distro... so, troubles with buster too it comes with newer python but again

root@octopi:~/DuetLapse/DuetLapse# apt-get install python3-numpy python-numpy
Reading package lists... Done
Building dependency tree
Reading state information... Done
python-numpy is already the newest version (1:1.16.2-1).
python3-numpy is already the newest version (1:1.16.2-1).
root@octopi:~/DuetLapse/DuetLapse# python3 DuetLapse.py -h
Python Library Module 'numpy' is required.
Obtain via 'sudo python3 -m pip install numpy'
root@octopi:~/DuetLapse/DuetLapse# python3 DuetLapse.py
Python Library Module 'numpy' is required.
Obtain via 'sudo python3 -m pip install numpy'
root@octopi:~/DuetLapse/DuetLapse# python3 DuetLapse.py -h
Python Library Module 'numpy' is required.
Obtain via 'sudo python3 -m pip install numpy'
root@octopi:~/DuetLapse/DuetLapse# python3 -m pip install numpy
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (1.18.4)
root@octopi:~/DuetLapse/DuetLapse# pip3 install numpy
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (1.18.4)
root@octopi:~/DuetLapse/DuetLapse# python3 DuetLapse.py -h
Python Library Module 'numpy' is required.
Obtain via 'sudo python3 -m pip install numpy'
root@octopi:~/DuetLapse/DuetLapse# python3 --version
Python 3.7.3
root@octopi:~/DuetLapse/DuetLapse#

root@octopi:~# python3
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> import sys
>>> import argparse
>>> import time
>>> import numpy as np
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/numpy/core/__init__.py", line 24, in <module>
    from . import multiarray
  File "/usr/local/lib/python3.7/dist-packages/numpy/core/multiarray.py", line 14, in <module>
    from . import overrides
  File "/usr/local/lib/python3.7/dist-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: libf77blas.so.3: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/usr/local/lib/python3.7/dist-packages/numpy/core/__init__.py", line 50, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.7 from "/usr/bin/python3"
  * The NumPy version is: "1.18.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libf77blas.so.3: cannot open shared object file: No such file or directory

>>>

I love how consistent APT system is

arhi

apt-get install libatlas-base-dev

solved the numpy error

arhi

one hint, I don't know python but might be cool to add argument to DuetLapse to point to DuetWebAPI library?

I added directly to code

diff --git a/DuetLapse.py b/DuetLapse.py
index eee4a51..5119d61 100755
--- a/DuetLapse.py
+++ b/DuetLapse.py
@@ -17,6 +17,9 @@ import subprocess
 import sys
 import argparse
 import time
+
+sys.path.insert(1, "../DuetWebAPI/");
+
 try:
     import DuetWebAPI as DWA
 except ImportError:

but would be cool to read that from the argument line so we can normally git pull changes and not need to copy stuff over each other especially as python is ignoring symlinks

arhi

@Danal you tested this with 3.01RC12? and DWC 2.1.7?

root@octopi:~/DuetLapse/DuetLapse# cd ../DuetWebAPI/
root@octopi:~/DuetLapse/DuetWebAPI# git pull
Already up to date.
root@octopi:~/DuetLapse/DuetWebAPI# cd -
/root/DuetLapse/DuetLapse
root@octopi:~/DuetLapse/DuetLapse# python3 DuetLapse.py -duet ender5.local.lan -camera usb -detect layer -pause no
Attempting to connect to printer at ender5.local.lan
http://ender5.local.lan  does not appear to be a RRF2 or RRF3 printer
Device at ender5.local.lan either did not respond or is not a Duet V2 or V3 printer.
root@octopi:~/DuetLapse/DuetLapse# fping ender5.local.lan
ender5.local.lan is alive
root@octopi:~/DuetLapse/DuetLapse#

root@octopi:~/DuetLapse/DuetLapse# telnet ender5.local.lan
Trying 192.168.89.246...
Connected to ender5.local.lan.
Escape character is '^]'.
M122
=== Diagnostics ===
RepRapFirmware for Duet 2 WiFi/Ethernet version 3.01-RC12 running on Duet Ethernet 1.02 or later
...
telnet> c
Connection closed.
root@octopi:~/DuetLapse/DuetLapse#

arhi

this last part was on me - works ok now

arhi

@Danal I don't know python so this should probbly be done better by somehow defining parameter to onePhoto() not mandatory, or if there's already a global with this info but this works awesome for me so if you like you integrate

root@octopi:~/DuetLapse/DuetLapse# git diff
diff --git a/DuetLapse.py b/DuetLapse.py
old mode 100755
new mode 100644
index eee4a51..1539257
--- a/DuetLapse.py
+++ b/DuetLapse.py
@@ -17,6 +17,9 @@ import subprocess
 import sys
 import argparse
 import time
+
+sys.path.insert(1, "../DuetWebAPI/");
+
 try:
     import DuetWebAPI as DWA
 except ImportError:
@@ -204,11 +207,15 @@ def unPause():
         printer.gCode('M24')
         alreadyPaused = False

-def onePhoto():
+def onePhoto(zl):
     global frame
     frame += 1
     s="{0:08d}".format(int(np.around(frame)))
-    fn = '/tmp/DuetLapse/IMG'+s+'.jpeg'
+    if (zl > -1):
+      zlayer="{0:8d}".format(int(zl))
+      fn = '--title "Layer '+zlayer+'" /tmp/DuetLapse/IMG'+s+'.jpeg'
+    else:
+      fn = ' /tmp/DuetLapse/IMG'+s+'.jpeg'

     if ('usb' in camera):
         if (camparms == ''):
@@ -226,6 +233,7 @@ def onePhoto():
         else:
             cmd = 'wget '+camparms+' -O '+fn+' "'+weburl+'" '

+    print(cmd)
     subprocess.call(cmd, shell=True)
     global timePriorPhoto
     timePriorPhoto = time.time()
@@ -240,20 +248,20 @@ def oneInterval():
             # Z changed, take a picture.
             checkForcePause()
             print('Capturing frame {0:5d} at X{1:4.2f} Y{2:4.2f} Z{3:4.2f} Layer {4:d}'.format(int(np.around(frame)),printer.getCoords()['X'],printer.getCoords()['Y'],printer.getCoords()['Z'],zn))
-            onePhoto()
+            onePhoto(zn)
         zo = zn
     global timePriorPhoto
     elap = (time.time() - timePriorPhoto)
     if ((seconds) and (seconds < elap)):
         checkForcePause()
         print('Capturing frame {0:5d} after {1:4.2f} seconds elapsed.'.format(int(np.around(frame)),elap))
-        onePhoto()
+        onePhoto( -1 )
     if ('pause' in detect):
         if ('paused' in printer.getStatus()):
             global alreadyPaused
             alreadyPaused = True
             print('Pause Detected, capturing frame {0:5d}'.format(int(np.around(frame)),elap))
-            onePhoto()
+            onePhoto( -1)
         unPause()

 def postProcess():
root@octopi:~/DuetLapse/DuetLapse#

(print(cmd) was just me debuging, that's not needed)

Danal

Python honors symlinks, that is exactly how I'm keeping one central copy of DuetWebAPI and not copying it around to each DuetLapse, TAMV, etc, etc. as I develop this stuff. If you want it not in the current directory, and/or don't want to use symlinks, then properly put it in Python's path please.

By the way, git clone will NOT update things, 'wget' has all kinds of edge cases, I haven't found a really bullet proof way to distribute scripts from github (besides set up a package server. Which I am reluctant to do, and even that requires entries in /etc directories on user endpoints).

Anyway, symlinks work fine, I'm curious why yours don't.

ln -s ../DuetWebAPI/DuetWebAPI.py DuetWebAPI.py right?

Screenshot to get colors:

arhi

@Danal said in DuetLapse available for Alpha testing:

Python honors symlinks, that is exactly how I'm keeping one central copy of DuetWebAPI and not copying it around to each DuetLapse, TAMV, etc, etc. as I develop this stuff. If you want it not in the current directory,

kill me if I know why but it didn't work .. I'll try again later but this work as is now so ...

and/or don't want to use symlinks, then properly put it in Python's path please.

Need to try that again, when I did with PYTHONPATH=/root/DuetLapse/DuetWebAPI python3 DuetLapse.py it was finding api ok but failed something else, but maybe now that I fixed the other thing that will work, will try

By the way, git clone will NOT update things

of course, git pull does ( git checkout or stash first if files are modified like they are now)

'wget' has all kinds of edge cases, I haven't found a really bullet proof way to distribute scripts from github (besides set up a package server. Which I am reluctant to do, and even that requires entries in /etc directories on user endpoints).

git clone
git pull
git checkout/stash

works awesome, the only issue I need to handle is this path for libs so it knows how to find api, will retest this but even as is it's not a biggy

hahaha retested the symthing now, it works, the %^$#*&^#

PYTHONPATH=/root/DuetLapse/DuetWebAPI python3 DuetLapse.py -h works too, I'll use this way .. the problem was obviously the numpy or whatever is that thing called...

anyhow, these are irrelevant parts of that diff .. what about that change in the onePhoto() ? makes sense?

arhi

Danal

@arhi said in DuetLapse available for Alpha testing:

hahaha retested the symthing now, it works, the %^$#*&^#

Don't you just love things like that!

@arhi said in DuetLapse available for Alpha testing:

anyhow, these are irrelevant parts of that diff .. what about that change in the onePhoto() ? makes sense?

Makes perfect sense, I just need time to look at it... probably next day or two.

arhi

@Danal said in DuetLapse available for Alpha testing:

Don't you just love things like that!

I'm not 40 any more, doing things between 3 and 5am obviously need to be doublechecked

Danal

@Adrian52 said in DuetLapse available for Alpha testing:

Great program, but I still have not got it quite right. I am currently using. /DuetLapse - duet 192.168.1.16 -camera pi -detect pause, with M25 and M400 added to each layer in gcode (easy to do in kisslicer). The problem is that two frames are taken, one when the head is moved away by pause.g, the second a bit later, when the print has resumed. I also get an error message on the console 'm24: cannot print because no file is selected'. It looks as though the m24 is being sent twice.
I have edited my pause.g so that there is no z shift.

Just pushed changes to the detect pause logic that should eliminate this. It has been tested to ensure no syntax errors, but has not been run through a print.

Adrian52

@Danal here is my first go

https://vimeo.com/415969620
Worked fine with M226 in the gcode file. Excellent. Will try detecting layer changes later. Great work.

phdrumlineka

I was attempting to run this on the same PI as my OctoPrint setup. I kept getting errors saying numpy wasn't installed. After attempting the suggested command to get numpy installed, it claimed that pip wasn't installed for python3. After a lot of googling I got pip and numpy installed, but I was still getting the same error saying numpy was required.

I finally added some additional output to DuetLapse.py file and found that I was getting the following error:
Original error was: libf77blas.so.3: cannot open shared object file: No such file or directory
It looks like the answer to that error was to run
sudo apt-get install libatlas-base-dev

I just thought I would post this here in case anyone else runs into this issue, or if anyone has a better way to solve it. I am novice with Linux and Python, so this may not be the best way to handle things.

I am using the following versions of OctoPrint/OctoPi:
OctoPrint version: 1.4.0
OctoPi version: 0.17.0

Danal

Interesting.

I developed the script on the "Duet" provided image for a Pi, and I'm sure that has things installed that were transparent to me. I've tried to have the script produce meaningful errors when something is missing, and I will need to do that for Numpy. Also, I will look into the libatals-base-dev.

pip is not needed by the script at runtime; it is needed to install things the script needs. As you say, python3 does not install pip by default.

A lot of instructions show how to "bootstrap" pip install via python itself. The actual proper way to install pip on a Python3 (or mixed) system is sudo apt install python-pip. This would (should?) get the libatlas dependence automatically.

I've updated the script to show the required commands, if numpy is not found.

Thank you.

phdrumlineka

@Danal
I am pretty sure that is not how I did it . Good to know. Thanks for looking into that. Hopefully it will help others in the future.

I would also like to mention that it is working now, happily taking pictures of my current print. Good work. I appreciate it.