Duet3D Logo Duet3D
    • Tags
    • Documentation
    • Order
    • Register
    • Login

    DuetLapse3

    Scheduled Pinned Locked Moved
    Third-party software
    20
    296
    30.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • fractalengineerundefined
      fractalengineer @stuartofmt
      last edited by

      @stuartofmt Thanks Stuart; I did try installing the 3.04 earlier this weekend following each step but got stuck at ffmpeg install; doesn't return anything when checking version.

      I guess I'm still way to beginner at linux terminal. I'll try again with the new version

      Railcore II ZL

      stuartofmtundefined 1 Reply Last reply Reply Quote 0
      • stuartofmtundefined
        stuartofmt @fractalengineer
        last edited by

        @fractalengineer said in DuetLapse3:

        @stuartofmt Thanks Stuart; I did try installing the 3.04 earlier this weekend following each step but got stuck at ffmpeg install; doesn't return anything when checking version.

        I guess I'm still way to beginner at linux terminal. I'll try again with the new version

        Don't worry - we will get you going.
        I'm assuming you are using a raspberry pi ?

        The latest version should work with the version of ffmpeg that installs by default with the pi. The only downside is not being able to use the -extratime option.

        I'd recommend you reinstall the default ffmpeg and take it from there.
        It sounds like you tried to follow my instructions and the installation of the latest ffmpeg failed somehow? If that is correct then lets try to clean it up and get you back to the default. Try the following in order (I just did this on my pi) 🙂

        cd ~
        sudo apt remove ffmpeg
        

        I'm guessing this will say ffmpeg is not installed.

        Next:

        sudo find / -name ffmpeg
        

        This will may give some output for example:)
        find: ‘/run/user/1000/gvfs’: Permission denied
        /usr/local/bin/ffmpeg
        /usr/local/share/ffmpeg
        /usr/local/share/doc/ffmpeg

        for each of the files / directories - we'll first delete everything for EXAMPLE

        sudo rm -rf /usr/local/bin/ffmpeg
        sudo rm -rf /usr/local/share/ffmpeg
        sudo rm -rf /usr/local/share/doc/ffmpeg
        

        Next make sure everything is at the latest release level

        sudo apt update
        sudo apt upgrade
        

        I just did this on my pi - it was quite the upgrade and took quite a while so be patient.

        Finally, install the standard ffmpeg

        sudo apt install ffmpeg
        

        after that and assuming no errors

        ffmpeg -version
        

        should return a bunch of information.

        Then reboot for luck 🙂

        fractalengineerundefined 1 Reply Last reply Reply Quote 0
        • fractalengineerundefined
          fractalengineer @stuartofmt
          last edited by

          @stuartofmt aww mate you're a saint; didn't expect such a detailed help.

          I went through those steps and when doing the find it didn't return anything; so I guess it didn't build properly in the first place (I used the cmake last time)

          Seems like everything is installed now; just got to figure out the usage exactly

          Thanks!

          Railcore II ZL

          1 Reply Last reply Reply Quote 0
          • SIamundefined
            SIam
            last edited by

            Hi,

            @stuartofmt I have a Problem with layer capturing if i use this nothing will be captured i use the following command to start it

            python3 DuetLapse3.py -port 8181 -duet 192.168.188.36 -detect layer -camera1 stream -weburl1 http://192.168.188.34:8081/
            

            This result in

            192.168.188.36 ################### Options selected for this run #####################
            192.168.188.36 #Environment Settings:
            192.168.188.36 # printer         = 192.168.188.36
            192.168.188.36 # basedir         = /home/pi/DuetLapse
            192.168.188.36 # poll            = 5
            192.168.188.36 # logtype         = both
            192.168.188.36 # verbose         = False
            192.168.188.36 # os              = Linux
            192.168.188.36 # host            = 0.0.0.0
            192.168.188.36 # port            = 8181
            192.168.188.36 # pid            = 7902
            192.168.188.36 #Execution Setings:
            192.168.188.36 # dontwait        = False
            192.168.188.36 # seconds         = 0
            192.168.188.36 # detect          = layer
            192.168.188.36 # pause           = no
            192.168.188.36 # standby     = False
            192.168.188.36 #Camera1 Settings:
            192.168.188.36 # camera1         = stream
            192.168.188.36 # weburl1         = http://192.168.188.34:8081/
            192.168.188.36 # Video Settings:
            192.168.188.36 # extratime   =     0
            192.168.188.36 ###################################################################
            192.168.188.36
            192.168.188.36
            192.168.188.36
            192.168.188.36 ###############################################################
            192.168.188.36 Connected to printer at 192.168.188.36 using version 3.2 and API access using rr_model
            192.168.188.36 ###############################################################
            192.168.188.36
            192.168.188.36
            192.168.188.36 ##########################################################
            192.168.188.36 ***** Started http listener *****
            192.168.188.36 ##########################################################
            192.168.188.36
            192.168.188.36
            192.168.188.36 ##########################################################
            192.168.188.36 Will start capturing images on first layer change
            192.168.188.36 ##########################################################
            192.168.188.36
            192.168.188.36
            192.168.188.36 ##########################################################
            192.168.188.36 Video will be created when printing ends.
            192.168.188.36 Or, press Ctrl+C one time to stop capture and create video.
            192.168.188.36 ##########################################################
            192.168.188.36
            192.168.188.36 ****** Printer State changed to waiting *****
            192.168.188.36 ****** Printer State changed to printing *****
            192.168.188.36 ****** Printer State changed to completed *****
            192.168.188.36 End of Print Job - Post Processing
            192.168.188.36 ++++++ Entering restart state ++++++
            192.168.188.36
            192.168.188.36 Camera1: Cannot create video with only 0 frames
            192.168.188.36 ****** Printer State changed to waiting *****
            
            

            If i use the Time capturing option it works as it should

            /DuetLapse $ python3 DuetLapse3.py -port 8181 -duet 192.168.188.36 -seconds 20 -detect none -camera1 stream -weburl1 http://192.168.188.34:8081/
            ..
            ..
            ..
            192.168.188.36 ##########################################################
            192.168.188.36 Video will be created when printing ends.
            192.168.188.36 Or, press Ctrl+C one time to stop capture and create video.
            192.168.188.36 ##########################################################
            192.168.188.36
            192.168.188.36 ****** Printer State changed to waiting *****
            192.168.188.36 ****** Printer State changed to printing *****
            192.168.188.36 ****** Printer State changed to startnow *****
            192.168.188.36 Camera1: capturing frame 0 at layer 72 after 20.0 seconds
            192.168.188.36 ****** Printer State changed to printing *****
            192.168.188.36 Camera1: capturing frame 1 at layer 73 after 20.0 seconds
            192.168.188.36 Camera1: capturing frame 2 at layer 74 after 20.0 seconds
            192.168.188.36 Camera1: capturing frame 3 at layer 75 after 20.0 seconds
            192.168.188.36 Camera1: capturing frame 4 at layer 76 after 20.0 seconds
            192.168.188.36 Camera1: capturing frame 5 at layer 78 after 20.0 seconds
            192.168.188.36 Camera1: capturing frame 6 at layer 79 after 20.0 seconds
            192.168.188.36 Camera1: capturing frame 7 at layer 80 after 20.0 seconds
            ^C192.168.188.36 !!!!!! Stopped by SIGINT - Post Processing !!!!!!
            192.168.188.36
            192.168.188.36 Camera1: Cannot create video with only 8 frames
            192.168.188.36 Waiting for http listener to shutdown.
            192.168.188.36 Program Terminated
            
            

            The printer is on the latest stable:

            RepRapFirmware for Duet 2 WiFi/Ethernet version 3.2 running on Duet WiFi 1.02 or later + DueX5
            

            Can you please tell me what i did wrong?

            Hypercube-Evolution-Hybrid, Piezo Orion, Orbiter
            Duet WiFi 1.02 or later + DueX5
            RepRapFirmware for Duet 2 WiFi/Ethernet 3.4.0beta4 (2021-09-27 11:30:36)
            Duet WiFi Server: 1.26
            Duet Web Control 3.4.0beta4 (2021-09-27)

            1 Reply Last reply Reply Quote 0
            • SIamundefined
              SIam
              last edited by

              my knowledge of python is not really available but when i look at this code:

               #update the layer counter
                  global zo1, zo2
                  zn = getDuetLayer(apiModel)
                  if (cameraname == 'Camera1'):
                      zo1 = zn
                  else:
                      zo2 = zn
                  
                  if ('layer' in detect):
                      if ((not zn == zo1 and cameraname == 'Camera1') or (not zn == zo2 and cameraname == 'Camera2')):
                          # Layer changed, take a picture.
                          checkForPause()
                          logger.info(cameraname+': capturing frame '+str(frame)+' at layer '+str(zn)+' after layer change')           
                          onePhoto(cameraname, camera, weburl, camparam)            
              
                  elif ('pause' in detect and printerState == 'paused'):
                          checkForPause()
                          logger.info(cameraname+': capturing frame '+str(frame)+' at layer '+str(zn)+' at pause in print gcode')
                          onePhoto(cameraname, camera, weburl, camparam)
              

              So this script can't access to the take picture routine because zo1 and zn is always the same

              I have now altered the code to:

                  #update the layer counter
                  global zo1, zo2
                  zn = getDuetLayer(apiModel)
                 # if (cameraname == 'Camera1'):
                 #     zo1 = zn
                 # else:
                 #     zo2 = zn
              
                  if ('layer' in detect):
                      if ((not zn == zo1 and cameraname == 'Camera1') or (not zn == zo2 and cameraname == 'Camera2')):
                          # Layer changed, take a picture.
                          checkForPause()
                          logger.info(cameraname+': capturing frame '+str(frame)+' at layer '+str(zn)+' after layer change')
                          onePhoto(cameraname, camera, weburl, camparam)
              
                  elif ('pause' in detect and printerState == 'paused'):
                          checkForPause()
                          logger.info(cameraname+': capturing frame '+str(frame)+' at layer '+str(zn)+' at pause in print gcode')
                          onePhoto(cameraname, camera, weburl, camparam)
              
                 # global zo1, zo2
                 # zn = getDuetLayer(apiModel)
                  if (cameraname == 'Camera1'):
                      zo1 = zn
                  else:
                      zo2 = zn
              
              

              And for now it works (but i don't know if i have now added a new failure)

              Hypercube-Evolution-Hybrid, Piezo Orion, Orbiter
              Duet WiFi 1.02 or later + DueX5
              RepRapFirmware for Duet 2 WiFi/Ethernet 3.4.0beta4 (2021-09-27 11:30:36)
              Duet WiFi Server: 1.26
              Duet Web Control 3.4.0beta4 (2021-09-27)

              stuartofmtundefined 1 Reply Last reply Reply Quote 0
              • stuartofmtundefined
                stuartofmt @SIam
                last edited by

                @SIam

                Hi Sam - sorry you are having problems. The changes you made to the code will most definitely cause problems 😞 That piece of code keeps track of layer changes.

                Having said that it was my error ...

                But first - I see you have started DuetLapse with:

                python3 DuetLapse3.py -port 8181 -duet 192.168.188.36 -detect layer -camera1 stream -weburl1 http://192.168.188.34:8081/
                

                You have BOTH your web camera (-weburl1) AND DuetLapse3 on the same port number. This is a no-no. They need to be different. In the next release (I am testing now) - I check to see if the port is already in use and if it is, prevent DuetLapse3 from starting. So please change the -port so that it is on a different and unused port number.

                Second - on the layer change - yikes! This is definitely wrong and thanks for pointing it out. I must have copy/pasted that portion into the wrong place when I did some 'tidy up' . I will fix that today. If you want to fix it yourself - move the assignment so that it looks like this:

                    #update the layer counter
                    global zo1, zo2
                    zn = getDuetLayer(apiModel)
                
                    
                    if ('layer' in detect):
                        if ((not zn == zo1 and cameraname == 'Camera1') or (not zn == zo2 and cameraname == 'Camera2')):
                            # Layer changed, take a picture.
                            checkForPause()
                            logger.info(cameraname+': capturing frame '+str(frame)+' at layer '+str(zn)+' after layer change')           
                            onePhoto(cameraname, camera, weburl, camparam)            
                
                    elif ('pause' in detect and duetStatus == 'paused'):
                            checkForPause()
                            logger.info(cameraname+': capturing frame '+str(frame)+' at layer '+str(zn)+' at pause in print gcode')
                            onePhoto(cameraname, camera, weburl, camparam)
                            
                    if (cameraname == 'Camera1'):
                        zo1 = zn
                    else:
                        zo2 = zn 
                
                

                In any case, the change to -seconds 20 -detect none is working as it should but that is bypassing the layer check so the change you made has no effect.

                1 Reply Last reply Reply Quote 0
                • SIamundefined
                  SIam
                  last edited by

                  Thank you for your assistance!

                  My start command is correct duetlapse runs on 8181 and the cam on 8081 😉

                  to your code change the only change i can see is that you have removed the comments is this right ?

                  Hypercube-Evolution-Hybrid, Piezo Orion, Orbiter
                  Duet WiFi 1.02 or later + DueX5
                  RepRapFirmware for Duet 2 WiFi/Ethernet 3.4.0beta4 (2021-09-27 11:30:36)
                  Duet WiFi Server: 1.26
                  Duet Web Control 3.4.0beta4 (2021-09-27)

                  stuartofmtundefined 1 Reply Last reply Reply Quote 0
                  • stuartofmtundefined
                    stuartofmt @SIam
                    last edited by

                    @SIam said in DuetLapse3:

                    Thank you for your assistance!

                    My start command is correct duetlapse runs on 8181 and the cam on 8081 😉

                    to your code change the only change i can see is that you have removed the comments is this right ?

                    Sorry - I misread the port numbers , you are of course correct.

                    As to the question on the code change - no.

                    This portion of code

                        if (cameraname == 'Camera1'):
                            zo1 = zn
                        else:
                            zo2 = zn
                    

                    needs to be after this portion (not before - which was the problem )

                        if ('layer' in detect):
                            if ((not zn == zo1 and cameraname == 'Camera1') or (not zn == zo2 and cameraname == 'Camera2')):
                                # Layer changed, take a picture.
                                checkForPause()
                                logger.info(cameraname+': capturing frame '+str(frame)+' at layer '+str(zn)+' after layer change')           
                                onePhoto(cameraname, camera, weburl, camparam)            
                     
                        elif ('pause' in detect and duetStatus == 'paused'):
                                checkForPause()
                                logger.info(cameraname+': capturing frame '+str(frame)+' at layer '+str(zn)+' at pause in print gcode')
                                onePhoto(cameraname, camera, weburl, camparam)
                    

                    NOTE THAT THE INDENTING HAS TO BE EXACTLY THE SAME - Python is very particular in that regard - so be careful as you edit 🙂

                    1 Reply Last reply Reply Quote 0
                    • SIamundefined
                      SIam
                      last edited by

                      ok! Thank you for your work i love it 😄

                      Hypercube-Evolution-Hybrid, Piezo Orion, Orbiter
                      Duet WiFi 1.02 or later + DueX5
                      RepRapFirmware for Duet 2 WiFi/Ethernet 3.4.0beta4 (2021-09-27 11:30:36)
                      Duet WiFi Server: 1.26
                      Duet Web Control 3.4.0beta4 (2021-09-27)

                      1 Reply Last reply Reply Quote 0
                      • SIamundefined
                        SIam
                        last edited by

                        is it possible to catch those errors in python ?

                        Traceback (most recent call last):
                          File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
                            chunked=chunked)
                          File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 384, in _make_request
                            six.raise_from(e, None)
                          File "<string>", line 3, in raise_from
                          File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 380, in _make_request
                            httplib_response = conn.getresponse()
                          File "/usr/lib/python3.7/http/client.py", line 1336, in getresponse
                            response.begin()
                          File "/usr/lib/python3.7/http/client.py", line 306, in begin
                            version, status, reason = self._read_status()
                          File "/usr/lib/python3.7/http/client.py", line 267, in _read_status
                            line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
                          File "/usr/lib/python3.7/socket.py", line 589, in readinto
                            return self._sock.recv_into(b)
                        ConnectionResetError: [Errno 104] Connection reset by peer
                        
                        During handling of the above exception, another exception occurred:
                        
                        Traceback (most recent call last):
                          File "/usr/lib/python3/dist-packages/requests/adapters.py", line 449, in send
                            timeout=timeout
                          File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
                            _stacktrace=sys.exc_info()[2])
                          File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 367, in increment
                            raise six.reraise(type(error), error, _stacktrace)
                          File "/usr/lib/python3/dist-packages/six.py", line 692, in reraise
                            raise value.with_traceback(tb)
                          File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
                            chunked=chunked)
                          File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 384, in _make_request
                            six.raise_from(e, None)
                          File "<string>", line 3, in raise_from
                          File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 380, in _make_request
                            httplib_response = conn.getresponse()
                          File "/usr/lib/python3.7/http/client.py", line 1336, in getresponse
                            response.begin()
                          File "/usr/lib/python3.7/http/client.py", line 306, in begin
                            version, status, reason = self._read_status()
                          File "/usr/lib/python3.7/http/client.py", line 267, in _read_status
                            line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
                          File "/usr/lib/python3.7/socket.py", line 589, in readinto
                            return self._sock.recv_into(b)
                        urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
                        
                        During handling of the above exception, another exception occurred:
                        
                        Traceback (most recent call last):
                          File "/home/pi/DuetLapse/DuetLapse3.py", line 964, in <module>
                            duetStatus=getDuetStatus(apiModel)
                          File "/home/pi/DuetLapse/DuetLapse3.py", line 694, in getDuetStatus
                            r = requests.get(URL, timeout=120)  #Long timeout to handle restart of Duet
                          File "/usr/lib/python3/dist-packages/requests/api.py", line 75, in get
                            return request('get', url, params=params, **kwargs)
                          File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
                            return session.request(method=method, url=url, **kwargs)
                          File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
                            resp = self.send(prep, **send_kwargs)
                          File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
                            r = adapter.send(request, **kwargs)
                          File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
                            raise ConnectionError(err, request=request)
                        requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
                        ^C
                        pi@raspberrypi:~/DuetLapse $ ./start-lapse stop
                        192.168.188.36 !!!!!! Stopped by SIGINT - Post Processing !!!!!!
                        192.168.188.36
                        pi@raspberrypi:~/DuetLapse $ 192.168.188.36 Camera1: now making 29 frames into a video
                        192.168.188.36 Video processing complete for Camera1
                        192.168.188.36 Video is in file "/home/pi/DuetLapse/192-168-188-36/Camera17144-Mon-11-25.mp4"
                        192.168.188.36 Waiting for http listener to shutdown.
                        192.168.188.36 Program Terminated
                        Exception ignored in: <module 'threading' from '/usr/lib/python3.7/threading.py'>
                        Traceback (most recent call last):
                          File "/usr/lib/python3.7/threading.py", line 1281, in _shutdown
                            t.join()
                          File "/usr/lib/python3.7/threading.py", line 1032, in join
                            self._wait_for_tstate_lock()
                          File "/usr/lib/python3.7/threading.py", line 1048, in _wait_for_tstate_lock
                            elif lock.acquire(block, timeout):
                          File "/home/pi/DuetLapse/DuetLapse3.py", line 467, in quit_gracefully
                            terminate()
                          File "/home/pi/DuetLapse/DuetLapse3.py", line 798, in terminate
                            sys.exit(0)
                        SystemExit: 0
                        
                        

                        Hypercube-Evolution-Hybrid, Piezo Orion, Orbiter
                        Duet WiFi 1.02 or later + DueX5
                        RepRapFirmware for Duet 2 WiFi/Ethernet 3.4.0beta4 (2021-09-27 11:30:36)
                        Duet WiFi Server: 1.26
                        Duet Web Control 3.4.0beta4 (2021-09-27)

                        stuartofmtundefined 1 Reply Last reply Reply Quote 0
                        • stuartofmtundefined
                          stuartofmt
                          last edited by

                          Version 3.2.0 has been release.
                          This is a significant upgrade and other than bug fixes I expect it will be stable for a while.

                          I added a helper program startDuetLapse3 to enable remote start/stop of DuetLapse3 instances using a browser or similar (e.g. curl). This provides a lot of flexibility in that you can have the helper program set to run at boot time and then remotely start / stop DuetLapse3 instances when you need them without having to use additional scripts or SSH into the machine etc.

                          ###Version 3.2.0###

                          [1] Adds the ability to work with startDuetLapse3. See the description here:
                          https://github.com/stuartofmt/DuetLapse3/blob/main/startDuetLapse3.md
                          In summary startDuetLapse3 provides the ability to remotely start / stop DuetLapse3.
                          [2] Changed the shutdown method to reduce low level 'noise' messages.
                          [3] Restructured to provide better efficiency. Typically less than 1% CPU when idle.
                          [4] Fixed a bug that was introduced in the prior version (not detecting layer changes)
                          [5] Made further improvements to the handling of pauses.
                          There are some edge cases where a printer can get stuck in a paused state.
                          The documentation has been updated. See especially -dontwait
                          [6]Added checks to prevent startup with ports that are already in use.

                          I have done a lot of testing but as usual - something may have slipped through the cracks .....

                          JohnOCFIIundefined 1 Reply Last reply Reply Quote 0
                          • stuartofmtundefined
                            stuartofmt @SIam
                            last edited by stuartofmt

                            @SIam said in DuetLapse3:

                            is it possible to catch those errors in python ?

                            Yes - it was one of the things that annoyed me too! one of those things that a 1000 google 'fixes' (all different) did not fix 😞 I found a way though 🙂
                            Its part of the 3.2.0 release.

                            Hope I got them all 🙂

                            1 Reply Last reply Reply Quote 0
                            • JohnOCFIIundefined
                              JohnOCFII @stuartofmt
                              last edited by

                              @stuartofmt I'm looking forward to testing this and eliminating my auto-start setup. I find my older setup is re-starting multiple times a day, and only noticed it doing longer prints, when the timelapse was only for the last period of time. Kind of sad after a 10 hour print yielded a timelapse of the last 4 minutes...

                              stuartofmtundefined 1 Reply Last reply Reply Quote 0
                              • stuartofmtundefined
                                stuartofmt @JohnOCFII
                                last edited by

                                @JohnOCFII said in DuetLapse3:

                                ..... Kind of sad after a 10 hour print yielded a timelapse of the last 4 minutes...

                                I hear ya! FWIW - I have "soak tested" this running overnight and into the next day with a small number of seconds (smaller than practical). 10K+ images and it was stable but I'm sure you can find something for me to fix 🙂

                                Note that it also runs on WIN10 (and maybe earlier ??) as well as WSL (which I like a lot). So if the Pi becomes a limitation there are other options.

                                Let me know how it works out.

                                1 Reply Last reply Reply Quote 1
                                • SIamundefined
                                  SIam
                                  last edited by

                                  hi,

                                  thanks for your work!! Now i have a question for what is the whitelist ? i get this error:

                                  Traceback (most recent call last):
                                    File "DuetLapse3.py", line 18, in <module>
                                      from DuetLapse3 import whitelist
                                    File "/home/pi/DuetLapse/DuetLapse3.py", line 18, in <module>
                                      from DuetLapse3 import whitelist
                                  ImportError: cannot import name 'whitelist' from 'DuetLapse3' (/home/pi/DuetLapse/DuetLapse3.py)
                                  
                                  

                                  the module is installed

                                  pi@raspberrypi:~/DuetLapse $ sudo find / -name whitelist*
                                  /home/pi/.local/lib/python3.7/site-packages/whitelist
                                  /home/pi/.local/lib/python3.7/site-packages/whitelist-0.2.8.dist-info
                                  
                                  

                                  Hypercube-Evolution-Hybrid, Piezo Orion, Orbiter
                                  Duet WiFi 1.02 or later + DueX5
                                  RepRapFirmware for Duet 2 WiFi/Ethernet 3.4.0beta4 (2021-09-27 11:30:36)
                                  Duet WiFi Server: 1.26
                                  Duet Web Control 3.4.0beta4 (2021-09-27)

                                  stuartofmtundefined 1 Reply Last reply Reply Quote 0
                                  • stuartofmtundefined
                                    stuartofmt @SIam
                                    last edited by stuartofmt

                                    @SIam said in DuetLapse3:

                                    hi,

                                    thanks for your work!! Now i have a question for what is the whitelist ? i get this error:

                                    The short answer is that startDuetLapse.py needs to be in the same directory as DuetLapse3.py AND DuetLapse3.py needs to be at least at V3.2.0 (i.e. the latest version) .

                                    I have never seen this error - but then I satisfy the above :-).

                                    I just tried to simulate it by putting startDuetLapse3.py in a different directory and get a related but slightly different error. So I suspect that you may have an older version of DuetLapse3.py ?

                                    python3 ./DuetLapse3.py -h
                                    

                                    Needs to have this line in its return (up near the top)
                                    Create time lapse video for Duet3D based printer. V3.2.0

                                    The longer answer is this: When DuetLapse3.py is started manually, the various input options are checked for validity (i.e. parsed through a whitelist).
                                    To help prevent misuse and provide a measure of security startDuetLapse3.py invokes the very same code (imported from DuetLapse3.py) BEFORE it tries to start DuetLapse3.py (which then dutifully does the same thing again).

                                    Why bother apparently doing it twice? To help prevent "someone" (not you, someone malicious) from accessing startDuetLapse3 via the http port and trying to execute some arbitrary code.
                                    The original version (proof of concept) just parsed through any old command line. Dangerous indeed.
                                    To feel comfortable - I needed startDuetLapse3 to be very specific to DuetLapse3. One of those defenses (minimal but likely 'good enough') was to use a whitelist. Since it did not make sense to try and maintain the whitelist it in two places - I restructured some of the code in DuetLapse3 3.2.0 to allow the whitelist to be imported. Hence the dependency on versions and location.

                                    1 Reply Last reply Reply Quote 0
                                    • SIamundefined
                                      SIam
                                      last edited by

                                      Sorry this was my fault my SD card was full after cleanup the SD card and Reinstallation the duet lapse is working.

                                      Hypercube-Evolution-Hybrid, Piezo Orion, Orbiter
                                      Duet WiFi 1.02 or later + DueX5
                                      RepRapFirmware for Duet 2 WiFi/Ethernet 3.4.0beta4 (2021-09-27 11:30:36)
                                      Duet WiFi Server: 1.26
                                      Duet Web Control 3.4.0beta4 (2021-09-27)

                                      1 Reply Last reply Reply Quote 0
                                      • jnofsingerundefined
                                        jnofsinger
                                        last edited by

                                        Hello,

                                        Trying out 3.2.0 and running into the following issue (below). The print starts fine and captures a handful of images till the connection reset issue. Both the duet and the pi are running on the same wifi subnet, and the duet is the latest firmware version.

                                        running with the following options (have more logs if needed)

                                        /usr/bin/python3 /home/pi/Desktop/images/DuetLapse3.py -port 8181 -verbose -duet 172.16.37.118 -detect layer -camera1 stream -weburl1 http://127.0.0.1:8080/?action=stream

                                        During handling of the above exception, another exception occurred:

                                        Traceback (most recent call last):
                                        File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
                                        self.run()
                                        File "/usr/lib/python3.7/threading.py", line 865, in run
                                        self._target(*self._args, **self._kwargs)
                                        File "/home/pi/Desktop/images/DuetLapse3.py", line 1068, in captureLoop
                                        oneInterval('Camera1', camera1, weburl1, camparam1)
                                        File "/home/pi/Desktop/images/DuetLapse3.py", line 612, in oneInterval
                                        zn = getDuetLayer(apiModel)
                                        File "/home/pi/Desktop/images/DuetLapse3.py", line 733, in getDuetLayer
                                        r = requests.get(URL, timeout=5)
                                        File "/usr/lib/python3/dist-packages/requests/api.py", line 75, in get
                                        return request('get', url, params=params, **kwargs)
                                        File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
                                        return session.request(method=method, url=url, **kwargs)
                                        File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
                                        resp = self.send(prep, **send_kwargs)
                                        File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
                                        r = adapter.send(request, **kwargs)
                                        File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
                                        raise ConnectionError(err, request=request)
                                        requests.exceptions.ConnectionError: ('Connection abort

                                        stuartofmtundefined 1 Reply Last reply Reply Quote 0
                                        • SIamundefined
                                          SIam
                                          last edited by

                                          @stuartofmt sorry but i have also a error for you

                                          Exception in thread Thread-2:
                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
                                              chunked=chunked)
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 384, in _make_request
                                              six.raise_from(e, None)
                                            File "<string>", line 3, in raise_from
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 380, in _make_request
                                              httplib_response = conn.getresponse()
                                            File "/usr/lib/python3.7/http/client.py", line 1336, in getresponse
                                              response.begin()
                                            File "/usr/lib/python3.7/http/client.py", line 306, in begin
                                              version, status, reason = self._read_status()
                                            File "/usr/lib/python3.7/http/client.py", line 267, in _read_status
                                              line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
                                            File "/usr/lib/python3.7/socket.py", line 589, in readinto
                                              return self._sock.recv_into(b)
                                          ConnectionResetError: [Errno 104] Connection reset by peer
                                          
                                          During handling of the above exception, another exception occurred:
                                          
                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/requests/adapters.py", line 449, in send
                                              timeout=timeout
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
                                              _stacktrace=sys.exc_info()[2])
                                            File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 367, in increment
                                              raise six.reraise(type(error), error, _stacktrace)
                                            File "/usr/lib/python3/dist-packages/six.py", line 692, in reraise
                                              raise value.with_traceback(tb)
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
                                              chunked=chunked)
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 384, in _make_request
                                              six.raise_from(e, None)
                                            File "<string>", line 3, in raise_from
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 380, in _make_request
                                              httplib_response = conn.getresponse()
                                            File "/usr/lib/python3.7/http/client.py", line 1336, in getresponse
                                              response.begin()
                                            File "/usr/lib/python3.7/http/client.py", line 306, in begin
                                              version, status, reason = self._read_status()
                                            File "/usr/lib/python3.7/http/client.py", line 267, in _read_status
                                              line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
                                            File "/usr/lib/python3.7/socket.py", line 589, in readinto
                                              return self._sock.recv_into(b)
                                          urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer                                                                 '))
                                          
                                          During handling of the above exception, another exception occurred:
                                          
                                          Traceback (most recent call last):
                                            File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
                                              self.run()
                                            File "/usr/lib/python3.7/threading.py", line 865, in run
                                              self._target(*self._args, **self._kwargs)
                                            File "/home/pi/DuetLapse/DuetLapse3.py", line 1041, in captureLoop
                                              duetStatus=getDuetStatus(apiModel)
                                            File "/home/pi/DuetLapse/DuetLapse3.py", line 707, in getDuetStatus
                                              r = requests.get(URL, timeout=120)  #Long timeout to handle restart of Duet
                                            File "/usr/lib/python3/dist-packages/requests/api.py", line 75, in get
                                              return request('get', url, params=params, **kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
                                              return session.request(method=method, url=url, **kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
                                              resp = self.send(prep, **send_kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
                                              r = adapter.send(request, **kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
                                              raise ConnectionError(err, request=request)
                                          requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by p                                                                 eer'))
                                          
                                          

                                          Hypercube-Evolution-Hybrid, Piezo Orion, Orbiter
                                          Duet WiFi 1.02 or later + DueX5
                                          RepRapFirmware for Duet 2 WiFi/Ethernet 3.4.0beta4 (2021-09-27 11:30:36)
                                          Duet WiFi Server: 1.26
                                          Duet Web Control 3.4.0beta4 (2021-09-27)

                                          stuartofmtundefined 1 Reply Last reply Reply Quote 0
                                          • stuartofmtundefined
                                            stuartofmt @jnofsinger
                                            last edited by

                                            @jnofsinger said in DuetLapse3:

                                            File "/home/pi/Desktop/images/DuetLapse3.py", line 733, in getDuetLayer
                                            r = requests.get(URL, timeout=5)

                                            Your command is almost identical to one I used successfully yesterday - the only meaningful difference being -verbose.

                                            The problem does not look like there was a timeout issue - if there was there should be a message in the logs like:

                                            getDuetLayer failed to get data. Code: <some number> Reason: <some text>

                                            SO that points to a communication issue between the pi and the Duet i.e. it looks like it could not even connect. The fact that it worked sometimes makes it hard to track down and suggests that the basic code is ok.

                                            Let me thing about how to trap the connection issue and report it - although to be frank, I don't want to disappear down the rabbit hole of debugging Duet connections 😞

                                            In any case, in parallel, can you trawl through the WDC forum and see what discussions are taking place on connection issues. My sense is that you will need to address some aspect anyway , independently of anything I can do in the code.

                                            I hope to have a patched version for you to try later today / tomorrow but as I said , I doubt it will fix anything - maybe just allow Duet to report the error or try a little harder.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Unless otherwise noted, all forum content is licensed under CC-BY-SA