Duet3D Logo Duet3D
    • Tags
    • Documentation
    • Order
    • Register
    • Login

    pydsfapi [v3.2.0] - Official Python Client Library for DSF

    Scheduled Pinned Locked Moved
    DSF Development
    6
    51
    3.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JBiscundefined
      JBisc @wilriker
      last edited by JBisc

      @wilriker For beginners (of APIs) it would be very useful, so see an example in which you interact more with the duet (like move/ call M115 / read pin). I feel slighly overwhelmed with your example....

      My Setup: Duet 3 MB6HC Main Board with attached RPI

      wilrikerundefined 2 Replies Last reply Reply Quote 0
      • wilrikerundefined
        wilriker @JBisc
        last edited by

        @JBisc There is an update to pydsfapi coming in the next couple of days. I'll update the examples then also. Thanks for pointing it out.

        Manuel
        Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
        with probably always latest firmware/DWC (incl. betas or self-compiled)
        My Tool Collection

        1 Reply Last reply Reply Quote 1
        • wilrikerundefined
          wilriker @JBisc
          last edited by

          @JBisc I haven't changed much with the examples but added a lot of comments (see https://github.com/Duet3D/DSF-APIs/blob/dev/pydsfapi/examples.py). Is that already enough to get you started? If not can you please provide a detailed set of things you want to achieve and I'll be happy to either add this to the examples.py or reply to it here in the thread.

          Manuel
          Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
          with probably always latest firmware/DWC (incl. betas or self-compiled)
          My Tool Collection

          JBiscundefined 2 Replies Last reply Reply Quote 1
          • JBiscundefined
            JBisc @wilriker
            last edited by JBisc

            @wilriker Thanks. I will try it directly tomorrow and give feedback to you. Just for others who are searching: these changes are still on the dev Branch.

            My Setup: Duet 3 MB6HC Main Board with attached RPI

            1 Reply Last reply Reply Quote 0
            • JBiscundefined
              JBisc @wilriker
              last edited by JBisc

              @wilriker Ok I tried. I took the latest commit on the dev branch (Hash: 7b7c9291aaa1e77bd570c2503f981a73410a0f32)

              I changed:

              cmd_conn.connect('/var/run/dsf/dcs.sock')
              

              I ran the example file with python 3.7 and the RPI and sudo rights (dcs.sock requires it)

              and I got the following error

              File "/home/shares/pi/HandwritingRoboter/src/DSF-APIs/pydsfapi/pydsfapi.py", line 234, in connect
                server_init_message.EXPECTED_SERVER_VERSION, server_init_message.version))
              AttributeError: 'ServerInitMessage' object has no attribute 'EXPECTED_SERVER_VERSION'
              

              Probably you have an Idea what that could be?

              My Setup: Duet 3 MB6HC Main Board with attached RPI

              wilrikerundefined 1 Reply Last reply Reply Quote 0
              • wilrikerundefined
                wilriker @JBisc
                last edited by wilriker

                @JBisc Yeah, the reason was me refactoring a name right before committing but after testing.

                But there is another problem why it would not have happened for me anyway: the version on the dev branch requires DSF 2.1.0 from unstable repo.

                EDIT: I pushed another change fixing this bug right before I posted here. So once you upgraded to DSF 2.1.0 the latest commit on dev should work.

                Manuel
                Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
                with probably always latest firmware/DWC (incl. betas or self-compiled)
                My Tool Collection

                1 Reply Last reply Reply Quote 0
                • wilrikerundefined
                  wilriker
                  last edited by wilriker

                  Release 2.1.0

                  I present to you the first official release of pydsfapi which can be found at GitHub Releases page as usual.

                  This is the first release of pydsfapi and it is target at compatibility with Duet Software Framework 2.1.0 and later (it specifically does not work with DSF 1.2.4 and earlier).

                  With this release the package structure has been brought to a Python standard layout and a setup.py has been added for easier installation via disttools.

                  What's missing

                  • There are no concrete class implementations representing the ObjectModel. For now there is a placeholder class MachineModel that has all the values as dict with dicts inside it.

                  Manuel
                  Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
                  with probably always latest firmware/DWC (incl. betas or self-compiled)
                  My Tool Collection

                  1 Reply Last reply Reply Quote 1
                  • wilrikerundefined
                    wilriker
                    last edited by

                    Release v2.1.2

                    This release can be found as usual on GitHub Releases page.

                    Port changes from upstream

                    • Increase PROTOCOL_VERSION to 6
                    • Add G53 to string output of commands that use it

                    Manuel
                    Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
                    with probably always latest firmware/DWC (incl. betas or self-compiled)
                    My Tool Collection

                    1 Reply Last reply Reply Quote 2
                    • wilrikerundefined
                      wilriker
                      last edited by

                      Release 3.1.0

                      This release syncs changes and version with latest DuetSoftwareFramework and can be found as usual on GitHub Releases page.

                      Ported changed from upstream

                      • Increase PROTOCOL_VERSION to 7
                      • Add new Aux2 CodeChannel
                      • Comments now use Q as CodeType (instead of C)
                      • Letter for unprecedented parameters has been changed from empty string to @

                      Manuel
                      Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
                      with probably always latest firmware/DWC (incl. betas or self-compiled)
                      My Tool Collection

                      1 Reply Last reply Reply Quote 2
                      • wilrikerundefined
                        wilriker
                        last edited by

                        Release 3.1.1

                        Minor release that just updates PROTOCOL_VERSION to 8 to be in line with DuetSoftwareFramework 3.1.1.

                        Can be found at GitHub Releases page.

                        Manuel
                        Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
                        with probably always latest firmware/DWC (incl. betas or self-compiled)
                        My Tool Collection

                        1 Reply Last reply Reply Quote 2
                        • achrnundefined
                          achrn
                          last edited by

                          Line 284 of pydsfapi/pydsfapi.py limits the json response to 32kB:
                          json_string = self.socket.recv(32 * 1024).decode('utf8')

                          The full machine model can easily exceed that when a print is under way, because it contains an entry for every layer, with at least 90 bytes per layer (depending on the values and number of filaments). Then the json string is just arbitrarily truncated at 32kB and you get an error such as:

                          Traceback (most recent call last):
                            File "./delta_lights.py", line 99, in <module>
                              mm = subscribe_connection.get_machine_model()
                            File "/usr/local/lib/python3.7/dist-packages/pydsfapi/pydsfapi.py", line 455, in get_machine_model
                              machine_model = self.receive(machinemodel.MachineModel)
                            File "/usr/local/lib/python3.7/dist-packages/pydsfapi/pydsfapi.py", line 275, in receive
                              return cls.from_json(json.loads(json_string))
                            File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
                              return _default_decoder.decode(s)
                            File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
                              obj, end = self.raw_decode(s, idx=_w(s, 0).end())
                            File "/usr/lib/python3.7/json/decoder.py", line 353, in raw_decode
                              obj, end = self.scan_once(s, idx)
                          json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 32768 (char 32767)
                          

                          If you change the (32 * 1024) to something bigger it goes away, but it doesn't take a very large print to overwhelm any sensible value - 32kB at 90 bytes per layer is only 364 layers, so another 32kB gets you less than a further 73mm of print height at 0.2mm layers.

                          As I understand it, there's the facility in the DSF to filter the machine model returned. However, I haven't figured out of that is in pydsfapi and if so how to use it. Can a filter be used to not return the 'layers' entries?

                          wilrikerundefined 2 Replies Last reply Reply Quote 0
                          • wilrikerundefined
                            wilriker @achrn
                            last edited by

                            @achrn Thanks for reporting. I will look into that.

                            Manuel
                            Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
                            with probably always latest firmware/DWC (incl. betas or self-compiled)
                            My Tool Collection

                            achrnundefined 1 Reply Last reply Reply Quote 0
                            • achrnundefined
                              achrn @wilriker
                              last edited by

                              @wilriker
                              With the self.socket.recv(32 * 1024) changed to self.socket.recv(64 * 1024) and a routine interrogating the machine model and doing some occasional command_connection.perform_simple_code('M573 P0') at the same time, I've had some prints stop part-way (but neglected to record the error in detail - at the time I thought I knew what had done it so thought I didn't need to pay attention to what it was saying).

                              This may be coincidence, though it's the only time the printer has stopped mid-print, but it may be that putting that receive up to 64 kB upsets something (possibly just takes too long).

                              wilrikerundefined 1 Reply Last reply Reply Quote 0
                              • ofliduetundefined
                                ofliduet
                                last edited by

                                I'm trying to use the api with a Duet3 board with firmware 3.1.1 (stable) and the initial connect for the subscription gives me the confirmation immediately followed by a complete machine model:

                                send: {"mode": "Subscribe", "version": 8, "SubscriptionMode": "Patch", "Filter": ""}
                                recv: {"success":true}{"boards":[{"bootloaderFileName":null,"canAddress":0,"firmwareDate":"2020-05-19b2","firmwareFileName":"Duet3Firmware_MB6HC.bin","firmwareName":"RepRapFirmware for Duet 3 MB6HC","firmwareVersion":"3.1.1"
                                

                                causing a parser error

                                  File "/home/pi/src/OctoPrint/venv3/lib/python3.7/site-packages/pydsfapi/pydsfapi.py", line 446, in connect
                                    return super().connect(sim, socket_path)
                                  File "/home/pi/src/OctoPrint/venv3/lib/python3.7/site-packages/pydsfapi/pydsfapi.py", line 238, in connect
                                    response = self.receive_response()
                                  File "/home/pi/src/OctoPrint/venv3/lib/python3.7/site-packages/pydsfapi/pydsfapi.py", line 280, in receive_response
                                    return json.loads(json_string, object_hook=responses.decode_response)
                                  File "/usr/lib/python3.7/json/__init__.py", line 361, in loads
                                    return cls(**kw).decode(s)
                                  File "/usr/lib/python3.7/json/decoder.py", line 340, in decode
                                    raise JSONDecodeError("Extra data", s, end)
                                

                                Is that a known problem?

                                I'm also wondering about whether the API is threadsafe. A complete worked example with a thread running in the background periodically updating a complete machine model with patches would be extremely helpful.

                                achrnundefined wilrikerundefined 2 Replies Last reply Reply Quote 0
                                • ofliduetundefined
                                  ofliduet
                                  last edited by

                                  Different question: Is it possible to use the API to access the socket from a different host? It looks so close to me but no catch.

                                  wilrikerundefined 1 Reply Last reply Reply Quote 0
                                  • achrnundefined
                                    achrn @ofliduet
                                    last edited by

                                    @ofliduet
                                    Could it be the same issue, but truncating the json in a different location triggering a different error in the json decoder? That is, mine was failing because the truncation happened part-way through a quoted string, if you're not part-way through a string, presumably you'd get a different sort of json decode error?

                                    One diagnostic would be to print something with very few layers (one or two) to reset the layers dict in the machine model, then try again.

                                    Also, when I was doing stuff initially I sometimes got an error that if I simply repeated the same script it didn't error. I didn't get to the bottom of that, but one thing I'd try is run the same script again and see if it works.

                                    ofliduetundefined 1 Reply Last reply Reply Quote 0
                                    • ofliduetundefined
                                      ofliduet @achrn
                                      last edited by

                                      @achrn Don't think so. This is an initial model, coming in unsolicited directly after the connection has been made. The printer is standing still and I can see the complete model in the recv line. The problem might actually be on the dsf side, where it shouldn't send out a machine model unless specifically requested.

                                      achrnundefined 1 Reply Last reply Reply Quote 0
                                      • achrnundefined
                                        achrn @ofliduet
                                        last edited by

                                        @ofliduet
                                        Standing still doesn't affect it - the machine model seems to contain the last layers list, no matter how long ago that was (though presumably not if the machine and/or the pi is switched off in the meantime, though I haven't tested that).

                                        However, if you've received the whole machine model, I agree that I don't think it can be the same issue.

                                        If I do a subscribe connection but don't explicitly request a full machine model, the first 'patch' I get is the full model, but I don't seem to get an unsolicited full model. That is:

                                        import json
                                        from pydsfapi import pydsfapi
                                        from pydsfapi.initmessages.clientinitmessages import SubscriptionMode
                                        subscribe_connection = pydsfapi.SubscribeConnection(SubscriptionMode.PATCH, debug=False)
                                        subscribe_connection.connect()
                                        mm_u_str = subscribe_connection.get_machine_model_patch()
                                        mm_update=json.loads(mm_u_str)
                                        

                                        works fine for me, mm_u_str is a json string of the whole model and json.loads is happy to parse it. Subsequent mm_u_str = subscribe_connection.get_machine_model_patch() give me just small patches, as expected.

                                        ofliduetundefined 1 Reply Last reply Reply Quote 0
                                        • ofliduetundefined
                                          ofliduet @achrn
                                          last edited by

                                          @achrn Playing with it a bit more last night I can confirm that the sample code does run correctly when called directly. Where I have problems is when I call the exact some code while there are other things going on.
                                          I'm trying to write an Octoprint plugin that provide a virtual printer in Octoprint, so that I can use TheSpaghettiDetective and the Canvas plugins for the Pallette device without having to go through the serial connection. So the code is called up on startup of the Octoprint server when it loads all plugins and the process is obviously busy at that time with multiple threads doing things at the same time. This seems to have an affect, hence my question on the tread safety of the API.

                                          1 Reply Last reply Reply Quote 0
                                          • wilrikerundefined
                                            wilriker @achrn
                                            last edited by

                                            @achrn said in pydsfapi [v3.1.1] - Official Python Client Library for DSF:

                                            @wilriker
                                            With the self.socket.recv(32 * 1024) changed to self.socket.recv(64 * 1024) and a routine interrogating the machine model and doing some occasional command_connection.perform_simple_code('M573 P0') at the same time, I've had some prints stop part-way (but neglected to record the error in detail - at the time I thought I knew what had done it so thought I didn't need to pay attention to what it was saying).

                                            If it happens again please record the error and report here.

                                            This may be coincidence, though it's the only time the printer has stopped mid-print, but it may be that putting that receive up to 64 kB upsets something (possibly just takes too long).

                                            I don't think so. This is all going through a local socket connection, i.e. speeds around RAM speed - 64 KiB will be transferred in under 1ms.

                                            Manuel
                                            Duet 3 6HC (v0.6) with RPi 4B on a custom Cartesian
                                            with probably always latest firmware/DWC (incl. betas or self-compiled)
                                            My Tool Collection

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Unless otherwise noted, all forum content is licensed under CC-BY-SA