Duet3D Logo Duet3D
    • Tags
    • Documentation
    • Order
    • Register
    • Login

    [SBC only] Solution For Persistent Global Variables

    Scheduled Pinned Locked Moved
    DSF Development
    3
    9
    488
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • oozeBotundefined
      oozeBot
      last edited by oozeBot

      We wanted to share our solution for setting persistent global variables in a SQLite database on the SBC. It uses ExecOnMCode to create three new mCodes and DSF-Python to return global variables within the ObjectModel. This isn't yet 100% bulletproof, and I'm sure we'll be expanding the functionality that we can share, but it is solid enough that we have now implemented it in our machines.

      This requires an understanding of installing/using ExecOnMCode, Python, SQLite, Linux, etc that we really don't have the time to teach. However, we will try to explain this in detail below.

      The three new mCodes are:

      • M9050 - load all variables
        • Inputs:
          • none
        • Description: Loads all saved variables to the ObjectModel
        • Use Case: Should be used in config.g to load all global variables after reset
        • Example: M9050
      • M9055 - set variable
        • Inputs:
          • N = name of variable
          • V = value of variable
        • Description: Creates/Updates variable within the database and the ObjectModel
        • Use Case: Should be used any time a persistent global variable is set
        • Example: M9055 N"wookie" V"Chewbacca"
      • M9060 - delete variable
        • Inputs:
          • N = name of variable
        • Description: Deletes variable within the database and sets it to Null within the ObjectModel
        • Use Case: Should be used any time a persistent global variable is removed
        • Example: M9055 N"wookie"

      A SQLite database needs to be created. There are plenty of resources on the web to explain how to do this. In the Python scripts that follow, the path/name of your database will need to be updated. Once created, connect to the database and run the following command within SQLite to create the variables table.

      create table variables(name varchar(50) PRIMARY KEY NOT NULL, value varchar(255) NOT NULL);
      
      1 Reply Last reply Reply Quote 3
      • oozeBotundefined
        oozeBot
        last edited by

        Service Code for M9050

        [Unit]
        Description=M9050 - Load All Global Variables
        After=duetcontrolserver.service
        Requires=duetcontrolserver.service
        
        [Service]
        ExecStart=/usr/local/bin/execonmcode -mCode 9050 -command "./mCodes/M9050.py" -returnOutput
        Restart=always
        RestartSec=5
        
        [Install]
        WantedBy=multi-user.target
        

        Python script for M9050.py

        #!/usr/bin/env python3
        
        import sys
        import sqlite3
        
        from dsf.connections import CommandConnection
        from dsf.commands.basecommands import MessageType
        from dsf.commands.basecommands import LogLevel
        
        Database = "/oozeBot/sql.db"
        
        ########################################################################################################
        
        def UpdateVariable(name:str, value:str):
          command_connection = CommandConnection()
          command_connection.connect()
          command_connection.perform_simple_code("if !exists(global." + name + ") || global." + name + "=null")
          command_connection.perform_simple_code(" global " + name + "=null")
          command_connection.perform_simple_code("else")
          command_connection.perform_simple_code(" set global." + name + "=null")
          command_connection.perform_simple_code("set global." + name + "=\"" + value + "\"")
          command_connection.close()
        
        ########################################################################################################
        
        def LoadAll():
          con = sqlite3.connect(Database)
          cur = con.cursor()
          res = cur.execute("select * from variables;").fetchall()
          for row in res:
            UpdateVariable(row[0] , row[1])
          con.close()
        
        ########################################################################################################
        
        def RecordsExists():
          con = sqlite3.connect(Database)
          cur = con.cursor()
          res = cur.execute("select count(*) from variables;").fetchone()
          return bool(res[0])
          con.close()
        
        ########################################################################################################
        
        if __name__ == "__main__":
          if (RecordsExists()):
            LoadAll()
        
        1 Reply Last reply Reply Quote 2
        • oozeBotundefined
          oozeBot
          last edited by oozeBot

          Service Code for M9055

          [Unit]
          Description=M9055 - Set Global Variable
          After=duetcontrolserver.service
          Requires=duetcontrolserver.service
          
          [Service]
          ExecStart=/usr/local/bin/execonmcode -mCode 9055 -command "./mCodes/M9055.py %%N %%V" -returnOutput
          Restart=always
          RestartSec=5
          
          [Install]
          WantedBy=multi-user.target
          

          Python script for M9055.py

          #!/usr/bin/env python3
          
          import sys
          import sqlite3
          
          from dsf.connections import CommandConnection
          from dsf.commands.basecommands import MessageType
          from dsf.commands.basecommands import LogLevel
          
          Database = "/oozeBot/sql.db"
          
          ########################################################################################################
          
          def UpdateVariable(name:str, value:str):
            command_connection = CommandConnection()
            command_connection.connect()
            command_connection.perform_simple_code("if !exists(global." + name + ") || global." + name + "=null")
            command_connection.perform_simple_code(" global " + name + "=null")
            command_connection.perform_simple_code("else")
            command_connection.perform_simple_code(" set global." + name + "=null")
            command_connection.perform_simple_code("set global." + name + "=\"" + value + "\"")
            command_connection.close()
          
          ########################################################################################################
          
          def RecordExists(name:str):
            con = sqlite3.connect(Database)
            cur = con.cursor()
            res = cur.execute("select count(*) from variables where name= \"" + name + "\";").fetchone()
            return bool(res[0])
            con.close()
          
          ########################################################################################################
          
          def UpdateRecord(name:str, value:str):
            con = sqlite3.connect(Database)
            cur = con.cursor()
            cur.execute("update variables set value=\"" + value + "\" where name=\"" + name + "\";")
            con.commit()
            con.close()
          
          ########################################################################################################
          
          def InsertRecord(name:str, value:str):
            con = sqlite3.connect(Database)
            cur = con.cursor()
            cur.execute("insert into variables (name, value) values (\"" + name + "\", \"" + value + "\");")
            con.commit()
            con.close()
          
          ########################################################################################################
          
          if __name__ == "__main__":
            if (RecordExists(sys.argv[1])):
              UpdateRecord(sys.argv[1], sys.argv[2])
            else:
              InsertRecord(sys.argv[1], sys.argv[2])
            UpdateVariable(sys.argv[1], sys.argv[2])
          
          1 Reply Last reply Reply Quote 2
          • oozeBotundefined
            oozeBot
            last edited by oozeBot

            Service Code for M9060

            [Unit]
            Description=M9060 - Delete Global Variable
            After=duetcontrolserver.service
            Requires=duetcontrolserver.service
            
            [Service]
            ExecStart=/usr/local/bin/execonmcode -mCode 9060 -command "./mCodes/M9060.py %%N" -returnOutput
            Restart=always
            RestartSec=5
            
            [Install]
            WantedBy=multi-user.target
            

            Python script for M9060.py

            #!/usr/bin/env python3
            
            import sys
            import sqlite3
            
            from dsf.connections import CommandConnection
            from dsf.commands.basecommands import MessageType
            from dsf.commands.basecommands import LogLevel
            
            Database = "/oozeBot/sql.db"
            
            ########################################################################################################
            
            def NullVariable(name:str):
              command_connection = CommandConnection()
              command_connection.connect()
              command_connection.perform_simple_code("if !exists(global." + name + ") || global." + name + "=null")
              command_connection.perform_simple_code(" global " + name + "=null")
              command_connection.perform_simple_code("else")
              command_connection.perform_simple_code(" set global." + name + "=null")
              command_connection.close()
            
            ########################################################################################################
            
            def RecordExists(name:str):
              con = sqlite3.connect(Database)
              cur = con.cursor()
              res = cur.execute("select count(*) from variables where name= \"" + name + "\";").fetchone()
              return bool(res[0])
              con.close()
            
            ########################################################################################################
            
            def DeleteRecord(name:str):
              con = sqlite3.connect(Database)
              cur = con.cursor()
              cur.execute("delete from variables where name=\"" + name + "\";")
              con.commit()
              con.close()
            
            ########################################################################################################
            
            if __name__ == "__main__":
              if (RecordExists(sys.argv[1])):
                DeleteRecord(sys.argv[1])
                NullVariable(sys.argv[1])
            
            1 Reply Last reply Reply Quote 1
            • oozeBotundefined
              oozeBot
              last edited by oozeBot

              Example:

              M9055 N"wookie" V"Chewbacca"
              echo global.wookie
              M9055 N"wookie" V"Chewy"
              echo global.wookie
              M9060 N"wookie"
              echo global.wookie
              

              Console Output:
              ed374cf3-9670-4003-a89a-adaef9a42de6-image.png

              resamundefined 1 Reply Last reply Reply Quote 1
              • resamundefined
                resam @oozeBot
                last edited by

                @oozebot great idea of putting this into a small sqlite DB!

                If you want to simplify the files and all the exec-on-m-code invocations a bit, you can directly subscribe to M-codes from a single Python script started by a single systemd service unit:

                from dsf.commands.basecommands import MessageType
                from dsf.connections import CommandConnection, InterceptConnection, SubscribeConnection
                from dsf.initmessages.clientinitmessages import InterceptionMode, SubscriptionMode
                
                ...
                
                            filters = ("M9055", "M9060")
                            intercept_connection = InterceptConnection(
                                InterceptionMode.PRE, filters=filters
                            )
                            intercept_connection.connect()
                            while True:
                                cde = intercept_connection.receive_code()
                                if cde.majorNumber == 9055:
                                     print("do something with parameter B", cde.parameter("B", 0))
                                     intercept_connection.resolve_code()
                               elif cd.majorNumber == 9060:
                                    print("do something else")
                                    intercept_connection.resolve_code()
                               ...
                
                oozeBotundefined 1 Reply Last reply Reply Quote 1
                • oozeBotundefined
                  oozeBot @resam
                  last edited by

                  @resam Thanks for the pointer. I knew of this but hadn't had a chance to dig into it yet. I'm guessing this is pretty close to how ExecOnMCode works. I'll give this a go here soon!

                  1 Reply Last reply Reply Quote 0
                  • mikeabuilderundefined
                    mikeabuilder
                    last edited by

                    This is and interesting thread. I'm not using an SBC but I've recently implemented a new macro that I call "create_global_variables.g". I run this macro as one of the earliest lines in config.g. What I have in the file is a bunch of "global <variable> = whatever" lines. I try not to create global variables elsewhere.

                    I did this for a few reasons:

                    • I can set the value of a global in any macro without first testing that it has been created. Makes easier to read gcode.

                    • I can more easily manage my globals and not have duplicates. I add new variables in alphabetic order.

                    • This file has "the feel" of a C header file. I'm not a C coder (I like python), but I do understand the header file concept.

                    It would be easy (but cumbersome and eventually a headache) to programmatically add new variables (or to change values) in this macro file, by using the "echo >>" (append to file) command in RRF3.4. New entries would not be in alphabetical order and eventually changes in values would leave a long collection of useless code. I suppose daemon.g could kick off another macro for cleanup of the file...

                    Clearly not as useful as a real database, but can get the job done.

                    oozeBotundefined 1 Reply Last reply Reply Quote 1
                    • oozeBotundefined
                      oozeBot @mikeabuilder
                      last edited by

                      @mikeabuilder This was posted, in part, to show a proof of concept in returning external data into DSF. Setting persistent global variables is a good use case, but I already have several more complex ideas in mind!

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post
                      Unless otherwise noted, all forum content is licensed under CC-BY-SA