Metrics from DCS?



  • Hi *,

    I understand that I can get the current metrics from the DCS

    By the M409 command, for example sent over a serial link. See Gcode
    By the rr_model HTTP command. This behaves just like the M409 command but is processed directly by the HTTP server, avoiding the command queue. It takes optional parameters "key" and "flags".
    

    (https://duet3d.dozuki.com/Wiki/Object_Model_of_RepRapFirmware)

    What I do not understand is the http endpint (wget http://localhost/rr_model returns a 404), and I'm not sure which serial port I should use as a alternative. I'm a bit lost.... ❔ ❔ ❔

    I want to collect the metrics and store them into a time series database which runs on the same pi as the dcs.

    Maybe @chrishamm ? Are you able to point me into the right direction?

    Cheers, Chriss



  • Oh... I have forgotten that the client is not able to speak with a websocket. HTTP(S) only. 😞





  • Yes wget http://localhost/machine/status.

    However, if you're happy with python you can use pydsfapi https://github.com/Duet3D/DSF-APIs/tree/master/pydsfapi

    One thing to note, if you use the subscription connection and get 'patches', what this returns is not a 'classic' json patch, but rather just the changed elements of the nested dict and list structure. So you need something to apply the patch to the copy of the machine model you are using.

    With pydsfapi I do this (This just prints out the setpoint and current value of each defined heater once every 3 seconds for 15 minutes. The majority of it is the routine applying the patch, to log something else or log it into a database you possibly only need to tamper with the lines in the vicinity of line 120):

    #!/usr/bin/python3 -u
    # subscribe to the machine model and report the temperatures of each heater
    # note '-u' to force unbuffered output
    
    # this is frequency of logging (seconds)
    interval=3
    
    # this is duration over which to log (seconds)
    duration=900
    
    import json
    import time
    from pydsfapi import pydsfapi
    from pydsfapi.initmessages.clientinitmessages import SubscriptionMode
    
    
    # update dictionary-of-dicts-and-lists a with a merge patch b
    # values in b over-write values with same key tree in a
    # note this function recurses
    # note pydsfapi patches are just a json string so need to be turned into a dict
    # before feeding to this function
    # if a and b are both lists and b is shorter than a, a will be truncated
    # this behaviour can be changed below - search 'truncated'
    # debug=True draws a diagram
    # debug=True and verbose=True reports input to the function on every entry
    def patch(a, b, path=None, debug=False, verbose=False):
        if path is None: path = []
        if debug and (len(path)==0 or verbose): 
            print('patch: ' + str(b))
            if verbose:
                print(' into: ' + str(a))
                print(' at :  ' + str(path))
    
        # if both a and b are dicts work through the keys
        if isinstance(a,dict) and isinstance(b,dict):
            for key in b:
                if debug:
                    # debug draws a sort of diagram indicating nesting depth
                    print('   ',end='')
                    for i in path: print(' > ', end='')
                    print(key)         
                if key in a:
                    # the element in the patch already exists in the original
                    if ((isinstance(a[key], dict) and isinstance(b[key], dict))
                     or (isinstance(a[key], list) and isinstance(b[key], list))):
                        # a[key] and b[key] are both dicts or both lists, recurse
                        a[key] = patch(a[key], b[key], path + [str(key)], debug, verbose)
                    else:
                        # mixed types, so treat as leaf
                        if debug:
                            for i in path: print('   ',end='')
                            print('    = '+str(b[key]))
                        a[key] = b[key]
                else:
                    # the element in the patch is not in the original
                    # treat as a leaf, but note either could actually be a dict or list
                    # e.g. a[key] could have been a single value and we now overwrite with a new dict
                    # or it could have been a dict and now we've overwritten a single scalar
                    if debug:
                        for i in path: print('   ',end='')
                        print('    = ' + str(b[key]))
                    a[key]=b[key]
    
        # if both a and b are lists we enumerate and work through the values
        elif isinstance(a,list) and isinstance(b,list):
            # if b is shorter than a, should a be truncated or should the trailing values be unaltered?
            # The following line assumes that trailing values not found in b are truncated from a
            # remove this line if desired behaviour is that trailing values in a are left in place
            # note that would mean that a list will grow but never shrink
            a=a[0:len(b)]
            # if b is longer than a you could use a try: except IndexError: below
            # but that requires putting an entire recursion of this function inside the try
            # which makes it difficult to track where the exception occurs, so instead
            # a is padded to be as long as b in advance
            a.extend([None] * (len(b)-len(a)))
            for idx, value in enumerate(b):
                # need a new diagram nesting line
                if debug:
                    print('   ',end='')
                    for i in path: print(' > ',end='')
                    # put [] around index to identify we're in a list
                    print('['+str(idx)+']')
                a[idx] = patch(a[idx], b[idx], path + [str(idx)], debug, verbose)
    
        else:
            # a and b are different types, replace a with b
            if debug:
                for i in path: print('   ',end='')
                print(' = '+str(b))
            a=b
    
        # all done
        return a
    
    
    # establish the connection
    subscribe_connection = pydsfapi.SubscribeConnection(SubscriptionMode.PATCH, debug=False)
    subscribe_connection.connect()
    
    # decide when to stop logging
    endat = time.time() + duration;
    
    try:
        # start with empty machine model
        machine_model = {}
    
        # Get updates
        while time.time() < endat:
            time.sleep(interval - time.time()%interval)
            # get machine model update as a string
            mm_u_str = subscribe_connection.get_machine_model_patch()
            # convert to a  dict
            mm_update=json.loads(mm_u_str)
            # apply to the saved machine model
            patch(machine_model,mm_update)
            # do something with the machine model
            # in this case just print out heater setpoints and current values
            print (time.strftime('%H:%M:%S'), end='')
            for heater in machine_model['heat']['heaters']:
                print (' {:5.1f} {:5.1f}'.format(heater['active'],heater['current']), end='')
            print()
    finally:
        subscribe_connection.close()
    


  • @achrn

    Well I would use ruby but this should not be a war. 😉 Thanks for your idea. I plan to use "inputs.http" in telegraf, so I can query a object in the json path directly: "json_query = "message_stats.publish_details.rate""

    That should do it for me for now. I do not plan to write a module for telegraf, at least not for now. 😉

    Cheers, Chriss


Log in to reply