Question About Reading Object Model (SBC)

pjl

Hi,

I want to get my python script to read object model variables on demand. I could use help grasping what is the best (cheapest/fastest) way of getting the data.

What Im trying now: following a dsf-python example, subscribing with a filter

Q1- Do filters only work in PATCH mode? Whatever filter I have, FULL mode gives out the whole thing.

filter_str = 'heat.heaters[1].current'
from dsf.connections import SubscribeConnection
from dsf.connections import CommandConnection
data = subscribe_connection.get_machine_model()

The above spits out

{'heat': {'heaters': [{}, {'current': 22.3}, {}]}}

Q2- The only way I managed to get to the float value I want is this monstrosity of call. Am I missing some obvious simple way of directly getting the value?

print(update['heat']['heaters'][1]['current'])

Q3- If I use a filter list, such as

filter = ['move.axes[0].machinePosition', 'heat.heaters[1].current']

Since its PATCH mode, the response does not include the move.axes part if there was no movement. I can work around that, but before I spend time reinventing the wheel, is there a way to just get all filter values on every call to the model? I imagined FULL subscription mode with filters would do that, but guess not since it just gives the whole model no matter what.

Thanks a bunch, you're all wonderful

chrishamm

@pjl Sorry for the delay, I was away over the weekend.

Q1: Yes, filters only work in Patch mode. I can check if I can add support for Full mode in v3.5 if it helps.
Q2+Q3: To be fair I did not write the Python bindings but from a first view at least the implementation of the subscription mode appears to be incomplete. The .NET client lets you merge object model updates in patch mode easily but I haven't found a corresponding overload in the Python client. Looks like I need to address this myself in v3.5. Likewise, in .NET you can access object model properties pretty much like from RRF macros, so e.g. model.heat.heaters[1].current - I'll check if I can port over the object model definition as well.

pjl

@chrishamm Thanks.. I have not a clue on how to even use the .NET client, so I will continue with that I barely know (python)

Interesting to note, at first I was establishing the socket connection once and then sending a command in a loop just to see how often I can do that:
I made a little class like so

class Get:

 def __init__(self):
    self.subscribe_connection = SubscribeConnection(SubscriptionMode.PATCH, filter_list = filter_str)
    self.subscribe_connection.connect()
    
 def receiveData(self):
    machine_model = self.subscribe_connection.get_machine_model()

And then in my code

#init code bits
subscriber = Get()
while not terminate_flag:
    subscriber.receiveData()

The above gets me between 1 to 3 responses per second
But if I subscribe to connection WITHIN the loop, like so:

#init code bits
while not terminate_flag:
    subscriber = Get()
    subscriber.receiveData()

I get about 250-300 responses per second. Am I missing something? I have no clue why this would be such a huge difference

Also, a side question (since now my bottlenck is sending commands, since I can do it max 6-7 times per seconds using perform_simple_code() )

How do I use the other methods, such as perform_code() ? As I assume it can send commands faster (please correct me if Im wrong). In the source its

def perform_code(self, cde: code.Code):
    """Execute an arbitrary pre-parsed code"""
    res = self.perform_command(cde, result.CodeResult)
    return res.result

so I must supply a parsed code to this, but I can't figure out how to parse it.. I dont understand what is meant by ""code type must be code.Code" Its just a class?

chrishamm

@pjl A subscriber connection in patch mode only supplies the changed properties except on the first message where it sends everything covered by the filter. You should only need one subscriber connection and for the time being perhaps consider using it in full mode - at least until the Python client can deal better with partial object model updates. If you change to full mode, you should keep receiving the entire object model with live values as part of every message. I guess in your second example you keep creating new subscriber connections but that doesn't sound right.

The reason why you are seeing such a low performance for simple_code is because it always waits for a response from the Duet before the command completes. To get reasonable throughput (i.e. to stream G-code instructions), you can send codes but don't wait for an immediate response:

With the Python client this is still a bit inconvenient but it should work. You need to convert your codes into code instances, set flags to CodeFlags.Asynchronous, and send them using perform_code. If you need to get the code replies, create a new code interceptor with mode set to Executed and look for the commands you sent before - the code reply is stored in Result.
DSF comes with a tool called CodeStream (/opt/dsf/bin/CodeStream) which lets you stream lots of G-codes (from stdin) without having to wait for every response first. The underlying connection mode is still relatively new so I'm afraid the current Python client doesn't support it yet either.
It might be possible to create multiple threads with an individual command connection per thread and to send codes evenly using the send_code function. But TBH I know too little about Python to judge whether this is actually feasible with the current client. Either way this isn't the easiest solution.

sinned6915

@pjl there was some stuff i as trying to do that nodered might have worked but I got sidetracked.

i did not delve too deep into that rabbit hole, way over my head as it turnes out. but here is the link-

https://flows.nodered.org/node/node-red-contrib-dsfnode

and full Object Model was available.

pjl

@chrishamm that clears things up a bit. I will attempt to go with your 1st suggestion.

The issue now is just me not understanding how to use the damned packages.. Im still new in python, hell any programming, dont lynch me.

CodeFlags class doesnt interact with Code class, so how can I set the flags to anything I send? CommandConnection does not have flags attribute either..

When I make a Code instance, it wants a command. The only requirement is that its a string, since the BaseCommand __ init__ is activated due to inheritance (so many inheritances... brain hurt). So I give it a string, but what then?

@mfs12 maybe can help? I only really need a single example to get to grips

from dsf.connections import CommandConnection
from dsf.commands.code import Code

command_connection = CommandConnection()
command_connection.connect()

container = Code('G01 X-2 F100')
command_connection.perform_code(container)

Results in internal server exception. perform_code() description says pre-parsed code input, but I see no methods in Code class how to parse anything, just a method to get back a string from json...

MintyTrebor

@pjl It might be worth parsing the 3rd party software list for python examples you can reference. My own MQTT4DSF may still prove useful as a reference in some way, even though it is depreciated. (I rarely use Python anymore)

If you are new to programming I would echo sinned6915 and have a look at node-red with nodedsf. It's a visual way of building simple programs and functions, and it may help you develop a working proof of concept easier.

Are you able to add a bit more detail around what you are trying to achieve?

pjl

@mintytrebor I think I've seen most of the plugins using dsfpython there are, but all of them were essentially just using parts of the examples present in the github folder..

I am in no position to switch to node-red, time limits are in place and I need to process my data with opencv and <<do math>> before sending out commands to dsf. Essentially its an IR sensor that gives me 21 FPS, and I wish to be able to send a command every cycle, so that I can have a simple synchronous piece of code.
I could try and do a workaround with threads and some buffer that saves what my control loop wants to send on every cycle and only sends the latest received command, after a response was received (if using send_simple_code for example).

But before going down the multithread path, I thought it would be fairly simple just to use a different dsfpython method to send code faster, since I do not care for the responses. But boy oh boy that dsfpython module is a bit above my code pay-grade haha

pjl

I got my stuff working with send_simple_code, but damn the send and receive output is flooding my terminal. So I want to know how to use send_code() just to stop the responses, increased performance is just a bonus haha.

Call me dumb, I cant figure out how to do this with python-dsf...

from dsf.connections import CommandConnection
from dsf.commands.code import Code
from dsf.commands.code import CodeFlags


com = 'M115'

def send_code():
    command_connection =  CommandConnection(debug=True)
    command_connection.connect

    instance = Code(com)

    test = CodeFlags 
    test.Asynchronous
    try:
        command_connection.perform_code(instance)
        
    finally:
        command_connection.close()


if __name__ == "__main__":
    send_code()

I thought this is what @chrishamm was saying, I still dont see how CodeFlags is playing a role here, but I get this when trying the above

halp? How can I debug what is going on in DCS? I tried

sudo systemctl stop duetcontrolserver

sudo /opt/dsf/bin/DuetControlServer -l debug -r

but then DWC does not connect anymore saying DCS is not started, and the terminal is just showing this "updated key boards" over and over, does not seem to end

just goes on and on

chrishamm

This would look more reasonable but I am not Python programmer so I can't really judge:

from dsf.connections import CommandConnection
from dsf.commands.code import Code
from dsf.commands.code import CodeFlags
from dsf.commands.code import CodeType


def send_code():
    command_connection =  CommandConnection(debug=True)
    command_connection.connect()

    test = Code()
    test.Type = CodeType.MCode
    test.MajorNumber = 115
    test.Flags = CodeFlags.Asynchronous
    try:
        command_connection.perform_code(test)
        
    finally:
        command_connection.close()


if __name__ == "__main__":
    send_code()

I hope we can get someone soon to take better care of the Python DSF client.

You don't need to fiddle with DCS itself, just make sure pi is part of the dsf group (should be the default if your DuetPi installation is somewhat recent) and then launch your script as usual.

pjl

@chrishamm thanks for the snip, helps with understanding (there are no methods like "Type" or "MajorNumber" defined under Code class so I had no clue this was possible). But I do not understand what you mean by "make sure pi is part of dsf group"

A problem though.. to make a Code() class instance, one must provide a command argument, which is supposed to be a string.. ~~then the same issue with "NoneType has no attribute sendall" being not there happens, as the screen in my last post shows.~~

Ok so I missed the damn brackets after command_connection.connect ()

Code instance still requires a command string, but In test_custom_m_codes.py I saw this

so I tried specifying command string as "Code"

 instance = Code("Code", Type='CodeType.MCode', MajorNumber=115)

and it works!

pjl

Can anyone explain why the parameters do not get executed when using the perform_code() method? This is the only way I managed to get this to send successfully:

from dsf.connections import CommandConnection
from dsf.commands.code import Code
from dsf.commands.code import CodeFlags
from dsf.commands.code import CodeType
from dsf.commands.codeparameter import CodeParameter

def send_code():
    command_connection = CommandConnection(debug=True)
    command_connection.connect()

    instance = Code("Code")
    instance.Type = CodeType.MCode
    instance.MajorNumber = 106
    instance.parameter = CodeParameter('S', 255)
    instance.Flags = CodeFlags.Asynchronous

It sends, responds with success, but nothing happens, fan speed does not change, but commands with no parameter work fine, such as G28, or T0 etc.

send: {"command":"Code","Type":"M","MajorNumber":106,"parameter":{"letter":"S","string_value":"255","_CodeParameter__parsed_value":255,"is_expression":false},"Flags":1}
recv: {"success":true}