Log sensor data in "real-time".

hauschka

Hi community,

I'm looking to log data from a sensor that is connected to a duet board into a file and time-stamp it.
Upon processing that data, the program should adjust printing parameters mid-print.

My aim is to get everything done via dsf-python.

I found this example for how to subscribe to the objetc model: https://github.com/Duet3D/dsf-python/blob/main/examples/subscribe_object_model.py

Is there a way to subscribe only to a single sensor value?
Is there a way to actively poll every eg 250 ms, or do I have to wait to be updated?
Is there a way to always run this script in the background when a print is executed, or do I always need to manually start it (or start the script via a custom M code?)
Are there more examples online of custom py-dsf scripts?

Pointers into any direction are greatly appreciated.
Thank you!

chrishamm

@hauschka said in Log sensor data in "real-time".:

Is there a way to subscribe only to a single sensor value?

Yes, you need to specify a filter in that case. If you use a filter, only a subset of the object model is passed to your application. You can experiment using the ModelObserver utility (SSH into your board and run /opt/dsf/bin/ModelObserver with a filter of your choice [can be a standard OM node, e.g. sensors.analog[0].lastReading]).

Is there a way to actively poll every eg 250 ms, or do I have to wait to be updated?

DSF polls the main OM every 250ms by default (customisable in /opt/dsf/conf/config.json, value ModelUpdateInterval) so if your Python app is fast enough, you should already receive changes every 250ms. Note that no update is sent if the value remains the same.

Is there a way to always run this script in the background when a print is executed, or do I always need to manually start it (or start the script via a custom M code?)

You need to start your program somehow and it can observe the print status, how you do it is up to you.

Are there more examples online of custom py-dsf scripts?

To be fair I don't maintain the Python library, perhaps @Falcounet can help there. There are a few examples at https://github.com/Duet3D/dsf-python/tree/main/examples

Falcounet

Looking at this I found out some issues with the OM in dsf-python that need to be fixed.
I should be able to fix it in the following days and may reply back here with a more detailed example.

hauschka

Thanks @chrishamm, appreciate your reply. Super useful, as always!

@Falcounet thanks, looking forward to hearing from you.

I'll give this a go in the next week and will reply back!

hauschka

I gave it a go, and got the send_simple_code.py to run.

However, upon trying to execute subscribe_object_model.py, I'm getting the following error:

Traceback (most recent call last):
  File "/home/pi/subscribe_object_model.py", line 31, in <module>
    subscribe()
  File "/home/pi/subscribe_object_model.py", line 18, in subscribe
    object_model = subscribe_connection.get_object_model()
  File "/usr/local/lib/python3.7/dist-packages/dsf/connections/subscribe_connection.py", line 35, in get_object_model
    object_model = self.receive(ObjectModel)
  File "/usr/local/lib/python3.7/dist-packages/dsf/connections/base_connection.py", line 78, in receive
    return cls.from_json(json.loads(json_string))
  File "/usr/local/lib/python3.7/dist-packages/dsf/object_model/model_object.py", line 65, in from_json
    return cls()._update_from_json(**preserve_builtin(data))
  File "/usr/local/lib/python3.7/dist-packages/dsf/object_model/object_model.py", line 164, in _update_from_json
    self._move = Move.from_json(kwargs.get('move'))
  File "/usr/local/lib/python3.7/dist-packages/dsf/object_model/model_object.py", line 65, in from_json
    return cls()._update_from_json(**preserve_builtin(data))
  File "/usr/local/lib/python3.7/dist-packages/dsf/object_model/move/move.py", line 186, in _update_from_json
    self._extruders = [Extruder.from_json(e) for e in kwargs.get('extruders', [])]
  File "/usr/local/lib/python3.7/dist-packages/dsf/object_model/move/move.py", line 186, in <listcomp>
    self._extruders = [Extruder.from_json(e) for e in kwargs.get('extruders', [])]
  File "/usr/local/lib/python3.7/dist-packages/dsf/object_model/model_object.py", line 65, in from_json
    return cls()._update_from_json(**preserve_builtin(data))
  File "/usr/local/lib/python3.7/dist-packages/dsf/object_model/move/extruder.py", line 174, in _update_from_json
    self._driver = DriverId(**preserve_builtin(driver)) if driver is not None else None
  File "/usr/local/lib/python3.7/dist-packages/dsf/utils.py", line 48, in preserve_builtin
    return {f"{k}_" if k in reserved_keys else k: v for k, v in data.items()}
AttributeError: 'str' object has no attribute 'items'

was that the error you referred to, @Falcounet , or is this a dfferent error, @Falcounet ?

Steps to reproduce:

Install dsf-python: pip3 install dsf-python
Create py file
Execute py file: sudo /usr/bin/python3.7 /home/pi/subscribe_object_model.py

Falcounet

@hauschka Yes
I did the tests with too basic configurations.
I already have a fix for this but I also figured out the OM patching doesn't work properly. I'm on it.

Note that you don't need to run your script using sudo which should be used only when needed as it elevate user privileges.

hauschka

Hi,

That's great to hear!
Thank you so much for maintaining this library and finding fixes so fast!
I'm running them as root, as I was running into this error previously:

Traceback (most recent call last):
      File "/home/pi/subscribe_object_model.py", line 11, in <module>
        from dsf.connections import CommandConnection
      File "/home/pi/.local/lib/python3.7/site-packages/dsf/__init__.py", line 2, in <module>
        import random_UA
    ModuleNotFoundError: No module named 'random_UA'

I realized this issue was a privilege error that would disappear when using sudo.
However, I think I'm introducing the error myself, as I'm coding the script via Remote VS code and the SSH FS extension to access the file system, which introduces certain quirks with privileges.

Looking forward to receiving your fix
Thanks again!

Falcounet

@hauschka You seems to have installed dsf-python as a normal user as it seems to be installed in your user home. I didn't tried that actually but it should be fine.

Not sure from where that import random_UA comes from though. Maybe something with the IDE you are using yes.

Falcounet

@hauschka I released dsf-python 3.4.5-1 which fix a major issue for ObjectModel patching. See that post.

As for what you are trying to achieve, I would just use the subscribe_object_model.py example script you mentionned and then use the model.update() method to update it with the patch changes and finally read the sensor value you need.
You can use time.sleep(<value in seconds>) if you want to add some delay (Using asyncio is probably a better approach but that's another story)

update = subscribe_connection.get_object_model_patch()
model = object_model.update_from_json(update)
sensor_value = object_model.sensors.analog[0].last_reading

The array index value (here 0) should match the S parameter value of M308 in your config.g

hauschka

Thanks @Falcounet
I somehow missed your message.

For now, I worked around it by querying a specific variable using the M409 command, which is not the most elegant way of doing it. I'll give your new solution a go right away.
Thanks for updating it!