Q for @DC42 and Chris Hammacher

Danal

I'm writing some long Python scripts on the Pi on a Duet 3. At this moment, my primitive call on which everything else is built looks like this:

def Gcode(cmd=''):
  print(cmd)
  g0Stream = os.popen('echo \''+cmd+'\' | sudo /opt/dsf/bin/CodeConsole')
  g0Line = g0Stream.readline()
  g0Line = g0Stream.readline()
  if (g0Line != "\n"):
    print('Response = '+g0Line)

My question is: Is there a better API than 'echo blah |CodeConsole'? Something else that I should be using? Something that I should bind to Python in a different way? Or is console OK for issuing many commands, back-to-back? (It seems a bit slow).

A Former User

Stepping in like I own the place as usual ... but could you not re-use the process handle from os.open and g0Stream.writeline() your g-code?

(that is what i'd do in non-python world for such a thing, and I'll be on my way now:)

Danal

Well Duh. Of course, in MY OWN CODE, open the connection once and re-use it. They are pipes after all.

DUH.

Thank you, and thank you for the gentle way you pointed that out. Something about forest and trees comes to mind...

A Former User

No worries; we've all been there! I was just a bit apprehensive answering a question directed at big cheese and dammit now I googled 'big cheese and' and crave a big cheese and bacon stack burger from McDonalds.

Anyways, attention span and happy to help:)

Danal

So, assuming that I change the way I'm handling the Python interface to open the connection and re-use it... assuming that...

The Q still stands: Is command line CodeConsole the best way, or is there another, deeper API that @DC42 or @chrishamm would recommend? That binds to Python? Or another language suitable for scripting on the Pi?

A Former User

I guess they be busy; but as far as I know you have the options of interfacing towards CodeConsole, or a WebSocket, pretty much the same. The next level down would be to more or less copy CodeConsole which talks to DuetControlServer throug the /var/run/dsf/dcs.sock ipc socket; I would think you would need a relatively high throughput before something like that would be necessary.

At the same time I only know what I've been able to spot from the sidelines. AFAIK they haven't released bindings for interfacing to DuetControlServer in other languages, and imho the learnig curve wouldn't make sense compared to reading and writing well documented G-Code over a socket.

Luke'sLaboratory

Why couldn't you use websockets thru python?

TLAS

@Danal
Totally not a Python expert (I prefer C++ / C# / JavaScript), but I have written a bit in it.

The JSON library in python is excellent and conversion works well. Not knowing exactly what you’re looking to do, I’d either digest the JSON object model from the PI server and use conversions for Python, or implement a function similar to the Gcode read function in the firmware that uses switch conversion. Either way is a very elegant and efficient way to deal with GCode.

Also, if you’re looking to issue commands to the Duet, you should be able to duplicate the same socket or http requests the web interface uses. Go dive into the server code or the web page code if you understand JavaScript. Would be well worth your time for writing something new.

gtj0

@bearer said in Q for @DC42 and Chris Hammacher:

The next level down would be to more or less copy CodeConsole which talks to DuetControlServer throug the /var/run/dsf/dcs.sock ipc socket;

BTW, that interface is just plain JSON. The problem is that it hasn't been documented yet. You could just monitor the socket to see what passes over it but opening an HTTP connection for commands/responses and a WebSocket for async events to the DuetWebServer is dead simple in Python.

Open a websocket to "ws://<dsf_host>/machine" to get the events.
To send GCode, do a POST to "http://<dsf_host>/machine/code" with the GCode string as the POST data.

Danal

Thanks, guys, all good ideas.

I've tried numerous ways to make a connection to CodeConsole more efficient; they all run into the fact that CodeConsole closes stdin when it sees EOF. Also, I'm not married to Python. I'm just scripting. Python was primarily because I wanted to lean toward something that is installed by default on a Pi.

So... websockets, or the dcs.sock ipc socket. I think I'm going down the websocket path next. Seems simplest and most consistent.

THANKS.

gtj0

@Danal said in Q for @DC42 and Chris Hammacher:

Thanks, guys, all good ideas.

I've tried numerous ways to make a connection to CodeConsole more efficient; they all run into the fact that CodeConsole closes stdin when it sees EOF. Also, I'm not married to Python. I'm just scripting. Python was primarily because I wanted to lean toward something that is installed by default on a Pi.

So... websockets, or the dcs.sock ipc socket. I think I'm going down the websocket path next. Seems simplest and most consistent.

THANKS.

You can't send commands via the WebSocket. It's just for receiving events (state changes, temp changes, current coordinates, etc). You'll still need to use the HTTP POST to send GCode.

Danal

In order to deal with only the socket, I'm going to send commands formatted like this:

{"code":"M122 B1","channel":0,"command":"SimpleCode"}

A Former User

@Danal said in Q for @DC42 and Chris Hammacher:

they all run into the fact that CodeConsole closes stdin when it sees EOF.

thats fair enough, but it shouldn't get an EOF before you call os.pclose() or g0Stream goes out of scope methinks. (although you clearly know more python than me, so I could be wrong, incorrect or all of the above)

Danal

@bearer said in Q for @DC42 and Chris Hammacher:

@Danal said in Q for @DC42 and Chris Hammacher:

they all run into the fact that CodeConsole closes stdin when it sees EOF.

thats fair enough, but it shouldn't get an EOF before you call os.pclose() or g0Stream goes out of scope methinks. (although you clearly know more python than me, so I could be wrong, incorrect or all of the above)

Yeah, you'd think. But whether invoked with any of several os.xxxx or fork, it ends after the first write to it's cloned stdin. I think python is set up so much to be nonblocking... maybe...

Anyway, I'm off in socket land, and I like it a LOT better.

Danal

Proof-of-concept level success in socketland. Decide to bypass the web socket and go straight to the control server's (internal) socket. This little program works:

#!/usr/bin/env python3
import socket
import json
import time

s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.connect('/var/run/dsf/dcs.sock')

j=json.dumps({"mode":"command"})
print(j)
s.send(j.encode())
time.sleep(0.002)
r=s.recv(128)
print(r.decode())

j=json.dumps({"code": "M122 B1","channel": 0,"command": "SimpleCode"})
print(j)
s.send(j.encode())
r=s.recv(128)
print(r.decode())
if (r.decode() == '{"success":true}'):
  r=s.recv(1024)
  print(r.decode())

Danal

the 'dumps' or 'loads' turn python dictionaries to/from string-style-json

the '.encode()' and matching dcode turn simple strings to/from "byte like objects", which are a python thing necessary for dealing with raw streams such as internal sockets.

There is a bit of socket stream handling here and there, and then there is a bit of specific payloads and/or sequences that are Dave and Chris's way of talking to the control server. I figured those out from traces of what DWC was doing.

Anyway, it should be possible to built the scripts I want to build on top of little wrappers for the above. Right this instant, my mind is fried and it is a VERY pretty day in North Texas. Going outside!!

A Former User

Yeah, if the unix socket runs off json then why not.

But CodeConsole runs fine with multiple commands through socat and across TCP (running my PanelDue now like so:) so its definitively a quirk with how its handled in Python; the only thing that strikes me is that it could be the echo cmd | bit that causes the issue with echo closing up its stdout causing CodeConsole to get the EOF right away? Anyway, the horse is dead, i'll stop beating it, or take a break at least:)

Danal

@bearer said in Q for @DC42 and Chris Hammacher:

Yeah, if the unix socket runs off json then why not.

But CodeConsole runs fine with multiple commands through socat and across TCP (running my PanelDue now like so:) so its definitively a quirk with how its handled in Python; the only thing that strikes me is that it could be the echo cmd | bit that causes the issue with echo closing up its stdout causing CodeConsole to get the EOF right away? Anyway, the horse is dead, i'll stop beating it, or take a break at least:)

I'm still kind of curious. Not the echo, that went away when I moved from shell to trying to keep a pipe open. Things like this. The first command M122 works, the second M114 hangs (stalls) forever. Tried LOTS of variants.

Anyway, like you said, horse, dead, beat, etc, etc.

rs, ws = os.pipe()
if not os.fork():
  #child
  os.dup(rs)    #stdin  to parent pipe
  os.dup2(ws,1) #stdout to parent pipe
  os.dup2(ws,2) #stderr to parent pipe
  os.execv('/opt/dsf/bin/CodeConsole',['dummy'])
  print('Failed to start CodeConsole')
  sys.exit(8)

#parent
wsf = os.fdopen(ws,'w')
rsf = os.fdopen(rs,'r')

line = rsf.readline()
print(line)
if ('Connected!\n' != line):
  print('Error connecting to DSF')
  print(line)
  for line in rsf:
    print(line)
  exit(8)


wsf.write('M122 B1\n')
line = rsf.readline()
print(line)

wsf.write('M114\n')
for line in rsf:
    print(line)

Danal

And... here is a library based on the POC that can be imported into a Python program, with everything following the library being more-or-less straightforward gcode calls.

Only very lightly tested... and at least some of the sequence/race in the POC were wrong, so they could still be wrong here.

import socket
import json

def openDSF():
    s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    s.connect('/var/run/dsf/dcs.sock')
    j=json.dumps({"mode":"command"}).encode()
    s.send(j)
    r=s.recv(128).decode()
    if (-1 == r.find('{"version":')):
          print("Failed to enter command mode - version not received")
          print(r)
          exit(8)
    if (-1 == r.find('{"success":true}')):   #could be in same buffer as version
        r=s.recv(128).decode()
        if (-1 == r.find('{"success":true}')):
              print("Failed to enter command mode - success not received")
              print(r)
              exit(8)
    return(s)

def closeDSF(s):
    s.close()

def Gcode(s,cmd=''):
    j=json.dumps({"code": cmd,"channel": 0,"command": "SimpleCode"}).encode()
    s.send(j)
    r=s.recv(2048).decode()
    return(r)

def getPos(s):
  result = json.loads(Gcode(s,'M408'))['result']
  pos = json.loads(result)['pos']
  print('getPos = '+str(pos))
  return pos

Very short test:

#!/usr/bin/env python3
from DWClib import *

DSFsock = openDSF()

r=Gcode(DSFsock,'M114')
print(r)
print('====================================')

r=Gcode(DSFsock,'M122 B1')
print(r)
print('====================================')

r=Gcode(DSFsock,'M408')
print(json.loads(r))
print('====================================')

p=getPos(DSFsock)
print(p[0])

closeDSF(DSFsock)

A Former User

@Danal Nice, now how about using a proper language? (...while I go hide under my desk:)

((My bad, that was just meant as praise with a sidedish of a poor joke; will poke it and see if it deals with M117, M118, M300 and other asyncronous stuff better than CodeConsole for mye PanelDue adventures. Thanks for doing the legwork!))