Q for @DC42 and Chris Hammacher



  • Q for @DC42 and @chrishamm :

    I'm writing some long Python scripts on the Pi on a Duet 3. At this moment, my primitive call on which everything else is built looks like this:

    def Gcode(cmd=''):
      print(cmd)
      g0Stream = os.popen('echo \''+cmd+'\' | sudo /opt/dsf/bin/CodeConsole')
      g0Line = g0Stream.readline()
      g0Line = g0Stream.readline()
      if (g0Line != "\n"):
        print('Response = '+g0Line)
    

    My question is: Is there a better API than 'echo blah |CodeConsole'? Something else that I should be using? Something that I should bind to Python in a different way? Or is console OK for issuing many commands, back-to-back? (It seems a bit slow).



  • Stepping in like I own the place as usual ... but could you not re-use the process handle from os.open and g0Stream.writeline() your g-code?

    (that is what i'd do in non-python world for such a thing, and I'll be on my way now:)



  • Well Duh. Of course, in MY OWN CODE, open the connection once and re-use it. They are pipes after all.

    DUH.

    Thank you, and thank you for the gentle way you pointed that out. Something about forest and trees comes to mind...



  • No worries; we've all been there! I was just a bit apprehensive answering a question directed at big cheese and dammit now I googled 'big cheese and' and crave a big cheese and bacon stack burger from McDonalds.

    Anyways, attention span and happy to help:)



  • So, assuming that I change the way I'm handling the Python interface to open the connection and re-use it... assuming that...

    The Q still stands: Is command line CodeConsole the best way, or is there another, deeper API that @DC42 or @chrishamm would recommend? That binds to Python? Or another language suitable for scripting on the Pi?



  • I guess they be busy; but as far as I know you have the options of interfacing towards CodeConsole, or a WebSocket, pretty much the same. The next level down would be to more or less copy CodeConsole which talks to DuetControlServer throug the /var/run/dsf/dcs.sock ipc socket; I would think you would need a relatively high throughput before something like that would be necessary.

    At the same time I only know what I've been able to spot from the sidelines. AFAIK they haven't released bindings for interfacing to DuetControlServer in other languages, and imho the learnig curve wouldn't make sense compared to reading and writing well documented G-Code over a socket.



  • Why couldn't you use websockets thru python?



  • @Danal
    Totally not a Python expert (I prefer C++ / C# / JavaScript), but I have written a bit in it.

    The JSON library in python is excellent and conversion works well. Not knowing exactly what you’re looking to do, I’d either digest the JSON object model from the PI server and use conversions for Python, or implement a function similar to the Gcode read function in the firmware that uses switch conversion. Either way is a very elegant and efficient way to deal with GCode.

    Also, if you’re looking to issue commands to the Duet, you should be able to duplicate the same socket or http requests the web interface uses. Go dive into the server code or the web page code if you understand JavaScript. Would be well worth your time for writing something new.



  • @bearer said in Q for @DC42 and Chris Hammacher:

    The next level down would be to more or less copy CodeConsole which talks to DuetControlServer throug the /var/run/dsf/dcs.sock ipc socket;

    BTW, that interface is just plain JSON. The problem is that it hasn't been documented yet. You could just monitor the socket to see what passes over it but opening an HTTP connection for commands/responses and a WebSocket for async events to the DuetWebServer is dead simple in Python.

    Open a websocket to "ws://<dsf_host>/machine" to get the events.
    To send GCode, do a POST to "http://<dsf_host>/machine/code" with the GCode string as the POST data.



  • Thanks, guys, all good ideas.

    I've tried numerous ways to make a connection to CodeConsole more efficient; they all run into the fact that CodeConsole closes stdin when it sees EOF. Also, I'm not married to Python. I'm just scripting. Python was primarily because I wanted to lean toward something that is installed by default on a Pi.

    So... websockets, or the dcs.sock ipc socket. I think I'm going down the websocket path next. Seems simplest and most consistent.

    THANKS.



  • @Danal said in Q for @DC42 and Chris Hammacher:

    Thanks, guys, all good ideas.

    I've tried numerous ways to make a connection to CodeConsole more efficient; they all run into the fact that CodeConsole closes stdin when it sees EOF. Also, I'm not married to Python. I'm just scripting. Python was primarily because I wanted to lean toward something that is installed by default on a Pi.

    So... websockets, or the dcs.sock ipc socket. I think I'm going down the websocket path next. Seems simplest and most consistent.

    THANKS.

    You can't send commands via the WebSocket. It's just for receiving events (state changes, temp changes, current coordinates, etc). You'll still need to use the HTTP POST to send GCode.



  • In order to deal with only the socket, I'm going to send commands formatted like this:

    {"code":"M122 B1","channel":0,"command":"SimpleCode"}



  • @Danal said in Q for @DC42 and Chris Hammacher:

    they all run into the fact that CodeConsole closes stdin when it sees EOF.

    thats fair enough, but it shouldn't get an EOF before you call os.pclose() or g0Stream goes out of scope methinks. (although you clearly know more python than me, so I could be wrong, incorrect or all of the above)



  • @bearer said in Q for @DC42 and Chris Hammacher:

    @Danal said in Q for @DC42 and Chris Hammacher:

    they all run into the fact that CodeConsole closes stdin when it sees EOF.

    thats fair enough, but it shouldn't get an EOF before you call os.pclose() or g0Stream goes out of scope methinks. (although you clearly know more python than me, so I could be wrong, incorrect or all of the above)

    Yeah, you'd think. But whether invoked with any of several os.xxxx or fork, it ends after the first write to it's cloned stdin. I think python is set up so much to be nonblocking... maybe...

    Anyway, I'm off in socket land, and I like it a LOT better.



  • Proof-of-concept level success in socketland. Decide to bypass the web socket and go straight to the control server's (internal) socket. This little program works:

    #!/usr/bin/env python3
    import socket
    import json
    import time
    
    s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    s.connect('/var/run/dsf/dcs.sock')
    
    j=json.dumps({"mode":"command"})
    print(j)
    s.send(j.encode())
    time.sleep(0.002)
    r=s.recv(128)
    print(r.decode())
    
    j=json.dumps({"code": "M122 B1","channel": 0,"command": "SimpleCode"})
    print(j)
    s.send(j.encode())
    r=s.recv(128)
    print(r.decode())
    if (r.decode() == '{"success":true}'):
      r=s.recv(1024)
      print(r.decode())
    


  • the 'dumps' or 'loads' turn python dictionaries to/from string-style-json

    the '.encode()' and matching dcode turn simple strings to/from "byte like objects", which are a python thing necessary for dealing with raw streams such as internal sockets.

    There is a bit of socket stream handling here and there, and then there is a bit of specific payloads and/or sequences that are Dave and Chris's way of talking to the control server. I figured those out from traces of what DWC was doing.

    Anyway, it should be possible to built the scripts I want to build on top of little wrappers for the above. Right this instant, my mind is fried and it is a VERY pretty day in North Texas. Going outside!!



  • Yeah, if the unix socket runs off json then why not.

    But CodeConsole runs fine with multiple commands through socat and across TCP (running my PanelDue now like so:) so its definitively a quirk with how its handled in Python; the only thing that strikes me is that it could be the echo cmd | bit that causes the issue with echo closing up its stdout causing CodeConsole to get the EOF right away? Anyway, the horse is dead, i'll stop beating it, or take a break at least:)



  • @bearer said in Q for @DC42 and Chris Hammacher:

    Yeah, if the unix socket runs off json then why not.

    But CodeConsole runs fine with multiple commands through socat and across TCP (running my PanelDue now like so:) so its definitively a quirk with how its handled in Python; the only thing that strikes me is that it could be the echo cmd | bit that causes the issue with echo closing up its stdout causing CodeConsole to get the EOF right away? Anyway, the horse is dead, i'll stop beating it, or take a break at least:)

    I'm still kind of curious. Not the echo, that went away when I moved from shell to trying to keep a pipe open. Things like this. The first command M122 works, the second M114 hangs (stalls) forever. Tried LOTS of variants.

    Anyway, like you said, horse, dead, beat, etc, etc.

    rs, ws = os.pipe()
    if not os.fork():
      #child
      os.dup(rs)    #stdin  to parent pipe
      os.dup2(ws,1) #stdout to parent pipe
      os.dup2(ws,2) #stderr to parent pipe
      os.execv('/opt/dsf/bin/CodeConsole',['dummy'])
      print('Failed to start CodeConsole')
      sys.exit(8)
    
    #parent
    wsf = os.fdopen(ws,'w')
    rsf = os.fdopen(rs,'r')
    
    line = rsf.readline()
    print(line)
    if ('Connected!\n' != line):
      print('Error connecting to DSF')
      print(line)
      for line in rsf:
        print(line)
      exit(8)
    
    
    wsf.write('M122 B1\n')
    line = rsf.readline()
    print(line)
    
    wsf.write('M114\n')
    for line in rsf:
        print(line)
    
    


  • And... here is a library based on the POC that can be imported into a Python program, with everything following the library being more-or-less straightforward gcode calls.

    Only very lightly tested... and at least some of the sequence/race in the POC were wrong, so they could still be wrong here.

    import socket
    import json
    
    def openDSF():
        s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
        s.connect('/var/run/dsf/dcs.sock')
        j=json.dumps({"mode":"command"}).encode()
        s.send(j)
        r=s.recv(128).decode()
        if (-1 == r.find('{"version":')):
              print("Failed to enter command mode - version not received")
              print(r)
              exit(8)
        if (-1 == r.find('{"success":true}')):   #could be in same buffer as version
            r=s.recv(128).decode()
            if (-1 == r.find('{"success":true}')):
                  print("Failed to enter command mode - success not received")
                  print(r)
                  exit(8)
        return(s)
    
    def closeDSF(s):
        s.close()
    
    def Gcode(s,cmd=''):
        j=json.dumps({"code": cmd,"channel": 0,"command": "SimpleCode"}).encode()
        s.send(j)
        r=s.recv(2048).decode()
        return(r)
    
    def getPos(s):
      result = json.loads(Gcode(s,'M408'))['result']
      pos = json.loads(result)['pos']
      print('getPos = '+str(pos))
      return pos
    

    Very short test:

    #!/usr/bin/env python3
    from DWClib import *
    
    DSFsock = openDSF()
    
    r=Gcode(DSFsock,'M114')
    print(r)
    print('====================================')
    
    r=Gcode(DSFsock,'M122 B1')
    print(r)
    print('====================================')
    
    r=Gcode(DSFsock,'M408')
    print(json.loads(r))
    print('====================================')
    
    p=getPos(DSFsock)
    print(p[0])
    
    closeDSF(DSFsock)
    


  • @Danal Nice, now how about using a proper language? (...while I go hide under my desk:)

    ((My bad, that was just meant as praise with a sidedish of a poor joke; will poke it and see if it deals with M117, M118, M300 and other asyncronous stuff better than CodeConsole for mye PanelDue adventures. Thanks for doing the legwork!))



  • @bearer said in Q for @DC42 and Chris Hammacher:

    @Danal Nice, now how about using a proper language? (...while I go hide under my desk:)

    Actually, I'm open to anything that's pre-installed on the Pi (and the Duet image, in particular), that leans even a little bit toward scripting.

    What would you suggest?



  • Nice work @Danal !



  • OK, switched to round hole. At this moment, hacked into the corner of the bed; for production would be more nicely integrated along an edge somewhere.

    Also got the script working consistently. Based on 4 back-back runs, repeatability seems to be about 0.09mm in Z and 0.02 in X and Y. These are probably better than any hand method, and usable for printing. At the same time, I'd like for them to be even more consistent and may tweak a few things.

    IMG_0169.jpeg



  • @Danal .09mm in Z repeatability seems rather high, and I would expect that to be much lower than the xy repeatability. Any thoughts as to why this is?



  • @timothyz said in Q for @DC42 and Chris Hammacher:

    @Danal .09mm in Z repeatability seems rather high, and I would expect that to be much lower than the xy repeatability. Any thoughts as to why this is?

    Yeah, I was concerned about that as well. Not sure.

    I need to literally "look" at the printer while this runs. Or single steps. With a dial gauge or similar.

    And/or, even at feedrate F100, I may back off slightly and retouch at an even lower feedrate.


Log in to reply