Boilerplate HTTP connection code for Python-based UI

poofjunior

Hi folks,

I'd like to write a python-based that emulates the web behavior of being able to:

poll the state of the machine
send over simple GCode commands or entire files.

First off, are there leads on doing this already? I'm super new to web programming, but I want to adhere to best practices for interacting with the exposed network-based API. From what I've read, I should be connecting to the Duet via websockets, not via http polling.

I started by opening up dev-tools in my browser on DWC's main page, I was able to track down the connection on this URL:

http://192.168.1.2/rr_connect?password=reprap&time=2019-10-30T17%3A29%3A0

(My Duet has a static IP for now.) It looks like a password, date, and time are encoded too.

Pasting that URL into a connection from Python's socketIO library throws a 404 with this code, though:

import socketio                                                                                                            
sio = socketio.Client()                                                                                                    
sio.connect("http://192.168.1.2/rr_connect?password=reprap&time=2019-10-30T17%3A29%3A0")

However, using vanilla HTTP Requests with Python works flawlessly:

import requests                                                                                                         
params = { "password": "reprap",
           "time": "2019-10-30T17:29:0"
         }
r = requests.get("http://192.168.1.2", params=params)  # this works!                                                                              
r = requests.get("http://192.168.1.2/rr_status?type=1")  # this also works!                                        
print(r.json())

What do folks recommend? I could simply setup a thread to poll the machine state and send GCode from the main thread using the Python requests library. Is there a better approach?

Thanks for taking a look and offering suggestions!

Finally, it looks like others have visited this topic before, but the conversation ended without any leads:
https://forum.duet3d.com/topic/441/web-ui-native-wrapper

I also referenced the Duet Web UI code, but I'm not a web programmer by trade.

Danal

Duet/RepRap V2 firmware (i.e. the printer) uses HTTP polling. Duet/RepRap V3 uses sockets. DWC V2 (i.e. the browser) code has a module that senses which is which as it initially attempts to connect, and uses the correct protocol. (DWC V1 uses only HTTP and therefore only connect to V2 firmware).

See: https://github.com/chrishamm/DuetWebControl/tree/master/src/store/machine/connector

If you want to do more than look at source, be aware that browser development consoles deal very poorly with minified files, and the distribution copy of DWC is minified. Therefore, if you wish to use run time breakpoints, examine data, etc, the sequence is roughly:

clone the DuetWebControl repository to your hard drive (git clone).
Edit the "vue.config.js" file to have " optimization: { minimize: false }," just after the "performance:" line
Execute "npm run build". This takes quite a while.

It will build a 'dist' directory that contains NON minified application. Upload the whole zip to your Duet via the "upload system files" button in DWC. (don't unzip it). And then... you can load the webpage in your browser, open the developer console (F12 in Chrome) and stop/trace the app, etc, etc.

Danal

P.S. the non-minified app will still be web-packed into one very large javascript file. I've found that Chrome crashes if you edit that in browser and try to save it (so that your edits take effect). To be clear, breakpoints, data watches, everything except save, that all works correctly for the non-minified file. So it is still the best way to trace things.

At the moment, I haven't found a browser than handles it any better.

Therefore, if you wish to actually change something (like inserting console.log statements), it is best to edit the very large app .js file, push it to the duet, and reload the browser (be sure you have development console open when you reload, as this bypasses all caching).

Directly editing the big app .js avoids re-running 'npm run build' which takes 5 to 20 min depending on your computer.

And... if you happen to be running Duet 3 hardware, you can do all of the above, all the builds, edits, pushes, etc, on the Rpi that is part of Duet 3 architecture. Quite nifty really.

If Duet 2 hardware, you'll still have to edit on a PC and upload all or part of the WWW directory to the Duet 2.

poofjunior

Thanks @Danal for the quick reply. I was definitely using RepRap 2 with the old interface. I'm now up-to-date with the RepRap 3 (beta) firmware and DWC 2.

Running this code

import socketio
sio = socketio.Client()
sio.connect("ws://192.168.1.2/machine", transports="websocket")

now errors out with:

websocket._exceptions.WebSocketBadStatusException: Handshake status 404 page not found<br>Check that the SD card is mounted and has the correct files in its /www folder

which looks closer. Ok, so now for some noob questions. Am I making the connection correctly? Attempting to connect to ws://192.168.1.2/machine is what my browser is connected to. Do I need to send the reprap password along in this transaction somehow?

Thanks again!

gtj0

Use the "websocket" module instead of socketio. You'll also need to send "OK" back on the websocket to get the next message.

#!/usr/bin/env python

#
# Hello, REST API! Requires the websocket-client library.
#

import websocket
import sys
import json
import urllib

def main(argv):
	ws = websocket.create_connection(
		"ws://%s/machine" % sys.argv[1])
	try:
		print ("Ready.")
		msg_str = ws.recv()
		while msg_str is not None:
			msg_json = json.loads(msg_str)
			print(json.dumps(msg_json, sort_keys=True,
				indent=2, separators=(',', ': ')))
			ws.send("OK\n")
			msg_str = ws.recv()
	except KeyboardInterrupt:
		print ("*** Closing")
	except websocket.WebSocketConnectionClosedException:
		print ("*** Closed")
	ws.close()


if __name__ == "__main__":
	sys.exit(main(sys.argv) or 0)

poofjunior

Thanks @gtj0. Does this code work for you?

I get the error:

websocket._exceptions.WebSocketBadStatusException: Handshake status 200 OK

when running it on "ws://192.168.1.2/machine"

This error happens on this line of execution:

ws = websocket.create_connection("ws://192.168.1.2/machine")

which prevents me from being able to reply back with the OK.

Is there any other sort of "handshaking" that the webui does before making this connection?

gtj0

Wait, I may have misunderstood something. You're using RRF3 on a Duet2? If so, then the websocket won't work. That's only for a Duet3 with the Duet Software Framework on a single board computer. Sorry about that. I saw you asking about websocket and didn't read the previous posts thoroughly.

Your original approach should work fine on a Duet2 regardless of the version of RRF you're using. You don't really have to use rr_connect at all for your application.

There's an explanation of the rr_* commands at
https://github.com/chrishamm/DuetWebControl/tree/legacy

poofjunior

Ahhh--that makes sense now! Ok, I will revert to my original plan of using HTTP requests.

Thanks a ton for clearing this up.

Just curious, do folks know if there's any plan to port websocket support to the Duet Ethernet/Wifi? (Perhaps there are hardware limitations that make this nice-to-have feature difficult in practice?) I just want to make sure I do development on something lasting.

gtj0

@poofjunior said in Boilerplate HTTP connection code for Python-based UI:

Ahhh--that makes sense now! Ok, I will revert to my original plan of using HTTP requests.

Thanks a ton for clearing this up.

Just curious, do folks know if there's any plan to port websocket support to the Duet Ethernet/Wifi? (Perhaps there are hardware limitations that make this nice-to-have feature difficult in practice?) I just want to make sure I do development on something lasting.

There definitely are limitations on the Duet2 which would make this unlikely. Having said that, @dc42 did say that it might be possible at some future point to re-purpose one of the SPI busses on the Duet2 to be used with the Duet Software Framework running on an SBC. I emphasize might.

I posed a follow-on question here...
https://forum.duet3d.com/topic/12322/duet2-spi-and-dsf
Unfortunately, for me, "next week" turned into "next month"

Gevorgun

Thanks for the idea, man. I'll try to correct my mistakes.

Zentro

I'm very poorly versed in python. Perhaps due to my ignorance of all the subtleties of programs written in python, I was constantly deleting files I needed or writing dumb code.