New M-code to send a network request



  • It would be great if we could have an M-code that would allow us to send a network request to another machine on the network. Use cases would be:

    • Triggering another machine to take an image for a timelapse
    • Triggering an alert when a print is finished/cooled off/faulted
    • Trigger an alert for manual filament change
    • etc

    I think we could do this via either via http or mqtt. http is probably a more mainstream option, but mqtt is perhaps a lighter-weight option to consider.

    The ideal M-code would take both a destination (http url or mqtt topic) as well as a string payload. I think it would be fine to limit the payload to a static string - the remote endpoint could then call back to the existing API to get whatever data it needs.

    I am definitely interested in helping to implement this, but figured it best to check in for strategy guidance before I go too deep.

    thanks!



  • That sounds very useful!



  • Hey I have an idea for you, one that I plan on using but over UART, in your case, you'll apply the same technique over HTTP

    All you need is a Raspberry Pi or any home PC. You need to run additional JavaScript code inside of DWC for this to work. You can do this another way but if you do it in any other way, then you can't run DWC and your own code at the same time.

    You know how when you use M291, a dialog shows up on DWC? Put a special trigger word in that M291 command. Have your script look for that trigger word. When the trigger word is detected, the rest of the M291 payload will be interpreted and the script will send a command (via HTTP or AJAX) to your camera or shoot an email. Your script can also send back M292 to signal that the task has completed.

    M118 already exists, but I think the majority of your feature depends on the target of your message. If you really don't want to do it the way I just described, then your new M code would need a destination IP address and port number as a minimum.

    Edit: The above idea is just something that is possible right at this moment with the least amount of hardware and lowest learning curve.



  • @frank26080115 no reason to get so complicated with injecting JS into DWC. Just use telnet and M118 - as some of us are already doing. For example to trigger timelapse images: https://github.com/Kriechi/DuetRRF-timelapse



  • Yes - I am familiar with watching telnet, and also know that people are hooking up outputs on the duet to a gpio pin on a raspberry pi to hook something similar to this up. I'm not a huge fan of needing to scab a pi into my printer just so I can use it for one gpio pin, nor does it seem like a great idea to turn telnet on just so you can watch what's going on.

    Modifying DWC to handle something like this is definitely possible, but relies on a running browser which is less than ideal, especially as browsers continue to get more aggressive about sleeping tabs that are not in focus.

    All seem like fairly specific and roundabout solutions, though, whereas sending an http or mqtt message out are very general purpose solutions that are common among connected/IoT type devices.

    Allowing the duet to push events, rather than relying on external processes to watch, would also scale better in a multi-device environment.



  • HTTP is ultimately based on TCP, which requires "opening" a "connection". Obviously, that connection should start when the first "M xxx URL payload" (for a given URL) is executed. When should it end? What action should be taken if it won't open? Should other G-Code commands 'wait' until the connection either suceeds or times out? How should duet manage memory if many, many, many M commands are executed all with different URLs? etc, etc.

    How about UDP? Keep it simple. "M xxx (resolvable name|ip|broadcast) payload" where the payload is limited to an ASCII string that fits normal Gcode syntax rules (including future Duet/RepRap only extensions for symbolics, etc. ).

    UDP can be "received" on almost any platform with similar syntax to opening a file stream. Also, specifying broadcast (meaning local subnet broadcast) allows multiple devices to be aware of say, a Z step change. HTTP can't do that (in one command). Multiple cameras, or other devices, could be listening.

    UDP would seem to meet all requirements and be MUCH simpler to code into Duet/RepRap. Yeah, it is "not assured"... on a local network, this shouldn't matter. If the net between the printer and a camera bolted to the printer drops packets, this should be resolved regardless of any other considerations.

    Thoughts?


  • administrators

    @danal said in New M-code to send a network request:

    HTTP is ultimately based on TCP, which requires "opening" a "connection". Obviously, that connection should start when the first "M xxx URL payload" (for a given URL) is executed. When should it end? What action should be taken if it won't open? Should other G-Code commands 'wait' until the connection either suceeds or times out? How should duet manage memory if many, many, many M commands are executed all with different URLs? etc, etc.

    How about UDP? Keep it simple. "M xxx (resolvable name|ip|broadcast) payload" where the payload is limited to an ASCII string that fits normal Gcode syntax rules (including future Duet/RepRap only extensions for symbolics, etc. ).

    UDP can be "received" on almost any platform with similar syntax to opening a file stream. Also, specifying broadcast (meaning local subnet broadcast) allows multiple devices to be aware of say, a Z step change. HTTP can't do that (in one command). Multiple cameras, or other devices, could be listening.

    UDP would seem to meet all requirements and be MUCH simpler to code into Duet/RepRap. Yeah, it is "not assured"... on a local network, this shouldn't matter. If the net between the printer and a camera bolted to the printer drops packets, this should be resolved regardless of any other considerations.

    Thoughts?

    UDP is a possibility. However, the interface between the Duet main processor and the WiFi module is socket oriented. So the simplest approach for me would probably be to open a TCP connection, send the message, close the connection, and discard any response received from the other end.



  • @danal said in New M-code to send a network request:

    Obviously, that connection should start when the first "M xxx URL payload" (for a given URL) is executed. When should it end? What action should be taken if it won't open? Should other G-Code commands 'wait' until the connection either suceeds or times out? How should duet manage memory if many, many, many M commands are executed all with different URLs? etc, etc.

    It should be one GET request and no keepalive. Reply optional. Blocking with a short timeout. Single instance at a time.

    It should never be used for mission critical actions like a filament swap. Fail safe and such.



  • @danal said in New M-code to send a network request:

    How about UDP? Keep it simple. "M xxx (resolvable name|ip|broadcast) payload" where the payload is limited to an ASCII string that fits normal Gcode syntax rules (including future Duet/RepRap only extensions for symbolics, etc. ).

    I offered the option of MQTT as a lighter alternative to HTTP. It's a super light protocol, comparatively, and since it's common among IoT devices, we wouldn't be plowing new ground with it. I think it's important in matters like this to adopt existing protocols where we can, rather than devising our own UDP protocol. If I wanted to run another bespoke service to collect events, I'd just connect over telnet and watch the stream, after all.

    Already, there's a lot of esp8266-based devices that send out MQTTT telemetry, so it's obviously well within reach of the device horsepower, and example code should exist.



  • @frank26080115 said in New M-code to send a network request:

    It should be one GET request and no keepalive. Reply optional. Blocking with a short timeout. Single instance at a time.

    I agree, the duet should do nothing to process a response. I would envision we might want a parameter for the m-code to specify a timeout and/or pause (depending on whether this was HTTP or MQTT), though, rather than trying to pick a universal default for everyone.

    I should know when I generate g-code that I need to stay put for up to a couple seconds for a picture, or if I'm just sending something "fire and forget"



  • MQTT: Where would the broker be in a typical 3D Printer scenario?

    The various discussions of "One HTTP request" and a parameter to wait for a response or not... that all sounds right.



  • @danal said in New M-code to send a network request:

    MQTT: Where would the broker be in a typical 3D Printer scenario?

    The various discussions of "One HTTP request" and a parameter to wait for a response or not... that all sounds right.

    Mine's on my NAS, but you can run that anywhere, or use a public one in the cloud.



  • I'm writing a "DuetTimelapseClient" in C# as a WinForms app for my own purposes. The laptop I use for slicing is in the same room (and same network) as the printer and it's a Windows 10 machine. The client is using a HttpWebRequest / WebRequest to communicate /w the Duet and takes care of polling the Duet. It stores the current layer #. When the current layer # on the app doesn't match the current layer coming from the Duet, that's a layer change, and it will hit my IP cam's static image URL and download that to a folder (which is the name of the timelapse + current date / time). At the end, it executes FFMPEG to stitch the images together into a timelapse. I'm also planning at having a timelapse by interval instead of just z-change. This will take me a while as I burst some discs in my back and my Hypercube build with my new Duet 2 Ethernet board was put on hold till I'm more mobile again. Plus my day job is a software dev so it's hard to put in a full day doing that and coming home and doing more of that - gotta keep that healthy work / life balance!

    Another idea I've been toying with. I've downloaded DWC to try and see if it's possible to raise a JS event upon layer change. I was hoping I could tap into the field that stores the IP / web camera URL (I'm using a Wyzecam v2), and upon layer change it raises a JS event, which uses that URL to call another "API" URL hosted on an old linux (Ubuntu) PC I'm using as a utility box (PiHole, Plex, file shares) - which is basically just a simple PHP script that takes in that URL, accesses it and downloads the resulting image to a file share.

    All the while I'm also checking printing status. When a job completes, I'd have another event raised to call that same PHP script, writing out a file called "TimeLapseName-END.jpg". I'd have a cron job wake up on the machine every 5 minutes which looks at the root folder ("Timelapses"), digs into each one looking at all *.jpg files and finding those that end with "-END". That tells the script to run FFMPEG to stitch all jpg images inside each folder into a timelapse. After stitching, it wipes the images, then the containing folder and copies the stitched timelapse to a file share and sends me an email.

    I think having two extra fields "API Image URL" (for taking the snaps) and "API Job Complete URL" (for rendering the timelapse) would make this possible - but this whole process only works if you're using an IP cam that has a static image capture function - direct web cam wouldn't work using this I don't think. For me, if I experiment with this, I'll probably just hardcode all of the values - I don't want to mess /w the general layout of DWC, just add some additional logic to it. 🙂

    Sorry to ramble! I'm sure there's much better solutions out there, but I love to tinker so 🙂



  • @lostapathy said in New M-code to send a network request:

    Mine's on my NAS, but you can run that anywhere, or use a public one in the cloud.

    No thanks. With all due respect, the concept of having a NAS, or server, or cloud be REQUIRED for my printer to trigger the camera that is bolted to it (or similar)...

    I totally get having such a service/broker for tens, hundreds, thousands, ... of IoT devices. For what should be totally local interaction between two or three endpoints, MQTT feels (to me) even "heavier" than some of the things we've already rejected.



  • HTTP with option for ultra-quick timeout seems to be a pretty good hammer.

    Still like to see an UDP option, but I seem to be in the miniority for that, and the existing internals of Duet/RepRap/WiFi-module also don't seem amenable.

    So... HTTP with timeout options. Sounds pretty good.

    Mnnn [resolvable name|IP] [timeout in ms] "payload" 
    

    Should there be an option to express a payload buffer entirely in Hexadecimal?



  • @oddkode said in New M-code to send a network request:

    Another idea I've been toying with. I've downloaded DWC to try and see if it's possible to raise a JS event upon layer change.

    Yes, but... DWC just polls. Nothing is ever "pushed" to DWC from the printer. If you are already polling, no need to 'hook' DWC.

    I was hoping I could tap into the field that stores the IP / web camera URL

    You can certainly get that from DWC internally (JS read the HTML field, among other things), but there is a potentially easier way: DWC stores its persistent settings in a file on the printer itself, and pulls them via an AJAX call. You can either:

    HTTP download the file

    http://192.168.1.100/rr_download?name=DWCconfig.txt
    

    Or make an AJAX call (still really an HTTP GET) for a JSON object (containing the same info)

    http://192.168.1.100/dwc.json
    

    Note: REALLY old DWC kept config in a cookie; then DWC kept it in local browser storage... but for some time now, the above has worked. So you should be able to ignore those old ways.


 

Looks like your connection to Duet3D was lost, please wait while we try to reconnect.