Feasibility of running DSF on X86

CthulhuLabs · 31 Mar 2023, 16:06

As many know Raspberry Pi 4s (and other SBCs for that matter) have gotten ridiculously expensive for what they are, which makes running a Duet in SBC mode costly. That said there are a number of benefits of running in SBC mode in terms of advanced configurations / customizations. This got me thinking a few weeks ago about running DSF on an x86 Linux server. Like many people I have an older PC laying around that should be able to run circles around a RPI4 in terms of performance and with tweaks to the power profiles actually doesn't draw that much power.

Now there are two main issues with running DSF on X86 that I see.

DSF is written for ARM so it would need to be ported.
The lack of GPIO and SPI on most x86 boards. I do know of some boards that do have GPIO pins, but I dont think I have see any with SPI pins broken out. I know many of them use SPI for things like sensors and the like, but that is all internal.

I know DSF is written in .NET which gives me hope that getting it to run on x86 might be as simple as a recompile. I have not actually dug into the code though. I don't know if there is any ARM specific stuff.

As for the lack of SPI / GPIO I was contemplating what it would take to use a USB to SPI/GPIO board. For instance there is this guy from Adafruit:

https://www.adafruit.com/product/2264

that can break out SPI and GPIO pins and has Linux drivers. It is a very low cost option and seems like it could fit the bill.

There are other options as well such as using a RPI Zero as a GPIO expander:

https://magpi.raspberrypi.com/articles/pi-zero-gpio-expander

This one is less desirable as RPI Zeros are just as hard to find as RPI4s.

I am pretty positive though that these devices do not break out the GPIO / SPI interfaces in the same way a RPI4 and other ARM SBCs do. I would need to purchase one and check in /dev to see. As such this will likely require code changes if DSF rather than just config.json changes to support these devices. I am also not sure if the SPI interface on these is fast enough.

Before I attempt to go down this path I was wondering a few things:

Have I missed anything crucial that would completely prevent this from working?
How much interest is there from the community?
Would anyone be interested in helping?

Falcounet · 31 Mar 2023, 18:16

@CthulhuLabs I'm not sure there is anything ARM specific on DSF.
As long you can get .NET on your architecture (and I believe it is on x86), you should be fine. The RPM packages are made for an armhf/arm64 though so you may need to build them yourself.

Apparently, the FT232H can generate a SPI clock signal up to 30MHz so it may be fine for DSF.

Is there a lot of x86 SBC cheaper than ARM SBC ?

CthulhuLabs · 31 Mar 2023, 19:07

@Falcounet x86 SBCs are just as expensive as ARM ones from what I am seeing.

What is not expensive is the computer you already have. For instance I recently upgraded my desktop. I turned my old PC into a Linux file server. It has an 8th gen Intel, 32gigs of RAM, an NVMe SSD, and several large drives. 99% of the time it is doing nothing. Running DSF on it should be perfectly fine.

If DSF on x86 becomes a thing, I could see people containerizing DSF in say Docker and running a print farm off a single server.

Falcounet · 31 Mar 2023, 20:57

@CthulhuLabs I'm not sure everyone is up to have a desktop computer near their printer to run it ... so not sure there will be much interest if x86 SBC aren't cheaper or more available than the ARM ones.
But I don't want to refrain you trying it.
As you say, maybe things can change if DSF on x86 become a thing.

T3P3Tony · 1 Apr 2023, 11:48

@CthulhuLabs one thing to take into consideration is the additional latency and potentially variable priority of performance of the USB connection and the USB to SPI convertor. One reason we picked the SPI bus on the PI (instead of using USB is that DSF "owns" it completely so the OS can't prioritise other traffic etc which it can with USB.

CthulhuLabs · 1 Apr 2023, 20:22

@T3P3Tony That is a good point. A user moving their mouse could potentially generate thousands of USB interrupts. I was wondering why you guys didnt make the SBC mode use USB directly. This is why I asked.

How tight do the timings need to be? Shouldnt be too hard to test with an arduino. Simply write a sketch that echo's input on the arduino's SPI back to PC SPI. Then send a "ping" to the arduino at a regular interval and log how long the response took to come back. While that is running do a bunch of USB intensive stuff.

o_lampe · 2 Apr 2023, 07:20

@CthulhuLabs What would be the benefit of running DSF on that X86 instead of (wireless) DWC?
I have an old laptop running Linux from an USB-stick. I use the browser to control my CNC. It only has a 15" screen and no fullHD capability, so running DWC is a bit awkward...
Why would you recommend using DSF? I never tried SBC at all, because I couldn't find a good reason (yet)

CthulhuLabs · 3 Apr 2023, 12:33

@o_lampe Mostly customization. For instance I have a python script that overrides the M3,M4, and M5 commands to control my external spindle with closed loop control. You have to be running in SBC mode for that to work.

chrishamm · 4 May 2023, 09:39

@CthulhuLabs You can certainly run DSF on x86 but GPIO and SPI capability is limited on x86 AFAICS. Note that DSF relies on the relatively new gpiod Linux driver to achieve reasonable transfer speeds so you need to make sure that's supported by your Linux kernel.

I am not sure how well USB <-> SPI adapters work because I'd expect them to introduce notable latency. If you want to stick with x86, perhaps consider trying something like a PCI(e) <-> SPI/GPIO card but be aware that I have not tried that yet. Hence I can't say how well or if that works at all.

PS: At least the MPX-24794S PCIe card plugs in via USB as well so probably the same limitation applies then. Not sure if there are other cards out there for GPIO/SPI.

CthulhuLabs · 6 Apr 2023, 16:55

@chrishamm what kind of latency / jitter in the SPI communication is acceptable for DSF to work properly?

chrishamm · 7 Apr 2023, 13:35

@CthulhuLabs Generally it should be as low as possible. DSF tolerates delays up to 500ms per transfer but if you see lots of delays, you'll probably get sluggish movement when performing circles or lots of short moves. Also note that RRF flags when it is ready to perform an SPI transfer and if it takes notable time before DSF sees that, it will slow down the transfer rate as well. For that reason I also rely on gpiod instead of sysfs because it reports pin changes much quicker.

CthulhuLabs · 7 Apr 2023, 14:32

@chrishamm Looks like people are claiming the latency is 1ms:

https://forums.adafruit.com/viewtopic.php?f=19&t=93211

There is also a latency timer built into the chip that FTDI's techs say can be set as low as 1ms. Not internally sure what that does.

I'll order one and play around with it to see what kind of results I can get.

CthulhuLabs · 26 Apr 2023, 14:05

I just got an Adafruit FT232H Breakout board. I am realizing that testing this is not as easy as I first though. Most of the simple microcontroller firmwares like Arduino / Circuit Python lack direct support for SPI Slave mode on the newer microcontrollers. They all seem to think that people only want to use their microcontroller as the SPI Master. That said it looks like the original AVR based Arduino's have good documentation on how to initialize SPI Slave through the SPI Registers. I have an Arduino MEGA from my original RAMPS 1.4 controller from 2014. Hopefully it is still working and I can get the SPI Bus speed up to 8Mhz in Slave mode.

My plan is to have the FT232H enable the CS and then transmit to the MEGA the current timestamp with as high of accuracy as I can get. It will then disable the CS. It will then wait for the MEGA to set its interrupt pin high. The MEGA will sleep for 1ms and then set its interrupt pin high. The FT232H will then enable the CS and read from the MEGA the last timestamp. It will then disable the CS and finally print on the CLI the current timestamp, the timestamp it received from the MEGA, and the difference minus 1ms (accounting for the MEGA's programmed delay). This should be the latency. I will have this whole process running in a loop. Depending on how much latency I am seeing, Ill probably have it loop like this 10 times a second. Then I will run this while doing other things on the USB bus, (moving a mouse, typing, accessing a USB2 thumb drive, accessing a USB3 thumb drive, etc.) Ill also test this with the FT232H going through a hub as well as being directly connected to the motherboards USB host controller. Ill take the output and generate some nice pretty graphs.

@chrishamm & @T3P3Tony does this sound like a valid test? Any recommendations on things to change?