Completely replace SD Card? (duet ethernet)

T3P3Tony · 22 Feb 2018, 21:37

We have been using FAT32 for the 4GB cards. I was not actually aware of the FAT16 performance increase that David mentioned.

Something to change on the next batch.

Phaedrux · 22 Feb 2018, 21:52

One final data point then.

[c]
Transcend 4gb Class 4 FAT16 with 64kb clusters 00:50
[/c]

10 seconds off the FAT32 format.

So there you go. For best speeds the stock card formatted at FAT16 with 64k clusters is the way to go. This might require you to reformat the card yourself though, as out of the box mine was FAT32 and 32k clusters.

If you need a larger card, FAT32 with 64kb clusters and A1 classification.

I may try again with an ideal wireless situation with the router directly line of site to the printer and my laptop on a wired connection, just to isolate the SD card performance. In actual practice there isn't much difference.

EasyTarget · 22 Feb 2018, 22:04

Sorry to be pedantic, would using fAT16 provide a significant upload speed boost?

If I am driving prints remotely (http xfer gcode to Duet, start print remotely, from a script on a server that it will automatically remove completed print files too); Would it provide a useful/noteworthy speed boost to the upload If I reformatted my card to FAT16.

(I'm suggesting not using the whole drive.. eg just force a FAT16 filesystem onto the first four gigs and ignore the rest.. I've done something similar to this before.)

Edit: Looking at the above.. I guess not.

T3P3Tony · 22 Feb 2018, 23:42

@Phaedrux:

One final data point then.

[c]
Transcend 4gb Class 4 FAT16 with 64kb clusters 00:50
[/c]

10 seconds off the FAT32 format.

So there you go. For best speeds the stock card formatted at FAT16 with 64k clusters is the way to go. This might require you to reformat the card yourself though, as out of the box mine was FAT32 and 32k clusters.

If you need a larger card, FAT32 with 64kb clusters and A1 classification.

I may try again with an ideal wireless situation with the router directly line of site to the printer and my laptop on a wired connection, just to isolate the SD card performance. In actual practice there isn't much difference.

Thanks for running the tests!

Phaedrux · 23 Feb 2018, 02:31

@EasyTarget:

Sorry to be pedantic, would using fAT16 provide a significant upload speed boost?

If I am driving prints remotely (http xfer gcode to Duet, start print remotely, from a script on a server that it will automatically remove completed print files too); Would it provide a useful/noteworthy speed boost to the upload If I reformatted my card to FAT16.

(I'm suggesting not using the whole drive.. eg just force a FAT16 filesystem onto the first four gigs and ignore the rest.. I've done something similar to this before.)

Edit: Looking at the above.. I guess not.

Short stroking a drive like that used to be popular with spinning platters back when they topped out at 120ish gigs, but for an SD card it might actually degrade performance further due to the wear leveling algorithm having less space to work with. If there is some performance to be gained from FAT16 it would probably be lost from that.

As it is, I think unless you have a really poor SD card, you're probably more limited by wireless transfer than write speed.

dc42 · 23 Feb 2018, 07:51

The wear levelling algorithm works at a level underneath the FAT format, so it shouldn't be affected.

dc42 · 23 Feb 2018, 13:57

I have tried to reproduce this issue, but so far without success. I copied a 384Mbyte file over wifi to the Duet. The speed started at over 1Mbyte/sec but had dropped to 657kbytes/sec by the time it finished. DWC shows file information for it. Then I tacked half a megabyte of useless comments on the end so that DWC would have to search a long way back from the end of the file without finding what it wanted, and uploaded that. Again, no problems but file info is not displayed because the Duet does not search back as far as half a megabyte from the end. When I refresh the file list, it pauses on that file for 4 seconds, but carries on after that.

So I conclude that you can use files up to at least 385Mbytes long if your choice of SD card and format is suitable. This card is a Sandisk 16Gb card with a class 4 speed rating, formatted to FAT32 with 64Kb clusters. In DWC I have the status update interval set to 250ms which I think is the default, and the maximum number of AJAX retries set to 1 which is wrong - I recommend at least 3 when using WiFi.

EasyTarget · 23 Feb 2018, 14:09

Thanks for the feedback, and the testing! super helpful

I'm certainly not going this route for a modest gain, I guess I was optimistically hoping for a dramatic (2x or whatever) speedup

Once this weekend's re-organising and wiring is complete I should have my Pi acting as the AP for my print cupboard, I just, errm, 'acquired' a 10m CAT5 cable for this purpose from the nest at work. This will put the AP ~10cm from the Duet in line-of-sight; so I'm hoping to push the speed if I can, and see if this solves the disconnects that plague me from webcontrol. (yes, I know that may be too close; I'll play with separating them if I see weirdness before complaining here)

dc42 · 23 Feb 2018, 16:12

If you are getting disconnects, check that you have the retry count set to at least 3 in DWC.

EasyTarget · 23 Feb 2018, 19:23

20 enough?I've seen all the advice, but since I have a signal strength that flaps between -50 and -70 I'll wait till that is stable before passing judgement.. at the moment I'm happy to continue blaming my end of the network.

dc42 · 23 Feb 2018, 19:26

With the signal strength changing that much, I suspect you are getting interference from another device in the same band. Have you tried changing the WiFi channel?

SpoonUnit · 10 Aug 2018, 12:44

Just found something odd. I have near identical printers setup, one which has a Duet Wifi, and one which is a Duet Ethernet. Using DWC to upload a 30MB file shows the performance is severely different between the two, but not in a way that I would expect. The Wifi connected Duet can be uploaded to significantly quicker. The Duet Wifi seems to top out at around 580k/sec, whereas the Ethernet only gets to about 340k/s. Could this be a feature of the firmware on the Ethernet version? Does anyone else have similar results?

The SD card in both is a brand new 16GB SanDisk Ultra SDHC Class 10.

T3P3Tony · 10 Aug 2018, 12:57

@spoonunit are both SD cards formatted with 64 KB blocks? That can make a difference.

dc42 · 8 Oct 2018, 13:40

You can run M39 to check the cluster size.

The other thing that can make a difference is fragmentation of the SD card. A heavily-fragmented SD card tends to slow down the speed of writing to it.

In my tests, both WiFi (with a good signal strength)and Ethernet top out at about 1Mbps if the SD card has a large cluster size and is not heavily fragmented. For very large files it drops to around 800kbytes/sec.

SpoonUnit · 10 Aug 2018, 14:12

Will double check. Both were straight out the packet, so I assume they're identically formatted. Assumptions can be wrong though.

dc42 · 10 Aug 2018, 14:14

It may be worth checking that the Ethernet physical link has connected at 100Mbps, not 10Mbps.

SpoonUnit · 10 Aug 2018, 14:14

Ethernet machine

M39
SD card in slot 0: capacity 15.93Gb, free space 15.89Gb, speed 20.00MBytes/sec, cluster size 32kb

Wifi Machine

M39
SD card in slot 0: capacity 15.93Gb, free space 15.92Gb, speed 20.00MBytes/sec, cluster size 32kb

SpoonUnit · 8 Oct 2018, 14:23

@dc42 Is there M command to check link speed established?

M552
Network is enabled, configured IP address: 0.0.0.0, actual IP address: 192.168.XXX.155

Looking at the router, the 100M light is lit. The cable went from duet to cable to a connector on the back of the box. I've plugged the cable in directly to the Duet to rule out that short cable, but the longer cable could in theory still be to blame.

dc42 · 10 Aug 2018, 15:44

@spoonunit said in Completely replace SD Card? (duet ethernet):

@dc42 Is there M command to check link speed established?

Not yet, but I'll add it to the network section of the M122 report in the forthcoming 2.02beta1 release.

SpoonUnit · 17 Aug 2018, 09:34

I ran some more tests to see if I could find an issue with the networking but ultimately it seems fine.

For clarity, when I originally reported the issue, I had the Duet connected to a short extension which provides an ethernet port at the back of the box. From there a cable to 10/100 switch (with a light on signifying that the Duet has a 100 link). The switch then goes to a power-line network plug, which then emerges into another switch and finally to the PC where I was testing.

To rule out all the gubbins in the middle, I took a laptop to the room with the printer in, took out the cable from the back of the printer and plugged it into the laptop. I then ran a speedtest to just check what throughput the cable should be able to receive through the powerline networking. It's less than stellar to be honest, but even so it can get 28Mbps, or about 3MB/s which should be more than enough to pump the full potential 660K/s that I see the wireless Duet able to pull.

Next stop was to take the powerline networking out of the equation, so I configured the laptop as a wireless bridge. I then connected the laptop directly to the printer and over that single cable still found the upload restricted, whether accessed directly from the laptop (single cable), or over the bridged networking.

It's not a huge issue really; uploads still work. I just feel they should be much faster on ethernet than wireless, but definitely at least equal. A 100Mbps link ought to be able to copy in at 10MB/s or more, but I guess there are constraints with the electronics, firmware, and SD card that are constraining that. Will the new Duet boards in development offer higher throughput from ethernet?