Possible FTP Upload Problem

fcwilt

@owend said in Possible FTP Upload Problem:

@fcwilt
I found lots of similar issues (empty files) when using FileZilla as the client.
I think the main problem is that the duet will only accept a single connection/transfer and most modern ftp clients try to establish multiple concurrent connections.
I think I had to also limit the transfer rate.

I thought it might be something like that. The FTP client I am hoping to use is part of my favorite editor.

I don't see any settings for the FTP client part of the editor that mention connections or transfer speed.

I'm seeing the problem with three different FTP clients so I think the problem lies in the Duet code.

Thanks.

Frederick

chrishamm

@fcwilt The FTP server in RepRapFirmware is very limited. You must limit the number of concurrent transfers to 1, else certain problems will show up (although I wouldn't have thought empty files would be a symptom). WinSCP has that setting:

And so does FileZilla:

fcwilt

@chrishamm

I installed FileZilla Pro and made the settings you mentioned regards number of transfers.

Doing a upload from FileZilla:

I still got empty files on the Duet.
Once there was an empty file it could not be overwritten but had to be deleted before the non-empty file could be uploaded.

I did several tests.

Hopefully there is a bug that can be found and fixed in the Duet?

Thanks.

Frederick

Phaedrux

@chrishamm It happens even with the FTP client set to a single file at a time.

It's a bit insidious, since the file will be created, but it is 0 bytes. So unless you're checking the file size you can miss that there is a problem.

chrishamm

@fcwilt What Duet board are you using?

fcwilt

@chrishamm said in Possible FTP Upload Problem:

@fcwilt What Duet board are you using?

I am using a variety: Duet 2 WiFi/Duex 5 combo, Duet 3 Mini 5+ and Duet 3 6HC/3HC combo.

The Duet 2s are running 3.4.0.

The Duet 3 Mini 5+ and the Duet 3 6HC/3HC are running 3.4.1

At the risk of repeating myself, the things I am seeing:

files of size 0
files of size 0 that could not be updated via FileZilla - they had to be deleted. FileZilla reported no errors.
files of truncated size

To date I have not found any pattern.

Frederick

Phaedrux

Uploading these seems to be a good way to provoke it.

https://forum.duet3d.com/topic/6181/tuning-macros-menus-accel-jerk-retraction-pressure-advance

Sometimes they look like they are full size files, but on refreshing the directory they are 0 byte.

fcwilt

@chrishamm

I contacted the developer of the editor I am using that has the FTP client built-in.

He said it always uses just a single connection and always transfer just one file at a time.

Frederick

wschadow

I use filezilla in many cases But together with teh Duet boards I found that WinSCP is a bit more reliable. I have not looked into details.

chrishamm

@fcwilt I could reproduce those problems on a Duet 3 Mini 5+ WiFi and on a MB6HC. For both board types I have firmware fixes ready but I still need to check if the Duet 2 Ethernet is affected, too.

fcwilt

@chrishamm said in Possible FTP Upload Problem:

@fcwilt I could reproduce those problems on a Duet 3 Mini 5+ WiFi and on a MB6HC. For both board types I have firmware fixes ready but I still need to check if the Duet 2 Ethernet is affected, too.

Would the Duet 2 WiFi also be fixed by the changes for a Duet 2 Ethernet?

Frederick

chrishamm

@fcwilt Yes.

paunboli2

@chrishamm I'm also getting problem with old file explorer

chrishamm

@fcwilt @Phaedrux @wschadow You can find my updated v3.4.1+1 firmware builds for standalone mode here: https://pkg.duet3d.com/RRF-3.4.1plus1.zip Please let me know if they fix the FTP transfer problems. The issue where uploading config.g resulted in an empty file may be fixed by these builds, too.

fcwilt

@chrishamm said in Possible FTP Upload Problem:

@fcwilt @Phaedrux @wschadow You can find my updated v3.4.1+1 firmware builds for standalone mode here: https://pkg.duet3d.com/RRF-3.4.1plus1.zip Please let me know if they fix the FTP transfer problems. The issue where uploading config.g resulted in an empty file may be fixed by these builds, too.

I'm out of town for a few days.

I will test this new firmware as soon as I return home.

Thanks much.

Frederick

OwenD

@chrishamm
I just tried the following using FileZilla set to single connection and transfer limited to 500kb/s
On past usage all of these tests gave multiple zero kb files both directions

Bulk upload of all my filaments (25 directories each with 3 files) - OK
Bulk download of same - OK
Bulk download of complete system (700 files in 25 directories) - Only one failed transfer.
File was 0/www/css/OnScreenKeyboar
It was not 0kb, it failed to transfer. Re-queuing worked.
FileZilla didn't give a clear reason why the transfer of that file failed, but as it was reported I would call the test successful.

droftarts

@owend Was that with @chrishamm's updated firmware?

Ian

OwenD

@droftarts said in Possible FTP Upload Problem:

@owend Was that with @chrishamm's updated firmware?

Yes, it was.

fcwilt

@chrishamm

I finally returned home and tested the new FTP code.

On all of my printers (two Duet 2 WiFi, one 6HC and one Mini 5):

I was able to download all of the files from the SYS folder via my FTP client
I was able to upload all of the files to the SYS folder via my FTP client

No sign of file loss or truncation.

I will now continue with my normal activities using FTP and see if I encounter any problems.

Frederick

fcwilt

@chrishamm

I have returned to using my favorite editor in, for lack of a official term, FTP mode, where files are automatically download/uploaded as they are opened for editing and then saved when editing is done.

So far it seems to be working fine - though I have been keeping my fingers crossed.

JOOC what was it that you found and seem to have fixed?

Frederick