Suddenly all files missing from DWC (Duet3+Rpi)

luckyflyer

This has happened twice now. For no apparent reason all of the files are missing on system page. A rotating notice searching for files (or something like that) is displayed. I'm thinking this is a pi issue, but not sure, cycling power restores the files. I'm running a pi4 with a new sandisk 32 gb card and Duet 3 with latest software. Tonight I imaged the card to another sandisk 32gb card, I'll wait and see if it does it again. Anyone else had this happen, solutions?

Danal

Next time, see if you can ssh into the Pi. If so:

ls -al /opt/dsf
ls -al /opt/dsf/sd
ls -al /opt/dsf/sd/www
ls -al /opt/dsf/sd/sys

If those look normal, see below. If they look like files are missing and a reboot brings them back, then you have an SD card problem, or a Unix problem, or similar.

After checking the directories, Depending on how hard you want to research this yourself, you may want to scan the logs. Problem is, I can't really tell you what you are looking for... just anything that looks like an error. Logs are available via:
sudo journalctl -u duetsoftwarecontrol

If dirs and the log look more or less normal, then try:
sudo service duetsoftwarecontrol restart

If that fixes it, there is strong evidence that it is a DSF bug. If not fixed, the question is more open.

And, finally, if none of the above works:
sudo reboot

Danal

Oh, and no, I haven't had files disappear.

I have had a few errors logged, and/or just plain "hang", all of which were resolved by reseating the ribbon cable between the D3 and the Pi.

luckyflyer

I'm hoping it's just a flaky sd card, if it does it again I'll do your recommend checks and report back.

A Former User

$0.02; odds that only /opt/dsf/ is affected by a dodgy SD card sounds slim.

Danal

@bearer said in Suddenly all files missing from DWC (Duet3+Rpi):

$0.02; odds that only /opt/dsf/ is affected by a dodgy SD card sounds slim.

Completely agree... at the same time, we don't know that was the only effect. I'm assuming that the entire SD card became inaccessible, yet the in-memory duetweb and control processes kept running. It is possible that the first visible effect of that scenario in DWC would be that the tabs that manipulate files in sys, macros, and jobs, would look as though the files were missing. I believe that OP checked sys (implicity), but not the others. Probably DWC was too hung to check anything else.

-and/or-

That the Pi could have been entirely hung, but the in-browser javascript's only visible symptom was the rotating file searcher.

luckyflyer

I put the replacement sd card in last night and will run it 24/7, so far so good.

luckyflyer

Tonight I was going to run the extruder motor from panel due, I entered the speed and length and hit extrude and got a t1 error. Realizing I had not set tool temperature I tried to set the temp from paneldue but couldn't. I came in to where the pc is and tried to set temperature and couldn't, went to system page where refresh is spinning and the searching for files message is showing. Here is the download recommend from above.

pi@duet3:~ $ ls -al /opt/dsf
total 96
drwxr-xr-x 6 pi   pi    4096 Nov  5 09:16 .
drwxr-xr-x 4 root root  4096 Nov  5 09:16 ..
drwxr-xr-x 2 pi   pi   73728 Mar 19 12:17 bin
drwxr-xr-x 2 pi   pi    4096 Mar 19 12:17 conf
drwxr-xr-x 5 pi   pi    4096 Mar 23 19:24 dwc2
drwxr-xr-x 6 pi   pi    4096 Nov  5 09:16 sd


pi@duet3:~ $ ls -al /opt/dsf/sd
total 24
drwxr-xr-x 6 pi pi 4096 Nov  5 09:16 .
drwxr-xr-x 6 pi pi 4096 Nov  5 09:16 ..
drwxr-xr-x 2 pi pi 4096 Sep 23  2019 filaments
drwxr-xr-x 2 pi pi 4096 Sep 23  2019 gcodes
drwxr-xr-x 2 pi pi 4096 Sep 23  2019 macros
drwxr-xr-x 2 pi pi 4096 Mar 26 11:04 sys
lrwxrwxrwx 1 pi pi    7 Oct 22 03:00 www -> ../dwc2


pi@duet3:~ $ ls -al /opt/dsf/sd/www
lrwxrwxrwx 1 pi pi 7 Oct 22 03:00 /opt/dsf/sd/www -> ../dwc2


pi@duet3:~ $ ls -al /opt/dsf/sd/sys
total 1480
drwxr-xr-x 2 pi   pi     4096 Mar 26 11:04 .
drwxr-xr-x 6 pi   pi     4096 Nov  5 09:16 ..
-rw-r--r-- 1 root root    261 Mar 24 08:42 bed.g
-rw-r--r-- 1 root root   3857 Mar 26 11:04 config.g
-rw-r--r-- 1 root root   3857 Mar 26 10:59 config.g.bak
-rw-r--r-- 1 root root   3490 Mar 24 08:42 config.json
-rw-r--r-- 1 root root    221 Mar 24 07:55 deployprobe.g
-rw-r--r-- 1 root root  10324 Mar 16 07:11 Duet3Bootloader-EXP3HC.bin
-rw-r--r-- 1 root root   9916 Mar 16 07:11 Duet3Bootloader-TOOL1LC.bin
-rwxr-xr-x 1 root root 157308 Mar 16 07:11 Duet3Firmware_EXP3HC.bin
-rwxr-xr-x 1 root root 560772 Mar 16 07:11 Duet3Firmware_MB6HC.bin
-rwxr-xr-x 1 root root 408648 Mar 16 07:11 Duet3Firmware_MBP05.bin
-rw-r--r-- 1 root root 160204 Mar 16 07:11 Duet3Firmware_TOOL1LC.bin
-rwxr-xr-x 1 root root  33804 Mar 16 07:11 Duet3iap_spi_MB6HC.bin
-rwxr-xr-x 1 root root  33804 Mar 16 07:11 Duet3iap_spi_MBP05.bin
-rwxr-xr-x 1 root root  19908 Mar 16 07:11 Duet3_SBCiap_MB6HC.bin
-rwxr--r-- 1 root root  19908 Mar 16 07:11 Duet3_SBCiap_MBP05.bin
-rw-r--r-- 1 root root   1236 Mar 26 09:48 dwc2settings.json
-rw-r--r-- 1 root root   1023 Mar 24 08:42 homeall.g
-rw-r--r-- 1 root root    521 Mar 24 08:42 homex.g
-rw-r--r-- 1 root root    521 Mar 24 08:42 homey.g
-rw-r--r-- 1 root root    422 Mar 24 18:33 homez.g
-rw-r--r-- 1 root root    397 Mar 24 08:42 pause.g
-rw-r--r-- 1 root root    400 Mar 24 08:42 resume.g
-rw-r--r-- 1 root root    224 Mar 24 07:55 retractprobe.g
-rw-r--r-- 1 root root    175 Mar 24 08:42 sleep.g
-rw-r--r-- 1 root root    207 Mar 24 08:42 stop.g
-rw-r--r-- 1 root root    162 Mar 24 08:42 tfree0.g
-rw-r--r-- 1 root root    222 Mar 24 08:42 tpost0.g
-rw-r--r-- 1 root root    166 Mar 24 08:42 tpre0.g


pi@duet3:~ $ sudo journalctl -u duetsoftwarecontrol
-- Logs begin at Thu 2020-03-26 17:17:01 AKDT, end at Thu 2020-03-26 18:31:34 AK
-- No entries --


pi@duet3:~ $ sudo service duetsoftwarecontrol restart
Failed to restart duetsoftwarecontrol.service: Unit duetsoftwarecontrol.service not found.
pi@duet3:~ $

A power cycle returned it to normal.
This is way above my pay grade.
Tia

Danal

Well... my apologies. I managed to typo the last two commands (the log and the restart). They should have referred to 'duetcontrolserver'

Also, lets do just these two commands. The journalctl I originally gave will produce many pages of output... too much... so let's just do these two:

sudo service duetcontrolserver status
sudo service duetcontrolserver restart

Despite that, we did get some data. We know the Pi itself was up and running, and that the Pi could see its own SD card. So we can pretty much rule out any further issues with the SD card itself.

Next time this happens, do just those two commands above, and lets see what we get.

luckyflyer

Sounds good, thank you sir.