Duet Pi Regression - How?



  • If a person chooses to regress DSF on a Pi, what is the most automated way? So far, I've come up with this:

    First - backup your files in /opt/dsf/sd/*, just in case.

    Then run:
    sudo apt remove duetsoftwareframework -y
    sudo apt autopurge -y
    sudo apt install duetsoftwareframework=1.2.3.0 -y

    Restore any files that went missing or overlay.

    AND, of course, if the board won't talk, bossa with the firmware found at /opt/dsf/sd/sys/Duet3Firmware_MB6HC.bin". If it will talk, the 'apt' process should have loaded at least the main board when it installed the 'duetfirmware' package.

    Thoughts?



  • Specifying a specific version for only DSF never has worked as apt seems to insist on installing the most recent version for all of DSF's dependencies which aren't compatible. Maybe aptitude can do a better job of it?

    also i think you need to remove duetruntime or that was a half finished thought. but seems autopurge might be excessive as it could remove configs from non duet packages; if purge is a goal i'd make sure to backup sd folder and specify apt purge duetcontrolserver duetruntime duetsd duetsoftwareframework duettools duetwebcontrol duetwebserver reprapfirmware (even though it may be subject to change if dsf gets new packages over time)

    pi@raspberrypi:~ $ sudo apt remove duetsoftwareframework -y
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    Package 'duetsoftwareframework' is not installed, so not removed
    0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
    pi@raspberrypi:~ $ sudo apt autopurge -y
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
    pi@raspberrypi:~ $ sudo apt install duetsoftwareframework=1.2.3.0 -y
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    distribution that some required packages have not yet been created
    or been moved out of Incoming.
    The following information may help to resolve the situation:
    
    The following packages have unmet dependencies:
     duetsoftwareframework : Depends: duetcontrolserver (= 1.2.3.0) but 2.1.1 is to be installed
                             Depends: duetsd (= 1.0.5) but 1.0.6 is to be installed
                             Depends: duettools (= 1.2.3.0) but 2.1.1 is to be installed
                             Depends: duetwebserver (= 1.2.1.0) but 2.1.0 is to be installed
                             Depends: duetwebcontrol (= 2.0.4-1) but 2.1.5 is to be installed
                             Depends: reprapfirmware (<= 1.2.3.0-999) but 2.1.1-1 is to be installed
    E: Unable to correct problems, you have held broken packages.
    

    this post has a 95% shortcut to get an apt command to deal with the issue

    https://forum.duet3d.com/post/149932

    pi@raspberrypi:~ $ bash dsfdep.sh 1.2.3.0
    sudo apt install \
      duetcontrolserver=1.2.3.0 \
      duetruntime=1.2.3.0 \
      duetsd=1.0.5 \
      duettools=1.2.3.0 \
      duetwebserver=1.2.1.0 \
      duetwebcontrol=2.0.4-1 \
      reprapfirmware=1.2.3.0-1 \
      duetsoftwareframework=1.2.3.0
    pi@raspberrypi:~ $ `bash dsfdep.sh 1.2.3.0` #note the backticks
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    The following NEW packages will be installed:
      duetcontrolserver duetruntime duetsd duetsoftwareframework duettools duetwebcontrol duetwebserver reprapfirmware
    0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
    Need to get 32.2 MB of archives.
    After this operation, 0 B of additional disk space will be used.
    Get:1 https://pkg.duet3d.com unstable/armv7 armhf duetruntime armhf 1.2.3.0 [28.7 MB]
    Get:2 https://pkg.duet3d.com unstable/armv7 armhf duetcontrolserver armhf 1.2.3.0 [173 kB]
    Get:3 https://pkg.duet3d.com unstable/armv7 armhf duetsd all 1.0.5 [1,998 B]
    Get:4 https://pkg.duet3d.com unstable/armv7 armhf duettools armhf 1.2.3.0 [49.9 kB]
    Get:5 https://pkg.duet3d.com unstable/armv7 armhf duetwebserver armhf 1.2.1.0 [70.7 kB]
    Get:6 https://pkg.duet3d.com unstable/armv7 armhf duetwebcontrol all 2.0.4-1 [2,811 kB]
    Get:7 https://pkg.duet3d.com unstable/armv7 armhf reprapfirmware all 1.2.3.0-1 [425 kB]
    Get:8 https://pkg.duet3d.com unstable/armv7 armhf duetsoftwareframework armhf 1.2.3.0 [1,710 B]
    Fetched 32.2 MB in 48s (671 kB/s)
    Selecting previously unselected package duetruntime.
    (Reading database ... 52338 files and directories currently installed.)
    Preparing to unpack .../0-duetruntime_1.2.3.0_armhf.deb ...
    Unpacking duetruntime (1.2.3.0) ...
    Selecting previously unselected package duetcontrolserver.
    Preparing to unpack .../1-duetcontrolserver_1.2.3.0_armhf.deb ...
    Unpacking duetcontrolserver (1.2.3.0) ...
    Selecting previously unselected package duetsd.
    Preparing to unpack .../2-duetsd_1.0.5_all.deb ...
    Unpacking duetsd (1.0.5) ...
    Selecting previously unselected package duettools.
    Preparing to unpack .../3-duettools_1.2.3.0_armhf.deb ...
    Unpacking duettools (1.2.3.0) ...
    Selecting previously unselected package duetwebserver.
    Preparing to unpack .../4-duetwebserver_1.2.1.0_armhf.deb ...
    Unpacking duetwebserver (1.2.1.0) ...
    Selecting previously unselected package duetwebcontrol.
    Preparing to unpack .../5-duetwebcontrol_2.0.4-1_all.deb ...
    Unpacking duetwebcontrol (2.0.4-1) ...
    Selecting previously unselected package reprapfirmware.
    Preparing to unpack .../6-reprapfirmware_1.2.3.0-1_all.deb ...
    Unpacking reprapfirmware (1.2.3.0-1) ...
    Selecting previously unselected package duetsoftwareframework.
    Preparing to unpack .../7-duetsoftwareframework_1.2.3.0_armhf.deb ...
    Unpacking duetsoftwareframework (1.2.3.0) ...
    Setting up duetruntime (1.2.3.0) ...
    Setting up duetwebcontrol (2.0.4-1) ...
    Setting up duetcontrolserver (1.2.3.0) ...
    Setting up duettools (1.2.3.0) ...
    Setting up duetwebserver (1.2.1.0) ...
    Setting up duetsd (1.0.5) ...
    Setting up reprapfirmware (1.2.3.0-1) ...
    [error] Duet is not available
    Setting up duetsoftwareframework (1.2.3.0) ...
    pi@raspberrypi:~ $
    
    
    


  • To the autopurge point, yah, I'd like to avoid that for exactly those two reasons: removes 'non package' files and could remove anything that apt thinks it should, outside the duet universe.

    At the same time, I thought it would create a state where apt install with version number didn't do the "broken" thing. Obviously, it does not.

    So... part of the reason for asking this question was to see if regression is worth scripting. Something that spits out all the commands, much like depcheck. Probably is...



  • Why is that 95%? Or what is the 5% edge case?



  • @Danal said in Duet Pi Regression - How?:

    So... part of the reason for asking this question was to see if regression is worth scripting. Something that spits out all the commands, much like depcheck. Probably is...

    afaik the only flaw is the recursive checking i limited to duetframework because it was the only package without a direct dependency to dsf (i.e. if that changes it fails, but then apt will complain so safe-ish)



  • @bearer said in Duet Pi Regression - How?:

    @Danal said in Duet Pi Regression - How?:

    So... part of the reason for asking this question was to see if regression is worth scripting. Something that spits out all the commands, much like depcheck. Probably is...

    afaik the only flaw is the recursive checking i limited to duetframework because it was the only package without a direct dependency to dsf

    Thanks!



  • and purging duetsd will clear out the most of /opt/dsf/sd folder configs, macros, gcode, everything.



  • @Danal said in Duet Pi Regression - How?:

    Why is that 95%? Or what is the 5% edge case?

    oh, and there is something weird with the reprapfirmeware n.n.n.n-1 and n.n.n.n-999 dependenices for otherwise identical versions. so i ignore the 999 in the rexeg



  • @bearer said in Duet Pi Regression - How?:

    and purging duetsd will clear out the whole /opt/dsf/sd folder configs, macros, gcode, everything.

    Agreed. I'm now on the warpath to avoid 'apt purge'.

    'apt remove' only removes things in the package manifest, and deletes empty directories. If non empty, it will leave them. I believe the one time I regressed (by a manual method) that everything stayed intact, EXCEPT config.g had been overlaid with the empty-ish distribution copy. All other files, like tfree and tpost and so forth, and macros, they were all intact.



  • @bearer said in Duet Pi Regression - How?:

    @Danal said in Duet Pi Regression - How?:

    Why is that 95%? Or what is the 5% edge case?

    oh, and there is something weird with the reprapfirmeware n.n.n.n-1 and n.n.n.n-999 dependenices for otherwise identical versions. so i ignore the 999 in the rexeg

    Me too.



  • With regard to "how to recurse" without depending on the exact structure of the package, this seems to work, then parse the "depends" line:

    apt show duetsoftwareframework=1.2.3.0
    Package: duetsoftwareframework
    Version: 1.2.3.0
    Priority: standard
    Section: electronics
    Maintainer: Duet3D Packaging Authority <pkg@duet3d.com>
    Installed-Size: unknown
    Depends: duetcontrolserver (= 1.2.3.0), duetsd (= 1.0.5), duettools (= 1.2.3.0), duetwebserver (= 1.2.1.0), duetwebcontrol (= 2.0.4-1), reprapfirmware (>= 1.2.3.0-1), reprapfirmware (<= 1.2.3.0-999)
    Homepage: https://pkg.duet3d.com
    Vcs-Git: https://github.com/chrishamm/DuetSoftwareFramework.git
    Download-Size: 1,710 B
    APT-Manual-Installed: yes
    APT-Sources: https://pkg.duet3d.com unstable/armv7 armhf Packages
    Description: Meta package for the full Duet software framework
    


  • @Danal said in Duet Pi Regression - How?:

    this seems to work,

    missing duetruntime which is why i turne to recusion that ultimately failed if left run to to all the leaf nodes.



  • Hmmm... I wonder how it gets installed? With no dependency?



  • @Danal said in Duet Pi Regression - How?:

    I wonder how it gets installed? With no dependency?

    duetcontrolserver has it as a dependency iirc (thus the recursion)



  • Ah, light bulb just came on. Thank you.



  • And, by the way, autopurge (which I am NOT going to use) actually does not purge the recursive stuff after a

    sudo apt install duetsoftwareframework=1.2.3.0 duetcontrolserver=1.2.3.0 duetruntime=1.2.3.0 duetsd=1.0.5 duettools=1.2.3.0 duetwebserver=1.2.1.0 duetwebcontrol=2.0.4-1 reprapfirmware=1.2.3.0-1

    style install, and an apt remove duetsoftwareframework.

    So.... it would not have worked anyway.



  • Just switch to RPM 😂



  • @gtj0 said in Duet Pi Regression - How?:

    Just switch to RPM 😂

    +2

    was wondering if aptitude* would be better suited, but decided to brush up on regex instead 😇

    *) not better than RPM..



  • @gtj0 said in Duet Pi Regression - How?:

    Just switch to RPM 😂

    How about //STEP1 EXEC PGM=SMPE? It handled prereqs, coreqs, and even postreqs flawlessly. Why, I've heard you could build and entire OS with it!



  • @Danal said in Duet Pi Regression - How?:

    and an apt remove duetsoftwareframework.

    apt remove duet* reprapfirmware is about the same length and oddly more specific. would work with remove and purge for that matter.



  • @bearer said in Duet Pi Regression - How?:

    @Danal said in Duet Pi Regression - How?:

    and an apt remove duetsoftwareframework.

    apt remove duet* reprapfirmware is about the same length and oddly more specific. would work with remove and purge for that matter.

    Yah, I thought about the nicely structured names. As you said though, if the package changes...



  • @Danal said in Duet Pi Regression - How?:

    As you said though, if the package changes...

    then it has a greater chance of catching it as duetruntime still doesn't depend on duetsoftwareframework.

    and if apt remove duetcontrolserver duetruntime duetsd duetsoftwareframework duettools duetwebcontrol duetwebserver reprapfirmware is the alternative then it needs to be updated anyways.

    to be honest I have more faith in a * than apt and dependencies defined for the packages should autoremove fail.



  • OK, here is a "first draft" that works. Comments welcome:

    ./DuetRegress.sh 2.1.0
    Currently Installed duetsoftwareframework=
    Command line argument is requesting duetsoftwareframework=2.1.0
    Which has associated reprapfirmware of 2.1.0-1 also known as 3.01-RC9
    
    ***********************************************************************************
    * Note: The next step will run for a while; the script is (probably) not hung.    *
    ***********************************************************************************
    
    To regress to DSF version 2.1.0 which is RRF version 3.01-RC9:
    1) Backup your files via:
      mkdir ~/duetback
      cp -r /opt/dsf/sd/* ~/duetback
    or similar. Verify that the backup worked: 'ls ~/duetback' should show directories: 'filaments firmware gcodes macros sys' or similar.
    
    2) Run this command: (it is long, be sure you get it all on one line)
      sudo apt -y  --allow-downgrades install duetsoftwareframework=2.1.0 duetcontrolserver=2.1.0 duetruntime=2.1.0 duetsd=1.0.6 duettools=2.1.0 duetwebcontrol=2.1.4 duetwebserver=2.1.0 reprapfirmware=2.1.0-1
    
    3) Verify system is running properly, web interface opens, etc
    
    4) If Pi runs, but won't connect to Duet Board, bossa the board
       with the file found at /opt/dsf/sd/sys/Duet3Firmware_MB6HC.bin
       Instructions here:
       https://duet3d.dozuki.com/Wiki/Getting_Started_With_Duet_3#Section_Updating_Duet_3_main_board_firmware
    
    5) Check files like config.g, macros, etc, and restore as required.
    

    There are no name or other recursion dependencies in the script. Also, I took out the "apt remove" command, I don't believe it is required, apt will just overlay things.



  • i opted for splitting the lines so it can be easy to read and copy, or if piped/backticked its not.

    i'd also reconsider the -y bit gives apt a chance to complain if it feels the need.

    pi@raspberrypi:~ $ bash rrfdep.sh 3.01-RC9
    sudo apt install \
      duetcontrolserver=2.1.0 \
      duetsd=1.0.6 \
      duettools=2.1.0 \
      duetwebserver=2.1.0 \
      duetwebcontrol=2.1.4 \
      reprapfirmware=2.1.0-1 \
      duetsoftwareframework=2.1.0
    pi@raspberrypi:~ $
    

    don't think i'll put up the version that resolves it from the reprap firmware version, smells like lots of unnecessary load on servers just to save people looking up a table



  • Excellent idea on the lines! I will do that. In fact, I was a bit concerned about line length... not only that, but I'm flattening a 'list' now, all I have to do is not do that.

    I go back and forth on the -y. Given that this is intended to be used by people who literally can't even answer the initial "Do you want me to do what you asked? [Y/n]" prompt... well... I'm inclined to leave it in.

    re: servers, yeah, this one is entirely local. In fact, it ought to insist on a recent (whatever that means) sudo apt update.


Log in to reply