Another weighing filament holder

[achrn]

In what is undoubtedly the biggest sofware release of the day I have implemented another filament weighing plugin for a system with an attached Pi.

Actually, half of it is a direct lift of Resam's plugin described at https://forum.duet3d.com/post/241194. I have taken the DWC plugin from that but written a new python DSF routine. Mine is for a HX711 directly connected to the Pi. Resam reports getting noisy / faulty readings with an HX711 connected directly, but I get very few - this may be subject to all sorts of vagaries of wiring etc.

My DSF routine uses a simple IQR-based filter to disregard outliers in the data, which should eliminate what faulty readings do occur. It takes one new reading every three seconds, and averages over the previous 12 readings. It tries to spot step-changes (e.g. lifting the spool off the holder) and if it does, it reads a whole new set of 12 readings at once, so you get a quicker update of weight (around ten seconds) and it should jump straight from previous weight to new weight without sliiiiding from one value to the next.

I also have a helper script that records tare weights and calibration values.

Hardware is a HX711 breakout and single-point load cell. Mine is an ebay-cheap-special 3kg cell, 13x13x80, and the cheapest available green breakout. Connect cell wires to the breakout (loads of instructions on probable colour scheme the web) and then breakout to a 3.3V pin, a 0V pin and two GPIOs on the Pi - the code assumes DAT to GPIO5 and CLK to GPIO6 (but there are parameters in the top of the .py file in the zip if you want to pick different ones). I created a spool holder in OpenSCAD, and because my printer is a delta the spool axis is at 30 degrees. Also beware, this holder is only 80mm wide (between upstands).

weigh_spool.scad

How it works:

The plugin creates an endpoint on the printer at http:<machine name or IP address>/machine/filament-load-cell/reading. The endpoint shows the latest measured filament weight (or a comment about why not). The DWC plugin periodically grabs the text from that endpoint and inserts it into the banner of the DWC page. In this version, the code of the endpoint is by me, the code that sticks the text into the DWC page is by Resam.

The plugin is here (but you'll need to remove the .txt - that's me circumventing the forum system): WeighFilament.zip.txt

I strongly suggest unzipping it and reading through before installing so you know I'm not doing something nefarious with your printer.

The helper script is here: weigh.txt (again, you probably want to rename it without the .txt)

Note that I don't consider myself much better than 'can make myself understood' in python, and I despise it, and my coding is not 'pythonic'. Feel free to improve it. It's MIT licence. I don't mind what you do with it, I don't warrant it fit for anything, if it breaks anything you get to keep the pieces.

Installation:

You need some python libraries: sudo apt install python3-rpi.gpio python3-numpy python3-setuptools (others too, but they are probably already installed).

You need to download and install the HX711 library from https://github.com/tatobari/hx711py This is pure python and relatively straightforward. It appears to be written for python2, but runs happily under python3 as well, so having downloaded and unpacked you cd into the repository folder and python3 setup.py install.

Enable the plugin service on the Pi. See https://forum.duet3d.com/post/219064 (but that says you won't need to do that soon). (Also, I don't think I put apparmor in my cmdline.txt, and it seems to work for me).

The helper needs to write to a state file somewhere. sudo touch /var/lib/misc/spoolweigh.txt and sudo chmod 666 /var/lib/misc/spoolweigh.txt. You can put it elsewhere but you'll need to edit both the helper script and the weigh_filament_endpoint.py file in the plugin zip file. This file will grow every time you calibrate or use a different weight spool. You can just prune older lines - the plugin uses the last relevant content it finds - it needs a 'zero' line, a 'calib' line and either a 'spool' or a 'spoolwt' line.

Do the initial calibration before running the plugin. This is all by the command line on the Pi, so you need a command terminal / window / whatever. Put the helper script somewhere, make it executable, and then you can run it. First, with nothing on the spool holder run weigh zero, this records the tare reading of the holder alone. Then you need something of a known weight not dissimilar to a spool of filament (preferably nearing the end) - I use a 200g weight - and run weigh calibrate. It will ask you what the weight is. Finally, preferably put an empty spool on the holder and run weigh spool. If you don't have an empty spool but do know what one weighs you can run weigh manual and enter a weight value. You can re-run any of these (e.g. when you change to a spool with a different empty weight), but you must stop the plugin first. There is no locking (for various reasons, not least my lack of competence), and the script and plugin will scramble each other's communication with the HX711 if they run simultaneously, normally resulting in a hang or garbled data.

Actually, you can abbreviate the options to the first letter, and scatter '-' around too - instead of weigh zero you can weigh z or weigh -z or weigh --zero or weigh zipadeedoodah etc.

If you get garbage answers or the script hangs, the plugin is probably running. Stop the plugin before running the script. (Also, the plugin only reads the file on startup - so you need to start the plugin after changing calibration values for it to get the new values anyway). Stop the plugin before running the script.

However, there is apparently the possibility of problems with bit ordering and byte ordering which can also lead to garbage values - if you get garbage values see the discussion on the HX711 library github page and make changes in definitions near the top of the helper script and the weigh_filament_endpoint.py file in the plugin zip file.

The helper also does weigh filament to report the weight of filament on a spool on the holder, weigh gross to tell you the weight of everything on the holder (i.e. typically spool plus filament), weigh quick / weigh quiet which does fewer readings and does less chit-chat, weigh raw which gives you the rawish data coming back from the HX711 and does no outlier filtering or conversion to weight units. Probably use these to check it's behaving itself before using the plugin.

Once the zero, calibrate and spool are done, you can install the plugin by dropping the .zip file onto the 'UPLOAD & START' button in DWC. If I haven't forgotten a critical step above, the weight of filament on your spool should appear alongside the machine name (here it is on my imaginatively named delta printer):

It reports to the nearest 0.1g, but don't confuse resolution with accuracy. I find the reported weight wanders around by about 2 or 3g day-to-day. Also, note that unless your filament exits the spool horizontally, the vertical component of pull force on the filament contributes (positive or negative) to the weight reading whenever the printer is extruding.

[zapta]

Very nice!

Can it also be used as a filament sensor, detecting jams and such?

[achrn]

@zapta said in Another weighing filament holder:

Very nice!

Can it also be used as a filament sensor, detecting jams and such?

Not reliably, I think.

My filament spool at least exhibits quite marked stick-slip, so when printing it doesn't decrement smoothly with print progression. In my setup the spool is on top of the printer and the filament feeds off almost vertically downwards. When printing, as the extruder feeds and filament is used, the weight reading creeps slowly upwards as the filament between spool and extruder is pulled in, then the spool slips and it drops down again before creeping upwards. The graph would be a saw-tooth on a slow downwards trend. I think by the time you could be confident the behaviour was adrift, the print would already be scrap.

While testing I did a lot of swapping spools to and fro, and got a bit slap-dash. I managed to introduce a filament tangle and got almost a kilogram of tension in the filament. The scale was weighing nearly 1000, even though I only had about 50g filament on that spool (hence tangle - it was all loosely looped). I guess you could detect that happening and pause if the weight ever builds up slowly to a much-higher-than-expected value, but that happens so rarely I don't think it's worth it.

It's really just to save the uncertainty of 'is there enough filament' at the start of a print.

[jhalewood]

Hi @achrn thanks for all your work on this
I've set this up, and I've got the helper script running, calibrated, and accurately outputing my filament weight!
It's fantastic! I'm also impressed at how accurate it is (c.f. my kitchen scales only 5g off)

My issue is that I can't get the plugin to work. It's installed ok.
Manually clicking start gives "partially started"
The JS component executes, "(no weight reading yet)" appears next to machine name, and I can see get requests for "http://192.168.1.4/machine/filament-load-cell/reading" but they all 404

greping python scripts shows that weigh_filament_endpoint.py isn't running

I can run weigh_filament_endpoint.py manually from a terminal, and it runs correctly, and outputs the correct weight that i got from the helper script

So it appears that the "plugin" isn't starting the script? Should it? have I missed something here?

[jhalewood]

@jhalewood

I managed to fix this
It was an apparmor problem preventing the running of python scripts
Setting apparmor to complain mode allowed the python script to run and the plugin works fine
Though after a reboot it goes back to "partially started" - the script isn't running. Manually clicking stop and then start on the plugin allows it to run again
(see also https://forum.duet3d.com/topic/26504/apparmor-prevents-plugins-with-python-scripts-running)

[achrn]

@jhalewood Hi, sorry - I was away visiting relatives before Christmas, and only just picking up things again.

I'm glad you solved the apparmor issue - as I noted in the first posting, I didn't do anything about apparrmor, but for me it 'just worked' - that may have been because of something I've previously done on the Pi in question. I don't understand apparmor (never had cause to delve into it).

With respect to the second issue - the 'partially started' - I have seen that myself, but not got to the bottom of it. As you note, a workaround is to manually stop and start, but that's far from ideal. However, I've been trying (this morning) to provoke it and I can't get it to not work now! That means I can't explore how to fix it.

I have updated my firmware to v3.4b6, so it might be related to that. However, the previous version of the plugin won't run on anything but v3.3. This is a version that thinks it will run on any v3 firmware, though plugins only start at v3.3, I believe, so that really means v3.3 onwards: WeighFilament.zip.txt

I'll be gobsmacked if that actually fixes the issue - I haven't changed anything other than the manifest file. I think it would only be if the process of installing an update to a plugin somehow kicks something, which seems unlikely.

If I do manage to provoke the failure again, I'll try and investigate it, and then if I find an update I'll post here. In the meantime, the plugin version above will at least work if/when you update to firmware 3.4, though I don't know if it will still have the 'partially started' annoyance.

[chrishamm]

@achrn @jhalewood Just a quick update here, I looked at the Python plugin support again as part of my work on 3.4-rc2 and using DWC I can now start Python plugins with pip libraries and env. Both things were not working correctly before. Also, I added a new AppArmor permission for GPIO access (including I2C and SPI) so that the hx711 can be addressed.

I uploaded a repacked version here but since I do not have a hx711 scale I could never test it completely.

I also moved the default path of spoolweigh.txt from /var/... to /opt/dsf/plugins/FilamentLoadCell/dsf/spoolweigh.txt because it should be accessible there without further permission changes. A better path might be the System directory (/opt/dsf/sd/sys/spoolweigh.txt), although that will require Read/WriteSystem permissions. If you wanted to, a default spoolweigh.txt file could be shipped as well in the ZIP either via dsf/spoolweigh.txt or sd/sys/spoolweigh.txt but that's up to you.

I'll be happy to help if you need further assistance packaging everything.

[achrn]

@chrishamm Thanks for the input. I've been doing some testing and thinking.

Unfortunately, when I try your package I get "Installation failed!" - "InternalServerException in InstallPlugin: Win32Exception in InstallPlugin: An error occurred trying to start process '/usr/bin/pip' with working directory '/'. No such file or directory"

I don't have pip installed (though I do have pip3), and both the 'weigh' helper script and the plugin python script are python3. I assume the package handling has decided to try and install python2 versions. One approach would be to let it (i.e. give it a python2 pip and let it get on with it), but that seems a little clumsy if I'm not using python2 versions. Does the plugin system's handling of python and pip distinguish v2 and v3? Does it just try and install both versions if any version is requested?

I notice you've also added a line "sbcPackageDependencies": ["libatlas-base-dev"], and I don't know what that is about - is that a part of the plugin handling internals?

With respect to location of the settings file, I must admit putting it in the system directory is attractive (and hadn't occurred to me), because then the user at the web interface can open it with ease and see what values are set. I have added a default one, but it just has some comments / instructions in it - I think values are too dependant on the vagaries of different load cells to have defaults pre-set. It only needs readSystem permissions (the plugin doesn't write to it, only the helper writes).

[As an aside, the page https://github.com/Duet3D/DuetSoftwareFramework/wiki/Third-Party-Plugins#list-of-permissions lists all the permission names as starting with an upper case letter. However, if I use permission "ReadSystem" I get an error "Invalid plugin manifest", it needs to be "readSystem".]

However, putting the file there needs some additional install steps, because the helper script (which is run from outside DSF) needs to read and write it.

New package

I've made an intermediate package somewhat between my previous and chrishamm's suggested changes. This doesn't use the pip functionality (for reasons as above) but does move the config file. I've also put the helper script into the zip file to slighly ease distribution.

Here it is (take the .txt off before trying to use it): WeighFilament.zip.txt

This has a slightly different install sequence compared to the post at the top of the thread:

Installation

You need some python libraries. At a command prompt: sudo apt install python3-rpi.gpio python3-numpy python3-setuptools (others too, but they are probably already installed).

You need to download and install the HX711 library from https://github.com/tatobari/hx711py This is pure python and relatively straightforward. It appears to be written for python2, but runs happily under python3 as well, so having downloaded and unpacked you cd into the repository folder and python3 setup.py install.

Enable the plugin service on the Pi. See https://forum.duet3d.com/post/219064 (but that says you won't need to do that soon). (Also, I don't think I put apparmor in my cmdline.txt, and it seems to work for me).

Drag and drop the plugin zip file onto the 'Install Plugin' button on the Plugins / External Plugins page of DWC, but don't start it.

At a command prompt: sudo chmod a+rw /opt/dsf/sd/sys/spoolweigh.txt This is so the helper script can write into the configuration file.

At a command prompt: sudo chmod a+x /opt/dsf/plugins/FilamentLoadCell/dsf/weigh This is the helper script and it needs to be executable.

At a command prompt: sudo ln -s /opt/dsf/plugins/FilamentLoadCell/dsf/weigh /usr/local/bin This is to make the helper script easily accessible.

With nothing on the spool holder at a command prompt run weigh zero, this records the tare reading of the holder alone.

Then you need something of a known weight not dissimilar to a spool of filament (preferably nearing the end) - I use a 200g weight - and run weigh calib. It will ask you what the weight is.

Then preferably put an empty spool on the holder and run weigh spool. If you don't have an empty spool but do know what one weighs you can run weigh manual and enter a weight value.

Some semi-cryptic entries should now have shown up in the spoolweigh.txt file in the system directory.

Now you should be able to start the plugin and get a weight readout.

I expect that installing an updated version of the plugin in future will require a repeat of teh subsequent steps above (or extracting the relevant lines from the spoolweigh.txt file first and then adding them into the new one, but probably easier to just redo the steps).