[3.6.0-rc.3] sticky probe with 1HCL board

ironhydroxide

@dc42 said in [3.6.0-rc.3] sticky probe with 1HCL board:

My best guess is that intermittent contact between the blade and the tube is generating several interrupts in succession,

I would say this is more likely the problem.
I'll test with the rc3+1 and let you know what happens, but at this point I think I will probably go back to the mainboard connected probe and see if I can get it to be happier there with trying the filtered vs unfiltered settings.

The system I'm trying to replace with this has a weird capacitive pump setup and measures the difference in voltage after a delay, timed to the pump cycling. But that system is 8bit, was hoping something modern would be able to handle a simpler setup.

dc42

@ironhydroxide it's likely that we could solve this using a R-C network to filter the input from the probe; but we should be able to solve this in software too. There is a debounce facility available in the MCU that I will try enabling if necessary.

We're hoping to release 3.6.0 stable later this week. It would be good to get this issue fixed first.

ironhydroxide

@dc42 Well I'm willing to work with you on it if you feel like it'd be beneficial for the project as a whole. I don't know how many people will have such a dirty probe input/debounce as this one probably is (i really need to get a scope setup on this to see just how bad it is)

As for a debounce facility, that's likely going to expect going from open to closed and staying closed. I can't guarantee this "probe" will do that, as the blade is removing material.
So it's almost as if (for this setup alone) I need something that'll fire the interrupt on first cross of the setpoint, but not fire again until the next probe command is sent. I'm not sure what that might do for/against accuracy, if it even is feasible to setup in such a way.

Tested with the rc3+1, still stuck the probe after a while. I have the event log from it, though didn't take the extra steps after stuck as last time.

I'm down for whatever tests you'd like me to try.

dc42

@ironhydroxide thanks for testing rc3+1. I'd really like to get this solved in time for the 3.6 stable release, so I'll keep working on it. Expect another test version tomorrow.

dc42

@ironhydroxide thanks again for your patience. Please try the latest firmware at https://www.dropbox.com/scl/fo/dumsdufoej44q97ek9joo/AIBRnU-wtKfMrbWPzZwH_XY?rlkey=idmyinvvcuiwmycbb1l2obz38&dl=0. After installing it, M115 B50 should report the build date as today.

ironhydroxide

@dc42 said in [3.6.0-rc.3] sticky probe with 1HCL board:

M115 B50

Duet EXP1HCL rev 1.0a or earlier firmware version 3.6.0-rc.3+1 (2025-05-13 11:00:29)

Looks like yesterday is what I'm getting reported... Guess it is grabbing from the "old" folder in the zip.
I unzipped, removed that folder, and rezipped. now I get

Duet EXP1HCL rev 1.0a or earlier firmware version 3.6.0-rc.3+1 (2025-05-14 08:29:27)

Is there any specific outputs you want tested? or just that I don't encounter a sticky probe again?

dc42

@ironhydroxide just test whether the probe gets stuck again please.

ironhydroxide

@dc42
Interesting results.

It seems as if the probe stuck, but reset itself without power cycle of mainboard.

Confirmed it was stuck, then decided to see if the green LED was still flashing or not, and manually triggering the probe caused LED to flash, after which it was no longer stuck.

Doing more testing to see if I get the same result again.

ironhydroxide

@dc42 Yup, second time did the exact same thing.

Stuck probe, but on the first "retrigger" the probe came back to life.

My probe is inverted, so that might be a pertinent factor in this.
when voltage at pin is low, probe triggered.

dc42

@ironhydroxide do you mean that it failed to trigger when it made contact, but when you broke and remade contact then it did trigger?

dc42

@ironhydroxide please try this EXP1HCL binary, which has debouncing enabled. After installation, M115 B50 should report the build date/time as 2025-05-13 11:05:26.

Duet3Firmware_EXP1HCL.bin

The debouncing in this build adds a latency of about 250us to the trigger detection. If this build fixes the issue then i can try reducing the latency.

ironhydroxide

@dc42 said in [3.6.0-rc.3] sticky probe with 1HCL board:

do you mean that it failed to trigger when it made contact, but when you broke and remade contact then it did trigger?

that it remained triggered after a probe was completed, and my script went through 5 instances of probe, seeing the probe triggered and backing off more from the tube. then it aborts the print if the probe points aren't close enough together.

So, probe success,
probe stays triggered,
probe commanded but errors due to already contacted,
backs off the tube a bit,
Probe commanded but errors due to already contacted
repeat 4x more.
Abort print,

I find probe in triggered state after abort
confirm voltage at pin is high (~3v)
manually ground the pin and watch the green light on 1HCL show up.
unground the pin and watch the green light go out, as well as probe status in DWC go from 1000 to 0.

I'll test that bin and let you know.

ironhydroxide

@dc42 said in [3.6.0-rc.3] sticky probe with 1HCL board:

M115 B50 should report the build date/time as 2025-05-13 11:05:26.

I'm getting
Duet EXP1HCL rev 1.0a or earlier firmware version 3.6.0-rc.3+1 (2025-05-14 08:29:27)

Downloaded twice, and loaded rc1, then this download just to make sure it's loaded correct.

dc42

@ironhydroxide I'm sorry, that was the correct firmware but I sent the M115 to the wrong board to check it, also the build date was being updated but not the build time. Here's another one, this time with build date 2025-05-15 08:01:41.

Duet3Firmware_EXP1HCL.bin

ironhydroxide

@dc42 Ah, no worries.

Loaded the latest, 5/15 build date,
Almost immediately the same symptoms.
Probe is good, then probe remains triggered,
next probe errors for already triggered,

Stays triggered until I manually trigger the probe and it resets.

Ran this 3x, symptoms the same each time.

and subjectively felt the occurrence to be much earlier in the run.

dc42

@ironhydroxide thanks. It looks like there were two issues:

Interrupt system getting locked up so that it doesn't detect further state changes. The last few builds have fixed this.
The remaining issue is that t doesn't detect a transition (i.e. probe contact) so you have to break contact, then it will detect the next contact. I think I have worked out what causes this, and how to fix it. I expect to have another firmware build to test this afternoon.

dc42

@ironhydroxide please try this build. The build time is 09:23 today.

Duet3Firmware_EXP1HCL.bin

dc42

@ironhydroxide did you have a chance to try this build? We plan to release 3.6.0-stable this week and I would like to include a fix for this issue in it.

ironhydroxide

@dc42 Sorry, been out the last 4 days on holiday. I'll test that build today and let you know asap.

ironhydroxide

@dc42
I am tentatively calling this one fixed.
A complete run of my test and no sticky probe. 5810 probes in succession, not a single "Probe already triggered" error as well.

I have started another run, but all things point to the sticky probe issue being fixed.

Z drift on the other hand.... seems to potentially be a further issue. I will address that in the other post, to keep things clean.