@schild0r, on Duet 3 MB6HC the un-calibrated errors can be quite large. Typically, TEMP0, TEMP1 and TEMP2 have significant error at low temperatures only, and different TEMP3 has significant error at high temperatures only.
When I built my printer and my XY carriage I tried to be clever and reduce the number of wires I need to run through my drag chain by combining all ground / power / etc connections as much as possible.
I wrongly assumed that VSSA can be substituted with GND. I essentially plugged my thermistor into GND+THERMISTOR1. This gave me a "good enough" reading apparently that I never bothered to double check. 🤦
This however creates a few tiny problems because @dc42 is even more clever and compensates for the extra fuse resistance of VSSA in the firmware - which means I have a wrong reading in certain cases.
Now I'm running my thermistor wiring the way god @dc42 intended, and I have a super clean temperature reading!
@hamiltonprops From re- reading your OP, you say that the temperature drops 20 degrees within a second or two. That would indicate it's not a "real" drop in temperature but rather a drop in the indicated (measured) temperature - unless your hot end has an insanely low thermal mass. So, yes I go for some sort of cross talk or electrical "noise". Are you sure you haven't got a bad crimp on the thermistor wiring somewhere?
@phaedrux it's still a gamble. No guarantee it will be accurate. IR beam is also quite large and hotend is small. In order for IR to measure temperature, the surface has to fill the beam completely. For hotbed also it is easy to just use contact thermometer, will be much more reliable.