Just to update the status in case someone comes across this post. I'm installing the bl touch now, which should work just fine, but to prove I'm not crazy I hooked the x5sa board up one more time just to make sure it wasn't my imagination, you can see a video of the results here:
You'll notice the duet board does respond to metal just fine, but even if I run the sensor into the plastic + aluminum bed, it won't trigger. Then I switch back to the x5sa board, and you'll see there's somewhat acceptable clearance in the same situation. Both tests had an unheated bed, running off 12v, on the same xy of the build plate.
I have no idea what's going on, but I'm tired of trying to chase down the blackmagic going on here, so I'm installing the bltouch and setting that up per the guidelines in the duet wiki. If anyone has any idea what's going on though I'd be edger to learn.