Got replacement BL touch (First one was apparently faulty) and that has significantly improved things, still not perfect but almost acceptable.
I am going to rebuild with stout linear rails and with busy work schedule should be back at it in a couple months. Hopefully that might improve things.