SOLVED!
To maybe help others:
Problem got worse, sometimes the carriage wouldn't move at all, and prints started to lean in the direction facing away from the tower. Solution was simple: Grub screw on the pulley came loose. I marked the shaft and the pulley, and after moving the carriage up and down 5 times I could measure that the carriage was further down and the markers on the shaft and pulley wouldn't align anymore. Realigned, fastened the screw and now all is well again!
Back to calibrating