If your previous config is still accurate I would change the Z and E axis speed settings as follows:
M566 Z120 E1200 ; set maximum instantaneous speed changes (mm/min)
M203 Z600 E4000 ; set maximum speeds (mm/min)
M201 Z120 E2500 ; set accelerations (mm/s^2)
Your original values were very very conservative and could explain the long delays on retractions and layer changes.
M566 Z12.00 E120.00 ; set maximum instantaneous speed changes (mm/min)
M203 Z180.00 E1200.00 ; set maximum speeds (mm/min)
M201 Z20.00 E250.00 ; set accelerations (mm/s^2)