There are many reasons for shifts. Crosstalk to endstop is one. Then it shifts aways from the endstop. You said it happens in both directions, so this is only if you have endstops at both sides.
Overheating of stepper driver makes short stops, here cooling drivers is a good solution.
Loose pulley, loosing steps when hitting a part then warped upwards, acceleration being too high are other reasons.
