If your screws really are 5 tpi, then I think you have not got the correct number of counts per inch.
Other than that, the motors will hunt for position. Since the resolution is only 250 counts per turn, and each turn is .2" then, the error range that the motor must bounce between is going to be near a thousandth of an inch.
More counts = higher resolution = more accuracy for the PID parameters for each motor.
First you get good, then you get fast. Then grouchiness sets in.
(Note: The opinions expressed in this post are my own and are not necessarily those of CNCzone and its management)