I am a little puzzled by the ratio you are using as 50800 would be typical for a metric screw. Do you have a mechanical reduction in there as well? Is this a backlash free reduction?
Are the thrust bearings on the ballscrew in perfect adjustment? Encoder couplings tight?
Have you tried a trial cut at two different diameters that span the range of swing of the machine? Does it cut those two diameters correctly.
What about heat? Could this be a heat related issue where the spindle housing is warming up?
While I would not expect the readouts of the display to demonstrate a mechanical error, a series of programmed moves could be used back and forth between two positions, and a dial indicator to check for migration of the endpoint.
Have you tried adding in a return to home command at the end of every run of the program?
Do you have a shielded encoder cable? Is the shield grounded at only one end? Is it running in its own conduit?
First you get good, then you get fast. Then grouchiness sets in.
(Note: The opinions expressed in this post are my own and are not necessarily those of CNCzone and its management)