Try running a test with the backlash set to zero, to see if that might be the source of the cumulative error.

How large were the circles that you were running? Is there any difference in magnitude of error between a 1" radius and a 10" radius movement?

Does your backlash adjustment work out exactly in whole encoder steps? I presume your encoder is running in quadrature resolution. How many lines/rev is your encoder?