If it's accurate for one condition and not for another (router running), the problem is likely due to something caused by the router. Could be electrical interference, missed steps from the motors under the load of cutting, or several other things.
However you haven't really described backlash. When you say you're getting an 'error' what exactly does it do?
If it only does it with the router running, does it do it when the router is running but you aren't cutting any material? Checking this would let you try to isolate it to an electrical interference problem or mechanical / motor strength problem.
What are your cutting parameters (speed, acceleration, depth of cut, size of bit, material being cut)?
CNC mill build thread: http://www.cnczone.com/forums/vertical_mill_lathe_project_log/110305-gantry_mill.html