As suggested, take the chuck apart and clean it well. Check the mounting plate for embedded chips, burrs, etc. Check the run out on the mounting plate. Make sure the jaws are indeed in the correct numbered slot. If there are three bolts attaching the chuck and no dowel pin it could go on in three positions; check all three.
If, after doing all this playing around you cannot get better than 0.004" then consider trying to true the jaws...incidentally is that 0.004" TIR, in other words the chuck is 0.002" off center or is it 0.004" off center.
If it is 0.004" TIR you may be better off living with it. It is difficult to load the jaws correctly to grind them because a ring at the back does not load the front. Also it is almost certain to vary with diameter as the scroll will have some wear.
An open mind is a virtue...so long as all the common sense has not leaked out.