Al,
I like Ken's suggestion, too, because reamers can often give you a surprise about the size you end up with.
Perhaps you could get a large 5C blank collet (like with a 2" or 3" diameter face) and bore it to hold your disks by their perimeter. Because of the large contact area, chucking pressure does not need to be extreme. The back side of the disk gets supported by the counterbore in the collet, too. You could sandwich some protective thin plastic around and behind the disk before inserting it into the collet. Think Stretch and Seal.
First you get good, then you get fast. Then grouchiness sets in.
(Note: The opinions expressed in this post are my own and are not necessarily those of CNCzone and its management)