If the nature of the toolmarks is mostly appearance and not actual fingernail-snagging roughness, Scotchbrite works fairly well. It is available in different coarsenesses, too. It doesn't load up like sandpaper.
If one must sand to remove actual toolmark gouges, then you've probably got a thousandth or two to remove and will have to use sandpaper. If you have small fillets and corners to get into, a selection of sizes of vinyl tubing comes in handy as a backing. You can wrap the paper around the tubing to approximate your fillet radius, yet the tubing is flexible enough to give and follow the contours better than fingertip or a rigid backing.
First you get good, then you get fast. Then grouchiness sets in.
(Note: The opinions expressed in this post are my own and are not necessarily those of CNCzone and its management)