Well Paul,
Apart from the fact that drilled holes may never be perfectly straight, you can set your machine up with a dial indicator so that it is theoretically possible at least
Mount a dial indicator on a little swing arm held in the spindle chuck. Then, bring it down and swing it around in a circle on the table of your machine. Adjust the table until the scribed circle has a zero out-of-square indication.
Do the same thing within the base of your vise, with the jaws set as far open as possible. If this is out of wack on your prefectly aligned table, then your vise is crap, and needs to be reworked.
If the base of the vise is okay, then check the perpendicularity of the back jaw. Run the dial indicator up the back jaw face with the quill mechanism of your drill. If the back jaw is serrated, then hold something that is reasonable straight in the vise, sticking up where you can get at it with the vise clamped. Actually, this may be the best condition to test under, because the clamping force may distort the vise a bit, anyways.
Drill with sharp tools. Do not drill past a partial cross hole and expect the drill not to wander off. That's about all you can do.
The next step may sometimes be to drill the hole undersize (by 1/64"), and then attempt to straighten it out by redrilling with an endmill. For this, I would recommend that you only drill a shallow pilot hole, because endmills don't reach very deep...maybe 3X diameter. So the endmill should help straighten out the first part of the hole, and if you then switch back to a drill bit that fits the endmilled hole, then you have created a sort of "guide bushing" to drill the remainder of the hole depth with.
First you get good, then you get fast. Then grouchiness sets in.
(Note: The opinions expressed in this post are my own and are not necessarily those of CNCzone and its management)