It can be a bear to get the hole started without wandering of the spot drill, especially when the part is steeply curved, such as is the case with small diameter rod. It takes a good solid spindle (like on a mill) to ensure a correct start.
it can help a great deal if you mill a small flat spot on the part, then spot drill it, then drill it, then ream it.
First you get good, then you get fast. Then grouchiness sets in.
(Note: The opinions expressed in this post are my own and are not necessarily those of CNCzone and its management)