If the buttons are secured by a threaded post I would open them up .010-.015. If they are pop in and retained by spring tabs, I would suggest filing to fit.
The spring tab mounted switch manufacturer's recommended dimensions usually are a good starting point but I have found that they are usually a little too sloppy for my taste. You don't want the fit too tight as if may interfere with the operation of the switch or button.
Scott
Consistency is a good thing....unless you're consistently an idiot.