Try plunging a hole with your bit and measure it check the diameter hole its actually cutting. I have found that my 1/4 wood cutting bit/router bit measures about. 2410 and my 3 flute 1/4" end mill for aluminum measures 0.247.. Also, make sure your steps per are correct. Measure, it, measures it again, and just to be sure, measure if again. If you call 1" movement and you get 1" of movement, you're good to go there. Make sure all the adjustment bolts are snug and then check backlash. Then just use the backlash compensation in mach3.