2024 AL is not 4130 steel. RPMs too high for the length of cutter. Taking too much material will cause tool too much deflection and chatter. Depending on your cut you may have to reduce your RPM & feed to as little as 3% the material removal of a ,5 dia x 1 1/4 em, being that it is 4" long. .03 - (1.25/4)^3. Of course you want to use the highest material removal rate which the machine and tool will allow. Adjust up from the minimum to get your optimum feed. I would use a drill, rough and pre-finish and finish cut. The finish cut should be closer to normal of what you would use, but with reduced RPM.
Safety - Quality - Production.