It will work but the ball makes contact on a very small area, actually unless the metal it contacts deforms the area is infinitesimally small. Aluminum will deform even when it is anodized, and eventually the ball will create a score mark and may tend to gall up.
Possibly a better approach would be to put a Delrin plastic plug ahead of the ball so the ball presses against the plastic with the plastic being the part that slides on the anodized surface.
An open mind is a virtue...so long as all the common sense has not leaked out.