Big machines do the same.
Previously, I had observed similar Z problems (like tower bending and head bobbing) on machines even weighing in a 10 tons.
I just hadn't figured out how to fix it until I had my wife pushing down on the head, then adding a weight and removing it at the correct moment.
I don't think it is the tower attachment on the X3. It just bends. It is only a small section. Having extra (permanent) mass on the head won't help either, but will help to absorb high frequency vibrations.
SX3 is still good value for money/performance/quality.
Super X3. 3600rpm. Sheridan 6"x24" Lathe + more. Three ways to fix things: The right way, the other way, and maybe your way, which is possibly a faster wrong way.