Old machine doesn't matter, the processing power for simple testing is not important, the only thing at the pc end that matters is the Mach3 kHz setting, which is usually 25kHz from memory, and the logic signals, which are very rarely, ever a problem. What I suspect your problem will be is missed signals due to the bad earth and analog/digital grounds, and the buffer/interface board. Until you resolve all the ground plane issues, and then the buffer/interface circuitry, you won't get anywhere I'm afraid. Even the opto circuitry which is supposed to help level logic signals, is so badly designed it provides no protection and some boards degrades the signal path. Out of curiosity have you powered up the spindle in it's Z mount while the machine is moving? Generally with these controllers the spindle powered up puts so much garbage back in the circuitry it moves on it's own accord like a drunk at a wine tasting.
You could seriously end up spending the next 6 weeks trying one after another, but in the end it generally comes down to, buy a Gecko or Lini, or similar.
It's absolutely unfathomable to me why all of these machines come with a completely useless controller. Even the few users who have gotten them sort of working, after a fashion, and within strict power up guidelines to avoid blowing the chips, don't have the great things good controllers have like mid-band reasonance damping, true micro-step, short circuit protection, idle holding current, etc, etc.
cheers,
Ian
It's rumoured that everytime someone buys a TB6560 based board, an engineer cries!