Do you have the Parameters set for Remote control plus J1 Jumper must be in the correct Position
Before you connect to the VFD Drive you must have the 0-10v working at the Breakout Board if not and you spindle starts at a low speed you can damage it so get the 0-10v output working and then connect to the VFD Drive
You normally test the output in MDI mode M3S12000 should give you around 5v Check with different RPM setting to see what the output Voltage is.
Have you set the Pulley ratio in Mach3
What do you have to PWM set to