You could eliminate the chatter if you used the right feed and speed and the correct cutter.
Discribe your cutters to us. It might not be the part that is chattering. It could be the cutter. If a cutter is too sharp and/or too long, it will sing like a canary.
Another way is to machine one side complete, then clean up that side and pack it with clay.
There is some stuff you can get from McMaster. I'm not sure what the actual name it, it's some kind of clay, but the industry name is bear shyt (the y is really an i). If you call them and ask for it, most of their people will know exactly what you're talking about.
The part I see does not look that difficult to machine.
You can buy GOOD PARTS or you can buy CHEAP PARTS, but you can't buy GOOD CHEAP PARTS.