No problem at all. A contactor on the input power side of the VFD is fine. I would put a time delay between powering up the VFD and the DYN4, I power mine on with a 1 second delay between them to keep the inrush as low as possible.

On my lathe, the VFD is directly connected to the panel main breaker, but the DYN4's have contactors that are computer controlled.

On my mill, the VFD is never turned off, direct wired to the shop breaker panel. My air compressor VFD is wired the same, direct to the breaker panel.