With that size cutter and DOC you are pretty much on the limit of the ER20 TTS holder, even with all the tweaks and tricks. Every tooling system has its' limit and you found the TTSs'
What you should try is a ER40 collet holder with a an R8 shank. Also forget the PDB, use a manual and do it up tight. It works for me when profiling with a very big DOC.
Phil