The grip on a magnetic chuck depends on the surface area of the part in contact with the chuck. Your 1-2-3 block has holes (probably) that subtract from the surface area so I think it is not surprising that you can move it.
Magnetic chucks are for grinding where the DOC is measured in thousandths of an inch; they are not often suitable for milling unless you area dealing with a large area.
To some extent you can compare magnetic chucks with vacuum tables; you have a fixed maximum hold-down force per square inch, the only way to increase the grip is increase the area.
An open mind is a virtue...so long as all the common sense has not leaked out.