You did not mention material, I've had issue with Plastic parts with variances exceeding .001".
Here's the thing, everything expands and contracts with temperature/humidity; simple physics tell us that.
The key is to have the part being measured and the instrument doing the measuring at the same/stable temperature/humidity for some period of time (at least an hour).
This is refered to as "Normalization".
Often times people see the effects of temperature/humidity change as it effects the measurement, not so much the part.
This then causes a false reaction(offset) when in fact your just measuring and un-stable condition (part/gage not normalized).
HTH
Good Luck!
Control the process, not the product!
Machining is more science than art, master the science and the artistry will be evident.