![Quote](images/misc/quote_icon.png)
Originally Posted by
Jason Marsha
After looking at more sites on conversion I am see that persons doing conversion are rounding off quite a few decimal places. Some sites have 1mm = 0.03937 inches which causes a problem when reconverting to mm.
If 25.4mm is defined as 1 inch
then this would imply that 1 mm = 1/25.4 inches,
so we get 1mm = 0.03937007874015748031496062992126 inches.
If we take the reciprocal of 0.03937007874015748031496062992126 we get 25.4 .
If you do not round off you will not get an error.
Since I am using the microsoft calculator on my computer I am limited by the number of decimal places that it can display.
Jason
This is all true if you hold an infinite number of decimal places during your calculations. In reality the mantissa of most calculators is only 8, after which roundoff errors do occure. I had an example I used to teach my students, and I can't remember it off the top of my head. (If I get really ambitious I'll look it up. But I doubt I'll get that ambitious.)
But it was something like doing the same calculation series and in one case rounding to 1 decimal place each set of the series while in the second case rounding to three during the course of the series of calculations. One answer turned out to be almost double the other. So roundoff can make a difference.
But, for 99% of what were doing here, you are right.
If you cut it to small you can always nail another piece on the end, but if you cut it to big... then what the hell you gonna do?
Steven