John Garner

04-09-2008, 11:11 PM

Good Evening, All --

An earlier thread dealing with level instruments compares the sensitivity of a premium carpenter's level, which the maker claims is "0.5 mm PER 1 METER", to the 0.005 inch per foot sensitivity of a standard millright / machinist level such as the Starrett Model 98 in its 6-inch-or-longer versions. The difference between them seems small -- 1 part in 2,000 in the first case, 1 part in 2,400 in the latter -- but the numbers alone are misleading.

The typical carpenter's level maker cites "attainable accuracy" assuming that the residual decentering of the bubble within the vial is not discernable, whereas the millwright level maker cites the tilt needed to displace the bubble 1 increment of vial graduation . . . typically 2 millimeters for European, Asian, and newer British vials, 1/10 inch for U.S. and older British vials.

So how well can the bubble actually be centered?

Well, one of the premier makers of carpenters' levels, Stabila, says on their website "The eye readily detects differences of one ten thousandth of an inch, so the reading of the difference in proximity to a reading ring from one end of the bubble to another is the least of anyone’s problem." (I'll editorialize: I'm skeptical.)

In contrast, Princeton University's Dr. Philip Kissam concluded in the early 1950s that the uncertainty in "reading" the position of the bubble in a vial graudated in 1/10 inch increments is, under good conditions, 1/5 of a division. This agrees well with a similar study performed by Kern Instruments of Aarau, Switzerland in the early 1980s that determined a skilled operator could reliably read the position of the bubble in a vial graduated in 2 millimeter increments to 1/4 division. (1/5 of a 1/10 inch graduation is 0.020 inch, as is 1/4 of a 2 millimeter graduation after rounding to the nearest 0.001 inch.)

What is probably the best way to state the sensitivity of level vials is to declare the vial's radius of curvature, which is unfortunately uncommon. Radius of curvature can be calculated from the second-best data format, tilt over bubble displacement -- a simple example being the 0.005 inch per foot per 1/10 inch graduation of the Starrett Model 98-12 vial . . . 0.1 inch divided by 0.005 inch per foot = 0.1 inch times 200 feet per inch = 20 feet.

I didn't find a radius-of-curvature value on Stabila's website, but the 1970 Interim Amendment to the U.S. Federal Specification GGG-L-211C requires a carpenter's level vial " . . . a radius of 7.645 inches, minimum, equivalent to a vial sensitivity of 45 minutes maximum . . . ". One major U.S manufacturer of carpenters' levels, Empire Level, hints at the sensitivity of their carpenters' levels by offering unmounted vials for sale that range in sensitivity from 38 arcminutes per 1/10 inch to 45 arcminutes per 1/10 inch of bubble displacement.

If a generally-recognized-as-superior Stabila carpenter's level is twice as sensitive as the competitive Empire level, the Stabila vial's radius of curvature would be in the 18 inch range. A full order-of-magnitude less than the radius of curvature of a millwright level vial.

Bottom line: If a high-grade carpenter's level is the best you have, use it. But don't try to fool yourself into thinking it's almost as good as a millwright's level.

John

An earlier thread dealing with level instruments compares the sensitivity of a premium carpenter's level, which the maker claims is "0.5 mm PER 1 METER", to the 0.005 inch per foot sensitivity of a standard millright / machinist level such as the Starrett Model 98 in its 6-inch-or-longer versions. The difference between them seems small -- 1 part in 2,000 in the first case, 1 part in 2,400 in the latter -- but the numbers alone are misleading.

The typical carpenter's level maker cites "attainable accuracy" assuming that the residual decentering of the bubble within the vial is not discernable, whereas the millwright level maker cites the tilt needed to displace the bubble 1 increment of vial graduation . . . typically 2 millimeters for European, Asian, and newer British vials, 1/10 inch for U.S. and older British vials.

So how well can the bubble actually be centered?

Well, one of the premier makers of carpenters' levels, Stabila, says on their website "The eye readily detects differences of one ten thousandth of an inch, so the reading of the difference in proximity to a reading ring from one end of the bubble to another is the least of anyone’s problem." (I'll editorialize: I'm skeptical.)

In contrast, Princeton University's Dr. Philip Kissam concluded in the early 1950s that the uncertainty in "reading" the position of the bubble in a vial graudated in 1/10 inch increments is, under good conditions, 1/5 of a division. This agrees well with a similar study performed by Kern Instruments of Aarau, Switzerland in the early 1980s that determined a skilled operator could reliably read the position of the bubble in a vial graduated in 2 millimeter increments to 1/4 division. (1/5 of a 1/10 inch graduation is 0.020 inch, as is 1/4 of a 2 millimeter graduation after rounding to the nearest 0.001 inch.)

What is probably the best way to state the sensitivity of level vials is to declare the vial's radius of curvature, which is unfortunately uncommon. Radius of curvature can be calculated from the second-best data format, tilt over bubble displacement -- a simple example being the 0.005 inch per foot per 1/10 inch graduation of the Starrett Model 98-12 vial . . . 0.1 inch divided by 0.005 inch per foot = 0.1 inch times 200 feet per inch = 20 feet.

I didn't find a radius-of-curvature value on Stabila's website, but the 1970 Interim Amendment to the U.S. Federal Specification GGG-L-211C requires a carpenter's level vial " . . . a radius of 7.645 inches, minimum, equivalent to a vial sensitivity of 45 minutes maximum . . . ". One major U.S manufacturer of carpenters' levels, Empire Level, hints at the sensitivity of their carpenters' levels by offering unmounted vials for sale that range in sensitivity from 38 arcminutes per 1/10 inch to 45 arcminutes per 1/10 inch of bubble displacement.

If a generally-recognized-as-superior Stabila carpenter's level is twice as sensitive as the competitive Empire level, the Stabila vial's radius of curvature would be in the 18 inch range. A full order-of-magnitude less than the radius of curvature of a millwright level vial.

Bottom line: If a high-grade carpenter's level is the best you have, use it. But don't try to fool yourself into thinking it's almost as good as a millwright's level.

John