Casual users of digital instrumentation often assume that more resolution on the display provides a more accurate measurement – that an extra digit in the readout gives a decade improvement in accuracy. This is rarely the case.
Resolution is the least value of the measured quantity that can be distinguished. In a digital meter, display resolution is usually the numeric value associated with the least significant digit on the readout. Examples: 0.001V on a voltmeter, 0.1mA on an ammeter. On meters with multiple ranges or functions, rather than list a value for every range, resolution is often expressed as the number of displayed digits.
Digits Displayed Values
3 0-999 (1,000 counts)
4 0-9999 (10,000 counts)
5 0-99999 (100,000 counts)
The user determines the resolution for a particular range based on the decimal position. For example, the least significant digit on the 10V range of a 3-digit meter is 0.01V.
Often a meter won’t display the full 0 to 9 range in the first digit. In this case, a fractional number of digits is used to describe the meter measurement span. Typical values are:
Digits Displayed Values
3½ 0-1999 (2,000 counts)
3¾ 0-3999 (4,000 counts)
4½ 0-19999 (20,000 counts)
4¾ 0-39999 (40,000 counts)
This additional span doesn’t change the resolution. On the same range, a 3-digit meter has the same resolution as a 3½-digit meter. The least significant digit on the 20V range of a 3½-digit meter is still 0.01V. Note that a meter that reads positive and negative signals, such as a DMM, has double this number of display counts. Again, the resolution doesn’t change.
Two Special Cases Are Worth Noting:
Some meters do not display every possible value of the least significant digit. One example of this is an inexpensive bath scale. While the readout has a tenth-pound resolution, the scale may only be capable of showing one-half-pound increments. Therefore, the resolution is one-half pound. Some temperature meters exhibit this characteristic due to the way sensor linearization is handled.
Averaging multiple readings can increase the measurement resolution beyond what the display shows. To do this, a user watches the display and interpolates an extra digit of resolution based on the number of readings at two adjacent values. For example: five readings of 103 and five readings of 104 interpolates to a value of 1035. While difficult to do visually, this can easily be done with meters that have a computer interface.
Accuracy is how far the measured value deviates from the true or ideal value. In a digital meter, depending on the format of the accuracy specification, resolution may affect accuracy. If the meter accuracy is specified as a percent of full scale or span, then resolution has no impact on accuracy. If a 3-digit meter and a 4-digit meter both have an accuracy of 1% of full scale, they have identical measurement accuracy at any level. The additional resolution does not provide better accuracy. Meters for physical parameters such as pressure, level, and temperature, often specify accuracy as a percent of span.
For electrical parameters such as voltage, current, and resistance, digital meter accuracy is typically specified as a percent of reading plus a number of counts in the least significant digit. Here additional resolution provides some accuracy improvement.
Example: Meter accuracy is 0.1% of reading plus 2 counts. For a 10V reading on the 10V range:
Display Reading % reading + counts Accuracy
3 digit 10.00 10mV + 20mV 30mV
4 digit 10.000 10mV + 2mV 12mV
The extra digit of resolution provides approximately a 2.5X better accuracy.
Fortunately, digital meters with higher resolution typically have a better ‘of reading’ specification. In this case, improved accuracy is due to a combination of both factors.
Example: 3-digit meter accuracy same as above. 4-digit meter accuracy 0.04% plus 2 counts. For the same 10V signal on the 10V range:
Meter Reading % reading + counts Accuracy
3 digit 10.00 10mV + 20mV 30mV
4 digit 10.000 4mV + 2mV 6mV
This 4-digit meter provides a 5X accuracy improvement.
More information on how accuracy is specified for both digital and analog meters can be found in an earlier posting – Meter Accuracy Explained.
In summary, the display resolution should not be confused with accuracy. Better resolution may contribute to better accuracy. However, the two parameters need to be considered separately.