Accuracy, Resolution and Repeatability

There are several key terms that are critical to specifying the optimal position sensor for a given application. This application note defines these key terms and how they relate to the sensor’s role in the overall system performance. It is important to consider that precision is very much a system issue and can be dominated by mechanical errors such as eccentricity. This paper discusses the most significant mechanical error for rotary applications (eccentricity) and presents eccentricity tolerant position sensor solutions.


Resolution defines the smallest position increment that can be moved or measured and is typically expressed in “counts”, where counts is the number of subdivisions in 360°. Often, counts are given in bits, for example a 16-bit encoder has 216 (65,536) counts per revolution (CPR).

Fig. 1 shows an encoder (A) with 360 CPR (1°) resolution and an encoder (B) with 90 CPR (4°) resolution. The numbers in each gray box indicate encoder count readings. The blue dot is the true position to be measured – 1.5°. Encoder A has a count reading of ‘1’ which means the position is between 1° and 2°. Encoder B reads ‘0’ which means the position is between 0° and 4°. Encoder A clearly gives more precise position information. (Note, the range covered by an encoder count is often referred to as a “bin”. 1.5° is in bin 1, the 1° to 2° bin).

Figure 1 – Defining position for a true position of 1.5° for two theoretical encoders of 1° resolution (Encoder A, Left) and 4° resolution (Encoder B, Right). Notice that due to its increased resolution Encoder A is more precise, as it would describe the angle as being between 1 and 2° as opposed to Encoder B, where it is between 0 and 4°.

Now consider a move of one count – the smallest possible position change. The new position targets (2.5° for A and 5.5° for B) are depicted by the green dots in Fig. 2. Clearly it is not possible to make a move of 1° with encoder B as the position change is not measurable.

Figure 2 – A move of one count, shown by the green dot. The yellow dot represents when the control system thinks the movement is complete.

The yellow dot indicates the position at which the control system believes the move is complete i.e., when the reading changes by one count. It is clearly possible to get closer to the true target position with the higher resolution encoder.

High resolution is required for high performance servo systems. A positioning system “dithers” between two counts so the higher the resolution the smaller the dither. Resolution also has a significant impact on velocity ripple at low speed. Since velocity is derived from position feedback, if the resolution is low there may be insufficient data in a sample to accurately derive velocity. At high speeds, high resolution devices can generate data rates beyond the tracking capability of the controller or servo drive.


Accuracy defines how close each measured position is to the actual physical position. Precision machine builders typically calibrate out errors via a lookup table of offsets. In general, high-resolution encoders have higher accuracy but it is important to understand that accuracy and resolution are very distinct parameters.


Repeatability defines the range of measured positions when the system is returned to the same physical position multiple times. Repeatability can be more important than absolute accuracy. For system inaccuracies to be effectively calibrated it is important for each position reading to be consistent. Sensor hysteresis (different readings depending on direction of approach to measure position) is an important factor in repeatability.

Figure 3 – More accurate shots (left), high repeatability shots (middle), calibrated (right)

Figure 3 provides some insight on the importance of repeatability. Although the left target has more accurate shots, after calibration the higher repeatability shots yield greater precision. Note, repeatability is tied to resolution as a measurement can only be as repeatable as the smallest measurable angle or as in the example in the Resolution section, one bin.

Eccentricity Error

Optical and magnetic encoders which employ a readhead at a specific location on the scale generate a significant angular error, known as eccentricity error, if the scale is not centered precisely on the axis of rotation. Bearing runout also contributes to eccentricity errors. Eccentricity error is repeatable (excluding non-repeatable bearing runout) and can be calibrated out, however this can be difficult as the calibration must occur after the encoder has been installed.

Figure 4 – Impact of mechanical eccentricity and its varying impact on angular error.

As discussed, angular error from eccentricity can dominate system accuracy. Some optical and magnetic encoders will employ two read heads, located opposite of one another. This helps to offset some of the delta, but comes with increased cost and complexity. Capacitive and inductive encoders are holistic, meaning they measure position over a full 360° and are much more tolerant to eccentricity. For more information on holistic encoders see the application note ‘Why Holistic Encoders are More Accurate’ .


As discussed, variations in scale pitch (linearity) can be calibrated out. If the variation in linearity is monotonic or slowly varying, the non-linearity can be easily calibrated using a few reference points. In the example shown below, Figure 5, a fairly non-linear transducer is calibrated into a highly linear device with a relatively low number of reference points.

Figure 5 – Calibration of a non-linear sensor with slowly varying errors

In this second example, Figure 6, however, a device with a rapidly varying error is calibrated with 10 points and its linearity hardly changes. It may take >1000 points for such a rapidly varying measurement characteristic to be linearized.

Figure 6 – Calibration of a non-linear sensor with rapidly varying errors


Resolution and repeatability are key parameters for high performance position control. It is important to realize that resolution and accuracy are distinct parameters and an application may require high resolution, but not necessarily high accuracy. Encoder accuracy is also critical to system performance but should not be considered in isolation as mechanical errors may have a greater influence. Repeatable errors can be calibrated out, however calibration does increase encoder latency and adds time to system setup.

Still have questions? That’s great! Everight Position is here to help. Contact us today, it would be exciting to learn more about your application and to help navigate all the sensing options currently available on the market.

"*" indicates required fields

This field is for validation purposes and should be left unchanged.