Question

A machine is taking measurements and giving me discrete numbers continuously like so:

1 2 5 7 8 10 11 12 13 14 18

Let us say these measurements can be off by 2 points and a measurement is generated every 5 seconds. I want to ignore the measurements that may potentially be same

Like continuous 2 and 3 could be same because margin of error is 2 so how do I partition the data such that I get only distinct measurements but I would also want to handle the situation in which the measurements are continuously increasing like so:

1 2 3 4 5 6 7 8 9 10

In this case if we keep ignoring the consecutive numbers with difference of less than 2 then we might lose actual measurements.

Is there a class of algorithms for this? How would you solve this?

Answer 1

In some lines of work, the standard way to deal with problems of this nature is by using the Kalman filter.

To quote Wikipedia:

Its [Kalman filter’s] purpose is to use measurements
observed over time, containing noise
(random variations) and other
inaccuracies, and produce values that
tend to be closer to the true values
of the measurements and their
associated calculated values.

The filter itself is very easy to implement, but does require calibration.