Question

Is there a way to programmatically detect a pattern in a set of data? For example, if I had the following data, how I find that the first column repeats for every 6th row and the second column repeats for every 7th row?

2   3
1   1
5   2
4   5
3   7
2   6
1   3
5   1
4   2
3   5
2   7

The actual data is far more complex than this. I can use PHP or Java. Is there a library or function for this?

Answer 1

I don’t have any great references handy, but for these fairly simple patterns, you would want to do auto correlation. Basically, you see how well the array correlates with itself with various deltas. It is defined so that +1 means the numbers are perfectly correlated, 0 means they are “perfectly” random with respect to each other, and -1 means that they are perfectly “opposite” of each other.

For a delta = 0, you are comparing the data to itself, to there is perfect correlation all the time

In your case, the first column will also show a peak (a number nearly 1) with a delta of 6, and the 2nd columns with a delta of 7.

For Java, I’d check the Apache Commons Math library.