Question

I am trying to program a noise reduction algorithm that works with a set of datapoints in a VB.NET DataTable after being helped with my other question. Basically, I want to take two integers, a coordinate value (yCoord for example) and a threshold smoothing value (NoiseThresh), and take the average of the values in the range of (yCoord - NoiseThresh, yCoord + NoiseThresh) and store that number into an array. I’d repeat that process for each column (in this example) and end up with a one-dimensional array of average values. My questions are:

1) Did anything I just say make any sense ;), and
2) Can anyone help me with the code? I’ve got very little experience working with databases.

Thanks!

An example of what I’m trying to do:

//My data (pretend it's a database)
1  4  4  9  2  //yCoord would equal 5
6  3  8  12 3  //yCoord = 4
8  3 -2  2  0  //yCoord = 3
9 17  3  7  5  //yCoord = 2
4  1  0  9  7  //yCoord = 1

//In this example, my yCoord will equal 3 and NoiseThresh = 1

//For the first column
    Array(column1) = //average of the set of numbers centered at yCoord = 3 _
//(in this case 8) and the NoiseThresh=1 number on either side (here 6 & 9)
//For the second column
    Array(column2) = //average of the numbers 3,3,17 for example
    //etc., etc.,

This would be performed on a large data set (typical numbers would be yCoord=500, NoiseThresh = 50, Array length = 1092) so there is no possibility of manually entering the numbers.

I hope this helps clarify my question!

P.S.: yes, I know that // isn’t a VB.NET comment.

Answer 1

I must admit that i’ve yet not understood the range part (NoiseThresh etc.), but this is a start:

Dim averages = (From col In tbl.Columns.Cast(Of DataColumn)()
               Select tbl.AsEnumerable().
                      Average(Function(r) r.Field(Of Int32)(col.ColumnName))).
               ToArray()

It calculates every average of each column in the DataTable and creates a Double() from the result (average can result in decimal places even if used on integers).

Edit: With your example i’ve now understood the range part. So basically yCord is the row-index(+1) and noiseThreas is the row-range (+/- n rows).

Then this gives you the correct result(made some code comments):

Dim yCord = 2 ' the row index(-1 since indices are 0-based) '
Dim noiseThresh = 1 ' +/- row '
' reverse all rows since your sample begins with index=5 and ends with index=1 '
Dim AVGs As Double() = (
    From colIndex In Enumerable.Range(0, tbl.Columns.Count)
    Select tbl.AsEnumerable().Reverse().
    Where(Function(r, index) index >= yCord - noiseThresh _
                     AndAlso index <= yCord + noiseThresh).
    Average(Function(r) r.Field(Of Int32)(colIndex))).ToArray()

The most important part of this this LINQ query is the Where. It applies your range on the IEnumerable(of DataRow). Then i’m calculating the average of these rows for every column. The last step is materializing the query to a Double().

Result:

    (0) 7.666666666666667   Double  => (6+8+9)/3
    (1) 7.666666666666667   Double  => (3+3+17)/3
    (2) 3.0                 Double  => (8-2+3)/3
    (3) 7.0                 Double  => (12+2+7)/3
    (4) 2.6666666666666665  Double  => (3+0+5)/3

Edit2:

One last thing. I assume that to do the same for the other axis I just
switch x & y and row & column?

It’s not that simple. But have a look yourself:

Dim noiseThresh = 1 ' +/- column '
Dim xCord = 2 ' the column index(-1 since indices are 0-based) '
' assuming that your x-cords now start with index=0 and ends with tbl.Column.Count-1 '
Dim AVGs As Double() = (
    From rowIndex In Enumerable.Range(0, tbl.Rows.Count)
    Select tbl.Columns.Cast(Of DataColumn)().
    Where(Function(c, colIndex) colIndex >= xCord - noiseThresh _
                        AndAlso colIndex <= xCord + noiseThresh).
    Average(Function(c) tbl.Rows(rowIndex).Field(Of Int32)(c.Ordinal))).ToArray()

Result:

    (0) 5.666666666666667   Double  => (4+4+9)/3
    (1) 7.666666666666667   Double  => (3+8+12)/3
    (2) 1.0                 Double  => (3-2+2)/3
    (3) 9.0                 Double  => (17+3+7)/3
    (4) 3.3333333333333335  Double  => (1+0+9)/3