Question

I am building an application that has to display data received from an external system. This data can come in very quickly while the amount of bytes every row takes up is relatively small. This means a lot of rows have to be added per time unit. I’m currently at a point where it appears I receive data faster than I can handle meaning my memory usage is going up.

I think a big part of this has to do with drawing the actual dataGridView. I have done little tweaks to the dataGridView in the hope it would increase performance already. (e.g. disable auto size, special styles etc)

In a recent addition I added coloring of rows, which is was required. Currently my application works as follows:

1. I receive data from the external system
2. I place the data in a queue (ConcurrencyQueue) by a thread
3. Another thread obtains data from that queue, processes it and adds it to the BindingList that is bound to the table.

The actual adding happens in a function that has 2 parameters:
1. A list containing the items for the columns (items)
2. A color for the row.(color)

it looks as follows (semi-pseudo):

/* Store the color for the row in the color list so it is accessible from the event */  

rowColors.Add(rowColor);    //Class variable that stored the colors of the rows used in the DataGridCellFormatting event

/* Create the row that is to be added. */
ResultRow resultRow = new ResultRow();

foreach(item in items)
{
    resultRow.Set(item); /* It's actually a dictionary because some fields are optional, hence this instead of a     direct constructor call) */
}

bindingList.Add(resultRow);

/* Row coloring based on error is done in the OnCellFormatting() */


/* Auto scroll down */
if (dataGrid.Rows.Count > 0)
{
    dataGrid.FirstDisplayedScrollingRowIndex = dataGrid.Rows.Count - 1;
}

As seen in the code above the color i receive is added to a List which is used in an event of the datagridview as follows:

void DataGridCellFormattingEvent(object sender, DataGridViewCellFormattingEventArgs e)
{
    // done by column so it happens once per row
    if (e.ColumnIndex == dataGrid.Columns["Errors"].Index)
    {
        dataGrid.Rows[e.RowIndex].DefaultCellStyle.BackColor = rowColors[e.RowIndex];
}
} 

The BindingList is defined as follows:

BindingList bindingList;

where ResultRow is a class with a structure like this:

public class ResultRow
{
    private int first = 0;
    private string second = "";
    private UInt64 third = 0;
    private IPAddress fourth = null;
    //etc

    public ResultRow()
    {
    }

    public void Set (<the values>) //In actuallity a KeyValuePair
    {
        //field gets set here
    }

    public UInt64 Third
    {
        get { return third; }
        set { third = value; }
    }

    /* etc. */

Are there any relatively simple things I can do to increase performance? I was thinking of possibly disabling drawing of the datagrid while processing is busy and draw when it’s done. (although not preferred) Another thing is possibly updating less frequently instead of after every received item. (BindingList seems to automatically update the DataGridView when something is added to it though)

I hope someone is willing/able to help.

-edit-

The responsiveness of the form because quite bad as well when it’s processing data in the way described above, especially after some time. (even though the above process takes place in backgroundworker(s) and background threads)

Answer 1

Yes, there are a few things that you could do to speed it up.

Firstly – virtualise the datagrid. This is a built in mechanism to the winforms datagrid that will only populate rows and paint the client region for dataitems that are visible. Therefore if your control is only showing 20 rows (and a scrollbar for the others) then only 20 items are actually processed into the datagrid as UI.
Then, when you scroll the grid to view the other items, the datagrid populates and shows the requested rows on demand.
There is a bit of tinkering you will need to do to this (CellValueNeeded events will need to be subscribed to) and you may need to edit your bindingsource dataitems.
See http://msdn.microsoft.com/en-us/library/ms171622.aspx for more information.

The second thing you can do is to suspend the UI from updating whilst you populate in a ‘chunk’ of your data.
As you have already indicated, bindinglist will automatically update the grid when you add items to it. But by suspending the UI from updating, then reenabling at a certain interval (say every second) you will be streaming the data at a less constant rate.
Note however that the same processing will still need to be done for the data, so it is unlikely this will completely solve your data, and may be more effective at reducing screen flicker.
See Control.SuspendLayout() and Control.ResumeLayout() for more information.

In my opinion virtualisation will be your most effective tool, as its sole purpose is for improving grid functionality for very large datasets.