Question

BackGround: i am executing around 80 threads, each thread will continuously add data to a database(around 30k times for each thread)(i am using sql server express). but the slow part of the program being the TableAdapter.Update part.

Why i am asking this question : To know of any faster methods to update the database.

What i am thinking of : Is that to assign a new compact database to each thread and after every 10k entries sync with the actual DB.(But i don’t know how to do any part of it)

Broken to bare-bones code :

class Do : Form1
{
    public int i;
    public void FillTable(string[] ws)
    {
        DataRow row = database1DataSet1.Result.NewResultRow();
        row["one"] = ws[0];

        try
        {
            database1DataSet1.Result.Rows.Add(row);
        }
        catch {}
        lock (this)
        {
            try { resultTableAdapter.Update(database1DataSet1.Result); }
            catch { }
        }
    }

    public void start()
    {
        for (;;) //Run about 30k to 40k times
        {
          string[] ws = SOMEFUNC();
          FillTable(ws);
        }   
    } 
}

Calling Code:

Do[] case1 = new Do[80];

Thread[] t = new Thread[80];
for (int ij = 0; ij < 80; ij++)
{
    case1[ij] = new Do();
    case1[ij].i = ij;
    t[ij] = new Thread(new ThreadStart(case1[ij].start));
    t[ij].Start();
}

Any ideas?

EDIT : one of the solution: http://www.dotnetcurry.com/ShowArticle.aspx?ID=323

Answer 1

You can do multiple things.

You could do bulk loading either via SqlBulkCopy or (since your on SqlServer 2008) you can pass a table to a stored procedure and have the stored procedure use the MERGE INTO sql command to quicking insert or update the table.

We can get around 200k inserts in 10 seconds using this method.