Question

We’re having to transform a datatable into another which involves mostly transposing the source datatable into a different format.
Thinking of it sequentially, I have implemented it as below:

DataTable riskTable;
this.InitializeRmmRiskTable(out riskTable); // Initializes the columns

var calculatedRisk = (from DataRow tradeRow in tradeTableToFilter.Rows
                  where tradeRow["TradeID"] != null
                  select new
                          {
                              ROW_ID = 0,
                              TCN = tradeRow["TradeID"].ToString(),                               
                              CCY = tradeRow["CURRENCY"],                                 
                              USD_VALUE = calculator.Invoke(tradeRow) // configured delegate that will fetch the value
                          }).Distinct();

foreach (var rowData in calculatedRisk)
{
    DataRow rowToAdd = riskTable.NewRow();

    rowToAdd["ROW_ID"] = rowData.ROW_ID;
    rowToAdd["TCN"] = rowData.TCN;
    rowToAdd["CCY"] = rowData.CCY;
    rowToAdd["USD_VALUE"] = rowData.USD_VALUE;

    riskTable.Rows.Add(rowToAdd);
}
return riskTable;

Any suggestions to optimize this in terms of memory-footprint and execution cycles?

Answer 1

You can remove a lot of column lookups easily enough:

object[] values = new object[4];
foreach (var rowData in calculatedRisk)
{
    values[0] = rowData.ROW_ID;
    values[1] = rowData.TCN;
    values[2] = rowData.CCY;
    values[3] = rowData.USD_VALUE;

    riskTable.Rows.Add(values);
}

This assumes you know the order of the columns. Otherwise, the DataColumn API is the most direct, so you could store the 4 DataColumns and use those in the indexer. This applies equally to the reading code, i.e.

var tradeId = tradeTableToFilter.Columns["TradeID"];
var currency = tradeTableToFilter.Columns["CURRENCY"];

then:

var calculatedRisk = (from DataRow tradeRow in tradeTableToFilter.Rows
    let tradeIdVal = tradeRow[tradeId]
    where tradeIdVal != null
    select new {
        ROW_ID = 0,
        TCN = tradeIdVal.ToString(),                               
        CCY = tradeRow[currency],                                 
        USD_VALUE = calculator.Invoke(tradeRow)
    }).Distinct();

etc.

The use of Distinct does mean that all the objects will be buffered in memory again; if you know that you need this, fine, but in many cases this may be redundant.