Question

I have N datatables, where N-1 datatables are representing some entities and 1 representing relationship between these entities.

like
Entity Country

Country DATATABLE

ID    | Country Name | Country Code
------------------------------------
ID1   | USA          | USA
ID2   | INDIA        | IND
ID3   | CHINA        | CHI

Entity Continent

Continent DATATABLE

ID     | Continent Name | Continent Code
------------------------------------
IDC1   | NORTH AMERICA  | NA
IDC2   | SOUTH AMERICA  | SA
IDC3   | ASIA           | AS

Entity Company

Company DATATABLE

ID    | Company Name   | Company Code
------------------------------------
CM1   | XYZ Company    | XYZ
CM2   | Fun Company    | Fun
CM3   | ABC Company    | ABC

Relationship among these.

Company_Country_Continent_Relationship DataTable 

ID    | Company        | Country    |  Continent     | Some Value1     | Some Value 2
-------------------------------------------------------------------------------------
R1    | CM1            | ID1        |  IDC1          | 100             | 150
R2    | CM2            | ID2        |  IDC3          | 200             | 200
R3    | CM3            | ID1        |  IDC1          | 150             | 250
R4    | CM1            | ID3        |  IDC3          | 100             | 150
R5    | CM2            | ID1        |  IDC1          | 200             | 200
R6    | CM3            | ID2        |  IDC3          | 150             | 250
R7    | CM1            | ID2        |  IDC3          | 100             | 150
R8    | CM2            | ID3        |  IDC3          | 200             | 200
R9    | CM3            | ID3        |  IDC3          | 150             | 250

Now I need to generate another relationship table, which will hold Name instead of ID.
In this example relationship data is storing ID for company, country and continent, now I want to transform these id value to there name i.e . instead or CM1 – XYZ Company.

For this transformation, I am using a method TramnsformRelationshipData, and it is working correctly.

    public static DataTable TramnsformRelationshipData(DataTable relationshipData, Dictionary<string, DataTable> mapping)
    {
        DataTable transformedDataTable = null;
        if (relationshipData == null || mapping == null )
           return null;

        transformedDataTable = relationshipData.Copy();

        foreach (DataColumn item in relationshipData.Columns)
        {
            if (mapping.ContainsKey(item.ColumnName))
            {
                var instanceData = mapping[item.ColumnName];
                if (instanceData == null)
                    return null;

                foreach (DataRow row in transformedDataTable.Rows)
                {
                    var filteredRows = instanceData.Select("ID = '" + row[item.ColumnName] + "'");
                    if (filteredRows.Any())
                        row[item.ColumnName] = filteredRows[0][1];
                }
            }
        }

        return transformedDataTable;
    }

But, this method iterating all the datatables, and being very slow, when relationshipdata having more entities to transform. So, How can i optimize this code, to work a large number of datatables with large number of rows.

Edited : In most conditions, These data are not stored in DB, These are in memory, and in memory the count for these datatables can be increased or decreased.

Thanks.

Answer 1

The solution here is to create a hash based collection (i.e. hashtables,dictionary,lookups in .NET)
with the ID column being the key and using that instead of the .Select(Id = x)

Code could look something like this…. Untested.

public static DataTable TramnsformRelationshipData(DataTable relationshipData, Dictionary<string, DataTable> mapping) 
    { 
        Dictionary<string,Dictionary<string,DataRow>> newMappings = new Dictionary<string,Dictionary<string,DataRow>>();
        foreach (var kvp in mapping)
        {
            newMappings.Add(kvp.Key,kvp.Value.Rows.Cast<DataRow>().ToDictionary(dr=>dr["ID"] as string));
        }

        DataTable transformedDataTable = null; 
        if (relationshipData == null || mapping == null ) 
           return null; 

        transformedDataTable = relationshipData.Copy(); 

        foreach (DataColumn item in relationshipData.Columns) 
        { 
            if (newMapping.ContainsKey(item.ColumnName)) 
            { 
                var instanceData = newMapping[item.ColumnName]; 
                if (instanceData == null) 
                    return null; 

                foreach (DataRow row in transformedDataTable.Rows) 
                { 
                //  var filteredRows = instanceData.Select("ID = '" + row[item.ColumnName] + "'"); 
                //  if (filteredRows.Any()) 
                    row[item.ColumnName] = instanceData[row[item.ColumnName]][1];                       
                } 
            } 
        } 

        return transformedDataTable; 
    }