Question

I’m running into what appears to be unexpected behavior when utilizing the FieldNotInFile attribute in my mapping file. Please see below, abbreviated examples of what I have configured.

The mapping for the header record is defined separately to keep the option open for MasterDetail engine:

public class HeaderMapping
{
    public string ID;
    public DateTime RptDateFrom;
    public DateTime RptDateTo;
    public DateTime GenerationDate;
....
}

I would like to combine the values retrieved from the header into the final record result so they are specified with the FieldNotInFile attribute to be added later.

public class RecordMapping
{
    // Values not in source
    [FieldNotInFile()]
    public string ID;
    [FieldNotInFile()]
    public DateTime RptDateFrom;
    [FieldNotInFile()]
    public DateTime RptDateTo;
    [FieldNotInFile()]
    public DateTime GenerationDate;

    // Start values from source
    public string RowPrefix;
    public string Field1;
    public string Field2;
    public string Field3;
....
}

In the engine execution I have defined two instances. The first to capture the single header record and parse out its values. The AfterReadRecord event is used to stop the engine after the first line.

static void Main(string[] args)
{
    // Extract the header
    FileHelperEngine<HeaderMapping> headerEngine = new FileHelperEngine<HeaderMapping>();
    headerEngine.AfterReadRecord +=
        new FileHelpers.Events.AfterReadHandler<HeaderMapping>(AfterHeaderRead);

    HeaderMapping[] headerRecord = headerEngine.ReadFile(source.FullName);

    // Capture Values
    companyId = headerRecord[0].ID;
    rptDateFrom = headerRecord[0].RptDateFrom;
    rptDateTo = headerRecord[0].RptDateTo;
    generationDate = headerRecord[0].GenerationDate;
....

Next the record engine is created. The BeforeReadRecord event is used to insert the previously captured values into the placeholders signified in the RecordMapping with FieldNotInFile attributes.

....
    // Extract the Records
    FileHelperEngine<RecordMapping> recordEngine = new FileHelperEngine<RecordMapping>();
    recordEngine.BeforeReadRecord +=
        new FileHelpers.Events.BeforeReadHandler<RecordMapping>(BeforeThisRecord);

    DataTable outputTable = recordEngine.ReadFileAsDT(source.FullName);
}
....
private static void BeforeThisRecord(EngineBase engine, BeforeReadEventArgs<RecordMapping> e)
{
    e.Record.ID = companyId;
    e.Record.RptDateFrom = rptDateFrom;
    e.Record.RptDateTo = rptDateTo;
    e.Record.GenerationDate = generationDate;
}

The outputTable result is not as expected. The fields marked as FieldNotInFile are completely omitted from the DataTable result. When debugging the process, the BeforeThisRecord method executes correctly and assigns the appropriate values but this is not reflected in the output. The DataTable columns are output as RowPrefix, Field1, Field2, etc. and not ID, RptDateFrom, RptDateTo, GenerationDate, RowPrefix, etc.

Strangely when I use the alternate method..

List <RecordMapping> recordList = recordEngine.ReadFileAsList(source.FullName);

The list items contain the RecordMapping objects with ALL of the correct values. It seems as though the DataTable translation of FieldNotInFile attributes is the culprit. Am I doing this wrong? Is this a bug?

Answer 1

You are correct that ReadFileAsDT() does not include the FieldNotInFile fields in the DataTable. It might be a bug, but honestly, I’m not sure how FieldNotInFile is supposed to be used – it’s not in the documentation here.

I think you’re better off using the Master Detail engine or alternatively just doing

 RecordMapping[] recordMappings = recordEngine.ReadFile(source.FullName);

and then if you really need a DataTable, populate it yourself with something like:

DataTable outputTable = new DataTable(); // New data table.
outputTable.Columns.Add("ID", typeof(int)); // Add all columns.
outputTable.Columns.Add("RptDateFrom", typeof(DateTime));
outputTable.Columns.Add("RptDateTo", typeof(DateTime));
outputTable.Columns.Add("GenerationDate", typeof(DateTime));
outputTable.Columns.Add("RowPrefix", typeof(String));
outputTable.Columns.Add("Field1", typeof(String));
outputTable.Columns.Add("Field2", typeof(String));
outputTable.Columns.Add("Field3", typeof(String));

foreach (RecordMapping recordMapping in recordMappings)
{
  outputTable.Rows.Add(
    companyId, 
    rptDateFrom, 
    rptDateTo, 
    generationDate, 
    recordMapping.RowPrefix, 
    recordMapping.Field1, 
    recordMapping.Field2, 
    recordMapping.Field3)
}