Question

I have a csv file with 48 columns of data.
I need to open this file, place it into a data structure and then search that data and present it in a DataRepeater.

So far I have successfully used CSVReader to extract the data and bind it to myDataRepeater. However I am now struggling to place the data in a table so that I can filter the results. I do not want to use SQL or any other database.

Does anyone have a suggestion on the best way to do this?

So far, this is working in returning all records:

Private Sub BindCsv()
    ' open the file "data.csv" which is a CSV file with headers"
    Dim dirInfo As New DirectoryInfo(Server.MapPath("~/ftp/"))
    Dim fileLocation As String = dirInfo.ToString & "data.txt"
    Using csv As New CsvReader(New StreamReader(fileLocation), True)
        myDataRepeater.DataSource = csv
        myDataRepeater.DataBind()
    End Using
End Sub

Protected Sub myDataRepeater_ItemDataBound(ByVal sender As Object, ByVal e As RepeaterItemEventArgs) Handles myDataRepeater.ItemDataBound

    Dim dataItem As String() = DirectCast(e.Item.DataItem, String())

    DirectCast(e.Item.FindControl("lblPropertyName"), ITextControl).Text = dataItem(2).ToString
    DirectCast(e.Item.FindControl("lblPrice"), ITextControl).Text = dataItem(7).ToString

    DirectCast(e.Item.FindControl("lblPricePrefix"), ITextControl).Text = dataItem(6)
    DirectCast(e.Item.FindControl("lblPropertyID"), ITextControl).Text = dataItem(1)
    DirectCast(e.Item.FindControl("lblTYPE"), ITextControl).Text = dataItem(18)
    DirectCast(e.Item.FindControl("lblBedrooms"), ITextControl).Text = dataItem(8)
    DirectCast(e.Item.FindControl("lblShortDescription"), ITextControl).Text = dataItem(37)

    Dim dirInfo As New DirectoryInfo(Server.MapPath("~/ftp/images/"))
    DirectCast(e.Item.FindControl("imgMain"), Image).ImageUrl = dirInfo.ToString & "pBRANCH_" & dataItem(1) & ".jpg"
    DirectCast(e.Item.FindControl("linkMap"), HyperLink).NavigateUrl = "http://www.multimap.com/map/browse.cgi?client=public&db=pc&cidr_client=none&lang=&pc=" & dataItem(5) & "&advanced=&client=public&addr2=&quicksearch=" & dataItem(5) & "&addr3=&addr1="
End Sub

Code add to filter results:

 Try
        Dim csv As New CSVFile(fileLocation)
        Dim ds As DataSet = csv.ToDataSet("MyTable")
        If Not ds Is Nothing Then


            Dim strExpr As String = "Bedrooms >= '3'"
            Dim strSort As String = "PropertyID ASC"

            'Use the Select method to find all rows matching the filter.
            Dim myRows() As DataRow
            'myRows = Dt.Select(strExpr, strSort)
            myRows = csv.ToDataSet("MyTable").Tables("MyTable").Select(strExpr, strSort)
            myDataRepeater.DataSource = myRows
            myDataRepeater.DataBind()
        End If
    Catch ex As Exception

    End Try

Which does return the two rows I am expecting but then when it binds to the datarepeater I get the following error:
DataBinding: ‘System.Data.DataRow’ does not contain a property with the name ‘PropertyName’.

Corrected code, filter not being applied:

Public Sub PageLoad(ByVal Sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
    If Not Page.IsPostBack Then
        ReadCsv()
        lblSearch.Text = "Lettings Search"
    End If
End Sub

Private Sub ReadCsv()
    Dim dirInfo As New DirectoryInfo(Server.MapPath("~/ftp/"))
    Dim fileLocation As String = dirInfo.ToString & "data.txt"

    Try
        Dim csv As New CSVFile(fileLocation)
        Dim ds As DataSet = csv.ToDataSet("MyTable")
        If Not ds Is Nothing Then
            myDataRepeater.DataSource = ds
            myDataRepeater.DataMember = ds.Tables.Item(0).TableName
            myDataRepeater.DataBind()
        End If
        ds = Nothing
        csv = Nothing
    Catch ex As Exception
        MsgBox(ex.Message)
    End Try
End Sub

Protected Sub btnSubmit_Click(ByVal sender As Object, ByVal e As System.Web.UI.ImageClickEventArgs) Handles btnSubmit.Click
    Dim rowCount As Integer

    rowCount = QueryCsv()
    pnlSearch.Visible = False
    lblResults.Visible = True
    lblSearch.Text = "Search Results"
    lblResults.Text = "Your search returned " & rowCount.ToString & " results"

    If rowCount > 0 Then
        myDataRepeater.Visible = True
        pnlResults.Visible = True
        btnBack.Visible = True
    End If

End Sub

Protected Function QueryCsv() As Integer

    Dim dirInfo As New DirectoryInfo(Server.MapPath("~/ftp/"))
    Dim fileLocation As String = dirInfo.ToString & "data.txt"
    Dim numberofRows As Integer

    Try
        Dim csv As New CSVFile(fileLocation)
        Dim ds As DataSet = csv.ToDataSet("MyTable")
        If Not ds Is Nothing Then


            Dim strExpr As String = "PropertyID = 'P1005'"
            Dim strSort As String = "PropertyID DESC"

            Try

                ds.Tables.Item(0).DefaultView.RowFilter = strExpr
                ds.Tables.Item(0).DefaultView.Sort = strSort
                myDataRepeater.DataSource = ds.Tables.Item(0).DefaultView

            Catch ex As Exception
            End Try

        End If
    numberofRows = ds.Tables("MyTable").Rows.Count
    Catch ex As Exception

    End Try
    Return numberofRows
End Function

Answer 1

One of the ways I’ve done this is by using a structure array and reflection.

First, set up your structure in a module: CSVFileFields.vb

Imports System.Reflection

Public Module CSVFileFields
 #Region " CSV Fields "
    Public Structure CSVFileItem
        Dim NAME As String
        Dim ADDR1 As String
        Dim ADDR2 As String
        Dim CITY As String
        Dim ST As String
        Dim ZIP As String
        Dim PHONE As String

        Public Function FieldNames() As String()
            Dim rtn() As String = Nothing
            Dim flds() As FieldInfo = Me.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
            If Not flds Is Nothing Then
                ReDim rtn(flds.Length - 1)
                Dim idx As Integer = -1
                For Each fld As FieldInfo In flds
                    idx += 1
                    rtn(idx) = fld.Name
                Next
            End If
            Return rtn
        End Function

        Public Function ToStringArray() As String()
            Dim rtn() As String = Nothing
            Dim flds() As FieldInfo = Me.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
            If Not flds Is Nothing Then
                ReDim rtn(flds.Length - 1)
                Dim idx As Integer = -1
                For Each fld As FieldInfo In flds
                    idx += 1
                    rtn(idx) = fld.GetValue(Me)
                Next
            End If
            Return rtn
        End Function

        Public Shadows Function ToString(ByVal Delimiter As String) As String
            Dim rtn As String = ""
            Dim flds() As FieldInfo = Me.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
            If Not flds Is Nothing Then
                For Each fld As FieldInfo In flds
                    rtn &= fld.GetValue(Me) & Delimiter
                Next
                rtn = rtn.Substring(0, rtn.Length - 1)
            End If
            Return rtn
        End Function
    End Structure

#End Region
End Module

Next we will make our own collection out of the structure we just made. This will make it easy to use .Add() .Remove() etc for our structure. We can also remove individual items with .RemoveAt(Index). File: CSVFileItemCollection.vb

 #Region " CSVFileItem Collection "

Public Class CSVFileItemCollection
Inherits System.Collections.CollectionBase

Public Sub Add(ByVal NewCSVFileItem As CSVFileItem)
    Me.List.Add(NewCSVFileItem)
End Sub

Public Sub Remove(ByVal RemoveCSVFileItem As CSVFileItem)
    Me.List.Remove(RemoveCSVFileItem)
End Sub

Default Public Property Item(ByVal index As Integer) As CSVFileItem
    Get
        Return Me.List.Item(index)
    End Get
    Set(ByVal value As CSVFileItem)
        Me.List.Item(index) = value
    End Set
End Property

Public Shadows Sub Clear()
    MyBase.Clear()
End Sub

Public Shadows Sub RemoveAt(ByVal index As Integer)
    Remove(Item(index))
End Sub

End Class

#End Region

Next you need your class to handle the reflection import: CSVFile.vb

Imports System.Reflection
Imports System.IO
Imports Microsoft.VisualBasic.PowerPacks

Public Class CSVFile
#Region " Private Variables "
Private _CSVFile As CSVFileItem, _Delimiter As String, _Items As New CSVFileItemCollection
#End Region

#Region " Private Methods "
Private Sub FromString(ByVal Line As String, ByVal Delimiter As String)
Dim CSVFileElements() As String = Line.Split(Delimiter)
If Not CSVFileElements Is Nothing Then
Dim fldInfo() As FieldInfo = _CSVFile.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
If Not fldInfo Is Nothing Then
Dim itm As System.ValueType = CType(_CSVFile, System.ValueType)
For fldIdx As Integer = 0 To CSVFileElements.Length - 1
fldInfo(fldIdx).SetValue(itm, CSVFileElements(fldIdx).Replace(Chr(34), ""))
Next
_CSVFile = itm
Else
Dim itms As Integer = 0
If Not fldInfo Is Nothing Then
itms = fldInfo.Length
End If
Throw New Exception("Invalid line definition.")
End If
Else
Dim itms As Integer = 0
If Not CSVFileElements Is Nothing Then
itms = CSVFileElements.Length
End If
Throw New Exception("Invalid line definition.")
End If
End Sub
#End Region

#Region " Public Methods "
Public Sub New()
_CSVFile = New CSVFileItem
End Sub

Public Sub New(ByVal Line As String, ByVal Delimiter As String)
    _CSVFile = New CSVFileItem
    _Delimiter = Delimiter
    FromString(Line, Delimiter)
End Sub

Public Sub New(ByVal Filename As String)
    LoadFile(Filename)
End Sub

Public Sub LoadFile(ByVal Filename As String)
    Dim inFile As StreamReader = File.OpenText(Filename)
    Do While inFile.Peek > 0
        FromString(inFile.ReadLine, ",")
        _Items.Add(_CSVFile)
        _CSVFile = Nothing
    Loop
    inFile.Close()
End Sub

#End Region

#Region " Public Functions "
Public Function ToDataSet(ByVal TableName As String) As DataSet
Dim dsCSV As DataSet = Nothing
If Not _Items Is Nothing AndAlso _Items.Count > 0 Then
Dim flds() As FieldInfo = _Items.Item(0).GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
If Not flds Is Nothing Then
dsCSV = New DataSet
dsCSV.Tables.Add(TableName)
For Each fld As FieldInfo In flds
'Add Column Names
With dsCSV.Tables.Item(TableName)
.Columns.Add(fld.Name, fld.FieldType)
End With
Next

            'Populate Table with Data
            For Each itm As CSVFileItem In _Items
                dsCSV.Tables.Item(TableName).Rows.Add(itm.ToStringArray)
            Next
        End If
    End If
    Return dsCSV
End Function

#End Region

#Region " Public Properties "
Public ReadOnly Property Item() As CSVFileItem
Get
Return _CSVFile
End Get
End Property

Public ReadOnly Property Items() As CSVFileItemCollection
    Get
        Return _Items
    End Get
End Property

#End Region
End Class

Okay a little explanation. What this class is doing is first getting the line of delimited (",") text and splitting it into a string array. Then it iterates through every field you have in your structure CSVFileItem and based on the index populates that structure variable. It doesn't matter how many items you have. You could have 1 or 1,000 so long as the order in which your structure is declared is the same as the contents you are loading. For example, your input CSV should match CSVFileItem as "Name,Address1,Address2,City,State,Zip,Phone". That is done with this loop here from the above code:

Dim fldInfo() As FieldInfo = _CSVFile.GetType.GetFields(BindingFlags.Instance Or BindingFlags.Public)
If Not fldInfo Is Nothing Then
    Dim itm As System.ValueType = CType(_CSVFile, System.ValueType)
    For fldIdx As Integer = 0 To CSVFileElements.Length - 1
        fldInfo(fldIdx).SetValue(itm, CSVFileElements(fldIdx).Replace(Chr(34), ""))
    Next
    _CSVFile = itm
Else
    Dim itms As Integer = 0
    If Not fldInfo Is Nothing Then
        itms = fldInfo.Length
    End If
    Throw New Exception("Invalid line definition.")
End If

To make things easy, instead of having to load the file from our main class, we can simply pass it the file path and this class will do all of the work and return a collection of our structure. I know this seems like a lot of setup, but it's worth it and you can come back and change your original structure to anything and the rest of the code will still work flawlessly!

Now to get everything going. Now you get to see how easy this is to implement with only a few lines of code. File: frmMain.vb

Public Class Form1

Private Sub Form1_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
    Try
        Dim csv As New CSVFile("C:\myfile.csv")
        Dim ds As DataSet = csv.ToDataSet("MyTable")
        If Not ds Is Nothing Then
            'Add Labels
            Dim lblSize As New Size(60, 22), lblY As Integer = 10, lblX As Integer = 10, lblSpacing As Integer = 10
            For Each fldName As String In csv.Items.Item(0).FieldNames
                Dim lbl As New Label
                lbl.AutoSize = True
                lbl.Size = lblSize
                lbl.Location = New Point(lblX, lblY)
                lbl.Name = "lbl" & fldName
                lblX += lbl.Width + lblSpacing
                lbl.DataBindings.Add(New Binding("Text", ds.Tables.Item(0), fldName, True))
                drMain.ItemTemplate.Controls.Add(lbl)
            Next
            drMain.DataSource = ds
            drMain.DataMember = ds.Tables.Item(0).TableName
        End If
        ds = Nothing
        csv = Nothing
    Catch ex As Exception
        MsgBox(ex.Message)
    End Try
End Sub

End Class

This really makes for some dynamic programming. You can wrap these in generic classes and call the functions for any structure. You then have some reusable code that will make your programs efficient and cut down on programming time!

Edit:

Added the ability to dump structure collection to a dataset and then dynamically fill a datarepeater.

Hope this helps. (I know this was a lot of information and seems like a lot of work, but I guarantee you that once you get this in place, it will really cut down on future projects coding time!)