Question

I need to parse an XML file and store the data to sql server in vb.net. I have been reading online and it seems that I should use xmltextReader class to read the file, since this will eliminate the out of memory issue. I started something like following. I need feedback that am I doing OK or should I used some other way that will be easier. Sample XML file goes like this

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<MainElement>
    <MainEle1>
        <InfoA>
            <InfoAId>121480431A</InfoAId>
            <firstName>Anne</firstName>
            <lastName>Stevens</lastName>
            <phone>6023122456</phone>
            <address>
                <address1>8735 Elm Road</address1>
                <address2></address2>
                <city>Nowhere</city>
                <country></country>
                <state>CA</state>
                <suite></suite>
                <zipCode>30141</zipCode>
            </address>
            <dob>09/08/1982</dob>
            <gender></gender>
            <primaryLanguage>Other</primaryLanguage>
            <otherLanguage>french</otherLanguage>
            <planName></planName>
            <contacts>
                <firstName>rajesh</firstName>
                <lastName>raj</lastName>
                <phone>1232343241</phone>
                <address>
                    <address1></address1>
                    <address2></address2>
                    <city></city>
                    <country></country>
                    <state></state>
                    <suite></suite>
                    <zipCode></zipCode>
                </address>
                <contactType>EMERGENCY</contactType>
                <relationship>friend</relationship>
                <typeDesc>Emergency Contact/Next of Kin</typeDesc>
            </contacts>
         </InfoA>
    </MainEle1>
    <MainEle2>
        <subMainEle2>
            <firstName>victor</firstName>
            <lastName>john</lastName>
            <phone>1233455678</phone>
            <address>
                <address1>123 arrow</address1>
                <address2></address2>
                <city>upland</city>
                <country></country>
                <state>ca</state>
                <suite>234</suite>
                <zipCode>76547</zipCode>
            </address>
            <contactType>PRIMARY</contactType>
            <relationship></relationship>
            <typeDesc></typeDesc>
        </subMainEle2>
    </MainEle2>
    <MainEle3>
    ..
    </MailEle3>
    <MainEle4>
    ..
    </MailEle4>
    ..
</MainElement>

Dim reader As XmlTextReader
reader = New XmlTextReader("c:\FileName")

While Not reader.EOF
    Select Case reader.NodeType
    Case XmlNodeType.Element 
        Select Case reader.Name
        Case "MainEle1"
            Call ProcessMainEle1(reader) ' passing reader as ref
        Case "MainEle2"
            call ProcessMainEle2(reader) ' passing reader as ref
        Case "MainEle3"
            call ProcessMainEle3(reader) 'passing reader as ref
                 ' and read many element
        End Select
    End Select
End While

and in Process method I am using similar approach and creating the insert statement.
Is that the right approach or is there a simple approach for large file with many element within elements. I would appreciate guidance for novice programmer

Answer 1

For your use case It is important to use an XML Parser which is not DOM based. (SAXParser)

From memory consumption point of view there is nothing worse in IT than a DOM based XML parser. Such a parser will need 100 – 1000 times more memory than a low level parsing, which is more work for you.