Question

I’m reading from 1-5mb log files outputting to a textview and also searching for specific lines outputting to another textview. Currently it takes about a minute for just a 1mb file. Does anyone know any faster methods of searching through lines or strings other than the method I’m using?

Imports EnterpriseDT.Net.Ftp

Public Class Form1

Private Sub SettingsToolStripMenuItem_Click(sender As System.Object, e As System.EventArgs) Handles SettingsToolStripMenuItem.Click

End Sub

Private Sub Button2_Click(sender As System.Object, e As System.EventArgs) Handles Button2.Click
    Dim sw As New Stopwatch
    Dim FullLine As String = ""
    Dim ScriptLine As String = ""
    sw.Start()

    Dim ll As New Queue(Of String)

    Dim i As String = ""

    Using TestFile As New IO.StreamReader("c:\test.txt", System.Text.Encoding.Default, False, 4096)

        Using OutFile As New IO.StreamWriter("c:\SBOutFile.txt", False, System.Text.Encoding.Default, 4096)

            While TestFile.EndOfStream = False

                i = TestFile.ReadLine

                If i.Contains(".sqf") And i.Contains("handleGear.sqf") = False Then
                    ScriptLine = ScriptLine & i & vbNewLine & vbNewLine
                    FullLine = FullLine & i & vbNewLine & vbNewLine
                Else
                    FullLine = FullLine & i & vbNewLine & vbNewLine
                End If

            End While

        End Using

    End Using

    sw.Stop()
    TextBox1.Text = FullLine
    TextBox2.Text = ScriptLine
    RichTextBox1.AppendText(String.Format("Run_Queue took {0} Milliseconds." & Environment.NewLine, sw.ElapsedMilliseconds))
End Sub

Private Sub Button1_Click_1(sender As System.Object, e As System.EventArgs) Handles Button1.Click
    Try
        'connect to ftp server
        Dim ftp As New FTPConnection
        ftp.ServerAddress = "-"
        ftp.ServerPort = "-"
        ftp.UserName = "-"
        ftp.Password = "-"
        ftp.Connect()
        ftp.ChangeWorkingDirectory("-")
        ftp.TransferType = FTPTransferType.BINARY

        'download a file
        ftp.DownloadFile("c:\test.txt", "scripts.log")

        'ftp.RenameFile("scripts.log", "scripts_test.log")

        'close the connection
        ftp.Close()

    Catch ex As Exception
        MessageBox.Show(ex.Message.ToString())
    End Try
End Sub

Private Sub Button3_Click(sender As System.Object, e As System.EventArgs) Handles Button3.Click

End Sub

End Class

Answer 1

Given the heavy reading and concatenating you are doing as you’re reading, I suspect its part of your time/performance issue. I would probably consider changing the declarations for ScriptLine and FullLine from String type to a StringBuilder, because Strings are technically immutable. That means each concatenation really turns out to be teardown of the previous object, and the creation of a new one in its place. StringBuilders are designed specifically for heavy concatenation scenarios. When the looping is finished, you can convert it back to a String.

Also, a compiled Regular Expression might search faster than String.Contains. Your regular expression string would be something like “(?!handleGear).sqf”, meaning “find any sequence of zero or more characters other than “handleGear” in front of the string “.sqf”.

I haven’t had a chance to test that expression, so it is offered with that caveat. If I get a chance to throw together a test, I’ll be glad to amend and let you know.

Good luck!