Question

I am currently using VBA to run a stored procedure and stored data that been extracted to table. Then, the VBA will query all the data in the data accordingly and put it into excel.

Here the problem, it takes so long for the VBA to extract all data (around 100k rows of data) into excel. Is there any other way to speed up the process? Below is part of my code. The one in bold is the insertion to excel code.

'Row number where data inserting starts
        Do
            current_sheet = owb.ActiveSheet

            With current_sheet
                'Insert header to worksheet in first row, ie. A1, B1, C1
                For i = 0 To data_cols.GetLength(0) - 1
                    cell = data_cols(i, 0) & header_row_num 'Change to header_row_num
                    .Range(cell).Value = data_cols(i, 1)
                Next i
            End With

            row_count = header_row_num + 1 'Change the first row count to a row after header_row_num
            'Insert data to worksheet
            While rs.EOF = False

                With current_sheet
                    'Set format of specic columns before inserting data

                    .Columns("A").NumberFormat = "@"

                    .Columns("B").NumberFormat = "@"

                    .Columns("C").NumberFormat = "@"

                    .Columns("D").NumberFormat = "@"

                    .Columns("E").NumberFormat = "@"

                    .Columns("F").NumberFormat = "@"

                    .Columns("G").NumberFormat = "@"

                    .Columns("H").NumberFormat = "@"

                    .Columns("I").NumberFormat = "@"

                    .Columns("J").NumberFormat = "@"

                    .Columns("K").NumberFormat = "@"

                    .Columns("L").NumberFormat = "@"

                    .Columns("M").NumberFormat = "@"

                    .Columns("N").NumberFormat = "@"

                    .Columns("O").NumberFormat = "@"

                    .Columns("P").NumberFormat = "@"

                    .Columns("Q").NumberFormat = "@"

                    .Columns("R").NumberFormat = "@"

                    .Columns("S").NumberFormat = "@"

                    **'Start inserting data
                    For i = 0 To data_cols.GetLength(0) - 1
                        'Get the cell name
                        cell = data_cols(i, 0) & row_count
                        'Populate data to the cell
                        If IsDBNull(rs.Fields(data_cols(i, 2)).Value()) Then
                            .Range(cell).Value = " "
                        Else
                            .Range(cell).Value = rs.Fields(data_cols(i, 2)).Value()
                        End If
                    Next i
                End With
                rs.MoveNext()
                'Indicates next row
                row_count += 1**

                If row_count > 60000 Then
                    owb.Worksheets.Add(, current_sheet)
                    need_new_sheet = True
                    Console.WriteLine("Added new sheet to workbook...")
                    Exit While
                Else
                    need_new_sheet = False
                End If

            End While
        Loop While (need_new_sheet And rs.EOF = False)

In case certain variable you need to know.

row_count = header_row_num + 1 'Change the first row count to a row after header_row_num

oxl = CreateObject("Excel.Application")
    oxl.Visible = False
    owb = oxl.Workbooks.Add

Dim data_cols(,) As String = {{"A", "Name", "NAME"}, _
                           {"B", "Age", "AGE"}}  (Not real columns, example)

Any advice or thoughts would be greatly appreciate. Thanks in advance 🙂

Answer 1

Filling up 100k rows in Excel will definitely take time.

This is what you can do to minimize the time

1. Use oxl.ScreenUpdating = False in the beginning of the macro and set it to True in the end.
2. You might want to store the data in the array and then writing the array to Excel in one go in the end. This would definitely decrease the execution time
3. Excel 2007 onwards Excel has a row limitation of 1048576 rows so you might want to take that into consideration if you cross that limit.
4. Console.WriteLine("Added new sheet to workbook...") is VB.net. Use Debug.print if you are using VBA.
5. BTW this will not have significant effect on the speed but you can write the following code

Which is

.Columns("A").NumberFormat = "@"
.Columns("B").NumberFormat = "@"
'
'
'
.Columns("R").NumberFormat = "@"
.Columns("S").NumberFormat = "@"

as

.Columns("A:S").NumberFormat = "@"