Question

I am uploading data from Excel to SQL Server using the following structure:

Private Sub ado_upload()

    Dim objConnection As New ADODB.Connection
    Dim objCommand As New ADODB.Command
    Dim strSQL As String
    Dim strDSN As String
    Dim intCounter As Integer

    strDSN = "provider=SQLOLEDB;" _
    & "server=<server>;" _
    & "database=<database>;" _
    & "uid=<uid>;pwd=<pwd>;" _
    & "trusted_connection=false;"

    With objConnection
        .ConnectionString = strDSN
        .Open
    End With

    strSQL = "SET NOCOUNT ON; " _
    & "INSERT INTO dbo.[table1] ( [col1] ) VALUES ( ? );"

    With objCommand

        .ActiveConnection = objConnection
        .CommandText = strSQL
        .Prepared = True
        .Parameters.Append .CreateParameter("col1", adInteger, adParamInput)

        For intCounter = 0 To 9
            .Parameters("Col1").Value = intCounter
            .Execute
        Next intCounter

    End With

End Sub

The speed of the procedure depends on the geographic distance between the server and the computer running the procedure. On the server itself it is fast (300,000 inserts in under 10 minutes), on the other side of the country it is relatively slow (300,000 inserts could take hours). Remote uploads operate over a VPN.

I think network latency must be slowing the process down. Is there any way to get around network latency, or tweak the VPN to make the uploads faster?

Thanks!

Answer 1

The speed of the procedure depends on the geographic distance between the server and the computer running the procedure.

It really depends on how many hops you and the machine you’re connecting to on the other end are from major trunks, and the routes between. It’s not always the same route, either. Use the tracert command to see, because the bottleneck could be packets going through a 10 Mb/s connection rather than an OC line.

On the server itself it is fast (300,000 inserts in under 10 minutes),

It should go without saying that performing something local to the machine will run at the fastest possible speed, outside of load on the machine at the given time.

…uploads operate over a VPN.

While it is secure, it’s compounding the problem because the overhead of that security means you can send less per packet than if the data were unencrypted.

As I mentioned before, you don’t any control over routes taken once your packets are outside your network. All I can suggest is to upload the files to the server, and then run from there.

Downloads are so much faster than uploads, even though it’s still just an ADO Command. Is that because the data is being downloaded from SQL Server as binary data? Is there any way to achieve that same performance on the upload without uploading the file to the server first?

Depends on the connection & the contract terms. Residential internet is always an asymmetric connection – you’ll have more download bandwidth than upload. The reason being that surfing/etc is download centric – you use very little bandwidth for uploads, like page requests and submitting forms. Until you want to upload files…

The only way to get better upload speeds is to get the better connection and/or contract terms.