I have a VB web application that reads from a CSV file which contains about 300,000++ lines.
The application will read line-by-line and for each line, it will query a table in a sql server database which contains about 100,000++ records and based on the results of the query, the application will do something else. sqlservr.exe maxes out memory consumption in my development machine of 4GB.
So I created a simple application to test the performance of sql server in terms of loop query.
Dim Connection As New SqlConnection(ConfigurationManager.ConnectionStrings("SiteSqlServer").ConnectionString)
Dim Command As New SqlCommand("", Connection)
Connection.Open()
For i As Integer = 0 To 20000
Command.CommandText = "SELECT CustomerID FROM Customer WHERE CustomerID = " & i
Command.ExecuteScalar()
Next
Connection.Close()
Every time this code is executed, sqlservr.exe will take up an additional 100MB++ of memory and it’s not releasing it back even though after the code has finished executing.
Is this normal? Is there any way to mitigate this effect?
Yes, it is normal. SQL Server in general doesn’t release memory once it has acquired it. It’s purpose is to deliver results to you as quickly as possible, and if it has data that is cached in RAM, it can be queried quicker. SQL Server might repurpose the memory blocks it has acquired internally, but it doesn’t release them while it is running. This is by design. Try reading this.
This is also why, in general, you should run SQL Server on a separate box in production, so it doesn’t contend with other applications for memory (such as IIS).