Question

I have a powershell 2.0 script on an XP OS. The purpose of the script is to extract data from an old database (Sybase) and populate a SQL Server 2008 database. The model that I am using is to create OLEDB connections to the Sybase database. The script calls a series of stored procedures from the Sybase database. The results are used to create an XML string. The XML string is queried for the input data for the SQL Server stored procedures. AFter each data element is created in the SQL Server database the XML string is saved to a file. Every database connection is closed after execution is completed. It is simple model, but uses staggering amounts of memory. For transferring only 1000 rows of data the script memory footprint grows to 3G. When the script completes the memory does not drop. In an attempt to rectify this problem I have added logic to free every variable when not used and call garbage collection in every finally clause of every try block. I am aware that this is overkill, but am trying to find anything that will reduce the memory usage. I am in the processing of looking for a memory trace tool, but I am also looking for expert opinions as to possibly start tracking down this critical issue. I know I am missing something obvious so any advice would be appreciated.

Answer 1

I found the source of the leak. I am using an OLEDB connection to a Sybase Server instance to get data to load into SQL Server. 95% of the leak was isolated to a single function that invoked a Sybase stored procdure. Instead of getting the results in a result set, this procedure had the option of iether returning the results as output parameters or a result set. I initially chose the output parameter option. For reasons that are not clear to me, this output parameter mechanism caused a massive memory leak. I changed the logic to use a result set and the resolved the leak. Not clear why output parameters were an issue, but I found an optional approach that corrected the problem.