I have a complicated algorithm which receives data from a socket connections, transformate the data and stores it as soon as possible on the HD..The data, because of that fact I don’t want the processing to slow down is stored by using a different thread.
The data storage algorithm resembles this structure. It essentially saves XML on the disk.
Begin Thread
beginthread:
XmlTextWriter xmltextWriter;
Save Xml file 1
xmltextWrite.close();
XmlTextWriter xmltextWriter;
Save Xml file 2
xmltextWrite.close();
goto beginthread:
End Thread
It works correctly but if I get a look to the Task Manager , I could notice that the amount of memory consumed by my program encreases quickly over the time (500mb after 1 hour of work). This could be justified because of the fact that the thread is not so fast as the data which comes in and the .NET framework stores temporary all in memory for me. But what I didn’t understand is why if the incoming socket connections will stop, even after a few minutes that the thread continues to work..the task manager continues to show 500Mb of memory..Why the memory is not relased ?! The XmlTextWriter object is a local variable and is closed every time.
As requested.. This is part of the code
beginthread:
if (sleeptime < 1000) sleeptime += 2;
try
{
while (hashBeginConn.Count > 0)
{
sleeptime = 0;
int connToApply = hashBeginConn[0];
if (olddate.ToShortDateString() != ListsockConnections[connToApply].beginDate.ToShortDateString())
{
JoinDataFromTempFile(ListsockConnections[connToApply].beginDate.Date.Subtract(olddate.Date).Days, false, d);
olddate = ListsockConnections[connToApply].beginDate.Date;
}
if (tocreate)
{
// XML Serialization
XmlTextWriter xmltextWriter;
Encoding enc = null;
if (ListsockConnections[connToApply].ENCfromCode) enc = Encoding.GetEncoding(ListsockConnections[connToApply].codepage);
if (ListsockConnections[connToApply].ENCDefault) enc = Encoding.Default;
if (ListsockConnections[connToApply].ENCfromText) enc = Encoding.GetEncoding(ListsockConnections[connToApply].codename);
if (enc == null) { enc = null; }
// xmltextWriter = new XmlTextWriter(folderPath + "\\" + cacheFileName, enc);
xmltextWriter = new XmlTextWriter(DataPath + "\\_temp.xml", enc);
xmltextWriter.Formatting = Formatting.Indented;
// Start document
// xmltextWriter.WriteStartDocument();
xmltextWriter.WriteStartElement("ConnectionList");
xmltextWriter.WriteStartElement("connection");
xmltextWriter.WriteStartElement("ConnectionCounter");
xmltextWriter.WriteValue(ListsockConnections[connToApply].ConnectionCounter.ToString());
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("IDConnection");
xmltextWriter.WriteValue(ListsockConnections[connToApply].IDConnection.ToString());
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("Parsed");
xmltextWriter.WriteValue("false");
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("connType");
xmltextWriter.WriteValue("TCP/IP");
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("beginConn");
xmltextWriter.WriteValue(ListsockConnections[connToApply].beginDate.ToString());
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("remoteAddressFamily");
xmltextWriter.WriteValue(ListsockConnections[connToApply].remoteAdressFamily);
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("remoteIP");
xmltextWriter.WriteValue(ListsockConnections[connToApply].remoteIP);
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("localIP");
xmltextWriter.WriteValue(ListsockConnections[connToApply].localIP);
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("remoteport");
xmltextWriter.WriteValue(ListsockConnections[connToApply].remoteport.ToString());
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("localport");
xmltextWriter.WriteValue(ListsockConnections[connToApply].localport.ToString());
xmltextWriter.WriteEndElement();
xmltextWriter.WriteStartElement("dataEncoding");
if (ListsockConnections[connToApply].codepage != 0 || ListsockConnections[connToApply].codename != "")
{
if (ListsockConnections[0].codepage != 0)
{ xmltextWriter.WriteValue(ListsockConnections[connToApply].codepage.ToString()); }
else
{ xmltextWriter.WriteValue(ListsockConnections[connToApply].codename.ToString()); }
}
else
{ xmltextWriter.WriteValue("NONE"); }
xmltextWriter.WriteEndElement();
xmltextWriter.WriteEndElement();
xmltextWriter.WriteEndElement();
xmltextWriter.Flush();
xmltextWriter.Close();
tocreate = false;
}
else
{
FileInfo fi;
FileStream fstream;
//fi = new FileInfo(folderPath + "\\" + cacheFileName);
fi = new FileInfo(DataPath + "\\_temp.xml");
fstream = fi.OpenWrite();
XmlTextWriter xmltextWriter;
Encoding enc = null;
if (ListsockConnections[connToApply].ENCfromCode) enc = Encoding.GetEncoding(ListsockConnections[connToApply].codepage);
if (ListsockConnections[connToApply].ENCDefault) enc = Encoding.Default;
if (ListsockConnections[connToApply].ENCfromText) enc = Encoding.GetEncoding(ListsockConnections[connToApply].codename);
if (enc == null) { enc = null; }
xmltextWriter = new XmlTextWriter(fstream, enc);
xmltextWriter.Formatting = Formatting.Indented;
fstream.Position = fstream.Length - 17;
xmltextWriter.WriteRaw(" <connection>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <ConnectionCounter>");
xmltextWriter.WriteValue(ListsockConnections[connToApply].ConnectionCounter.ToString());
xmltextWriter.WriteRaw("</ConnectionCounter>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <IDConnection>");
xmltextWriter.WriteValue(ListsockConnections[connToApply].IDConnection.ToString());
xmltextWriter.WriteRaw("</IDConnection>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <Parsed>");
xmltextWriter.WriteValue("false");
xmltextWriter.WriteRaw("</Parsed>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <connType>");
xmltextWriter.WriteValue("TCP/IP");
xmltextWriter.WriteRaw("</connType>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <beginConn>");
xmltextWriter.WriteValue(ListsockConnections[connToApply].beginDate.ToString());
xmltextWriter.WriteRaw("</beginConn>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <remoteAddressFamily>");
xmltextWriter.WriteValue(ListsockConnections[connToApply].remoteAdressFamily);
xmltextWriter.WriteRaw("</remoteAddressFamily>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <remoteIP>");
xmltextWriter.WriteValue(ListsockConnections[connToApply].remoteIP);
xmltextWriter.WriteRaw("</remoteIP>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <localIP>");
xmltextWriter.WriteValue(ListsockConnections[connToApply].localIP);
xmltextWriter.WriteRaw("</localIP>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <remotePort>");
xmltextWriter.WriteValue(ListsockConnections[connToApply].remoteport.ToString());
xmltextWriter.WriteRaw("</remotePort>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <localport>");
xmltextWriter.WriteValue(ListsockConnections[connToApply].localport.ToString());
xmltextWriter.WriteRaw("</localport>" + Environment.NewLine);
xmltextWriter.WriteRaw(" <dataEncoding>");
if (ListsockConnections[connToApply].codepage != 0 || ListsockConnections[connToApply].codename != "")
{
if (ListsockConnections[connToApply].codepage != 0)
{
xmltextWriter.WriteValue(ListsockConnections[connToApply].codepage.ToString());
}
else
{
xmltextWriter.WriteValue(ListsockConnections[connToApply].codename.ToString());
}
}
else
{
xmltextWriter.WriteValue("NONE");
}
xmltextWriter.WriteRaw("</dataEncoding>" + Environment.NewLine);
xmltextWriter.WriteRaw(" </connection>" + Environment.NewLine);
xmltextWriter.WriteRaw("</ConnectionList>");
xmltextWriter.Flush();
xmltextWriter.Close();
fstream.Close();
if (fi.Length >= (maxFileTempSize * 1000000))
{
JoinDataFromTempFile(0, false, enc);
}
}
lock (lockThis)
{
hashBeginConn.RemoveAt(0);
}
}
A number of answers say that you must call Dispose. Though those answers mean well, they won’t actually help you. You are calling Close, and Close and Dispose do the same thing. It is a better practice to use a “using” block so that you get Dispose called for you automatically, but your code is fine as it is.
The real answer to your question is “stop worrying about it”. You are thinking about this at the wrong level. I assume that you are looking at either “working set” or “private bytes” in the task manager, but you might not understand what those actual means. Most people do not. This answer gives a good summary:
What is private bytes, virtual bytes, working set?
OK, now that you know what “private bytes” are, it should be more clear why this is not a problem. Suppose you are the CLR garbage collector. You allocate a bunch of memory on behalf of the user and use it to store managed objects. Every now and then, the garbage collector runs, compacting the objects in memory and marking the memory formerly used by now-dead objects as available. But why would the GC return all those blocks of memory to the operating system? The GC has evidence that you are the sort of person who writes programs who use that much memory, so it keeps the empty pages around so that it does not have to incur the expense of allocating them again later when you use that much memory again two milliseconds from now.
So, stop worrying about it. Everything is probably fine. Using 500 million bytes of memory is not a problem. If you start getting low on virtual memory, the GC will likely start decommitting the unused pages. If it doesn’t — if that keeps growing without bound, then start worrying.
If you’re still worried, use the right tool for the job. “Private bytes” tells you very little about what is actually happening with memory in your program. If you want to know what is really happening in the garbage collector then you need to use a managed memory profiler. It will give you a report of exactly what is going on.