Question

I have some code that uses deferred execution and lazy loading:

    public static IEnumerable<XElement> GetStreamElementP(string fileId, ListProgressEventHandler progressHandler, int total)
    {
        var filePath = Utility.GetEContentFilePath(fileId);
        using (var reader = XmlReader.Create(filePath, new XmlReaderSettings { IgnoreWhitespace = true, }))
        {
            var cnt = 0;
            reader.MoveToContent();
            // Parse the file and display each of the p nodes.
            reader.Read();
            while (reader.NodeType == XmlNodeType.Element && reader.Name == "p")
            {
                cnt++;
                var returnedValue = XElement.ReadFrom(reader) as XElement;

                int rem = cnt % _streamElementCallBackSize;
                if (progressHandler != null && rem == 0)
                {
                    progressHandler(null, new ListProgressEventArgs { ItemsProcessed = cnt, TotalItemsToProcess = total, });
                }
                yield return returnedValue;
            }
            reader.Close();
        }

    }

I’m looking to get a simple count on the number of elements. The current code we are using is:

    public static int FileElementsCount(string fileId)
    {
        var cnt = 0;
        foreach (XElement e in GetStreamElementP(fileId))
        {
            cnt++;
        }
        return cnt;
    }

Can I improve this to?

    public static int FileElementsCount(string fileId)
    {
        return GetStreamElementP(fileId).Count<XElement>();
    }

Or will this cause more memory to be used when getting the count? We are dealing with very large files in some cases and attempting to keep memory usage to a minimum where possible.

I have tried to find a concrete example that explains how the memory is used in each case without any success.

Thanks in advance for any help.

Answer 1

In your case, both ways of computing the count are will do the same.

The memory consumption of this function should be proportional the size of the <p> elements. So, if there is lots of small elements, it shouldn’t consume large amounts of memory. If you have relatively few huge elements, this could consume quite a lot of memory, because you’re creating an XElement out of each of them. If this was the case, the memory consumption could be made much smaller by not creating them at all.