Question

I have html with nested elements (mostly just div and p elements)
I need to return the same html, but substring’ed by a given number of letters. Obviously the letter count should not enumerate through html tags, but only count letters of InnerText of each html element.
Html result should preserve proper structure – any closing tags in order to stay valid html.

Sample input:

<div>
    <p>some text</p>
    <p>some more text some more text some more text some more text some more text</p>
    <div>
        <p>some more text some more text some more text some more text some more text</p>
        <p>some more text some more text some more text some more text some more text</p>
    </div>
</div>

Given int length = 16 the output should look like this:

<div>
    <p>some text</p> // 9 characters in the InnerText here
    <p>some mo</p> // 7 characters in the InnerText here; 9 + 7 = 16;
</div>

Notice that the number of letters (including spaces) is 16. The subsequent <div> is eliminated since the letter count has reached variable length. Notice that output html is still valid.

I’ve tried the following, but that does not really work. The output is not as expected: some html elements get repeated.

public static string SubstringHtml(this string html, int length)
{
    HtmlDocument doc = new HtmlDocument();
    doc.LoadHtml(html);
    int totalLength = 0;
    StringBuilder output = new StringBuilder();
    foreach (var node in doc.DocumentNode.Descendants())
    {
        totalLength += node.InnerText.Length;
        if(totalLength >= length)
        {
            int difference = totalLength - length;
            string lastPiece = node.InnerText.ToString().Substring(0, difference);
            output.Append(lastPiece);
            break;
        }
        else
        {
            output.Append(node.InnerHtml);
        }
    }
    return output.ToString();
}

---

UPDATE

@SergeBelov provided a solution that works for the first sample input, however further testing presented an issue with an input like the one below.

Sample input #2:

some more text some more text 
<div>
    <p>some text</p>
    <p>some more text some more text some more text some more text some more text</
</div>

Given that variable int maxLength = 7; an output should be equal to some mo.
It does not work like that because of this code where ParentNode = null:

lastNode
    .Node
    .ParentNode
    .ReplaceChild(HtmlNode.CreateNode(lastNodeText.InnerText.Substring(0, lastNode.NodeLength - lastNode.TotalLength + maxLength)), lastNode.Node);

Creating a new HtmlNode does not seem to help because its InnterText property is readonly.

Answer 1

The small console program below illustrates one possible approach, which is:

1. Select relevant text nodes and calculate the running total of length for them;
2. Take as many nodes as required to get to the running total past the max length;
3. Remove all element nodes from the document except the ones that are ancestors of the nodes we selected during steps ##1, 2;
4. Cut the text in the last node of the list to fit the max length.

UPDATE: This should still work with a text node being the first; probably, a Trim() is required to remove the whitespace from it as below.

    static void Main(string[] args)
    {
        int maxLength = 9;
        string input = @"
            some more text some more text 
            <div>
                <p>some text</p>
                <p>some more text some more text some more text some more text some more text</
            </div>";

        var doc = new HtmlDocument();
        doc.LoadHtml(input);

        // Get text nodes with the appropriate running total
        var acc = 0;
        var nodes = doc.DocumentNode
            .Descendants()
            .Where(n => n.NodeType == HtmlNodeType.Text && n.InnerText.Trim().Length > 0)
            .Select(n => 
            {
                var length = n.InnerText.Trim().Length;
                acc += length;
                return new { Node = n, TotalLength = acc, NodeLength = length }; 
            })
            .TakeWhile(n => (n.TotalLength - n.NodeLength) < maxLength)
            .ToList();

        // Select element nodes we intend to keep
        var nodesToKeep = nodes
            .SelectMany(n => n.Node.AncestorsAndSelf()
                .Where(m => m.NodeType == HtmlNodeType.Element));

        // Select and remove element nodes we don't need
        var nodesToDrop = doc.DocumentNode
            .Descendants()
            .Where(m => m.NodeType == HtmlNodeType.Element)
            .Except(nodesToKeep)
            .ToList();

        foreach (var r in nodesToDrop)
            r.Remove();

        // Shorten the last node as required
        var lastNode = nodes.Last();
        var lastNodeText = lastNode.Node;
        var text = lastNodeText.InnerText.Trim().Substring(0,
                lastNode.NodeLength - lastNode.TotalLength + maxLength);
        lastNodeText
            .ParentNode
            .ReplaceChild(HtmlNode.CreateNode(text), lastNodeText);

        doc.Save(Console.Out);
    }