Home/ Questions/Q 6017079
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-23T03:04:39+00:00

I imagine this is common enough that it’s a solved problem, but being a

  • 0

I imagine this is common enough that it’s a solved problem, but being a bit of a newbie with Loofah and Nokogiri I haven’t found the solution yet.

I’m using Loofah, a HTML scrubber library that wraps Nokogiri, to scrub some HTML text for display. However, that text sometimes happen to things like e-mail addresses and such between < and > characters, for example, < foo@domain.com >. Loofah is considering that as an HTML or XML tag, and is stripping it away from the text.

Is there a way to prevent this from happening while still doing a good job of scrubbing away the actual tags?

Edit: Here’s a failing test case:

require 'test/unit'
require 'test/unit/ui/console/testrunner'
require 'nokogiri'

MAGICAL_REGEXP = /<([^(?:\/|!\-\-)].*)>/

def filter_html(content)
  # Current approach in a gist: We capture content enclosed in angle brackets.
  # Then, we check if the excerpt right after the opening bracket is a valid HTML
  # tag. If it's not, we substitute the matched content (which is the captured
  # content enclosed in angle brackets) for the captured content enclosed in
  # the HTML entities for the angle brackets. This does not work with nested
  # HTML tags, since regular expressions are not meant for this.

  content.to_s.gsub(MAGICAL_REGEXP) do |excerpt|
    capture = $1
    Nokogiri::HTML::ElementDescription[capture.split(/[<> ]/).first] ? excerpt : "&lt;#{capture}&gt;"
  end
end

class HTMLTest < Test::Unit::TestCase
  def setup
    @raw_html = <<-EOS
<html>
<foo@bar.baz>
<p><foo@<b class="highlight">bar</b>.baz></p>
<p>
<foo@<b class="highlight">bar</b>.baz>
</p>
< don't erase this >
</html>
EOS

    @filtered_html = <<-EOS
<html>
&lt;foo@bar.baz&gt;
<p>&lt;foo@<b class="highlight">bar</b>.baz&gt;</p>
<p>
&lt;foo@<b class="highlight">bar</b>.baz&gt;
</p>
&lt; don't erase this &gt;
</html>
EOS
  end

  def test_filter_html
    assert_equal(@filtered_html, filter_html(@raw_html))
  end
end

# Can you make this test pass?
Test::Unit::UI::Console::TestRunner.run(HTMLTest)

We’re currently using some pretty evil regex hackery to try and accomplish this, but as the comment above states, it doesn’t work for tags “nested” inside non-tags. And we actually want to preserve the <b class="highlight"> elements as well.

The sample below isn’t using Loofah, but the application itself does in other places so it wouldn’t be hard to add it here. We’re just not sure of what configuration options we should use, if any.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    As the main issue was HTML tags enclosed in HTML entities angle brackets — which is totally mangled by Nokogiri — we solved it by just removing the aforementioned HTML tags, escaping the non-HTML-tag angle brackets and then putting the HTML tags back. It sounds a little hackish but it’s working perfectly. Our first goal was escaping email addresses enclosed in angle brackets, but this approach (supposedly) works for any kind text.

    # Does not run on ruby 1.9

require 'test/unit'
require 'test/unit/ui/console/testrunner'
require 'nokogiri'
require 'active_support/secure_random'

def filter_html(content)
  # Used to mark highlighted words.
  random_hex = SecureRandom.hex(6)

  # Remove highlighting.
  highlighted_terms = []
  without_highlighting = content.to_s.gsub(/<b class="highlight">(.*?)<\/b>/) do |match|
    highlighted_terms << $1
    "highlight-#{random_hex}:#{$1}"
  end

  # Escape non-HTML angle brackets.
  escaped_content = without_highlighting.to_s.gsub(/<(?:\s*\/)?([^!\-\-].*?)>/) do |excerpt|
    capture = $1
    tag = capture.split(/[^a-zA-Z1-6]/).reject(&:empty?).first
    !!Nokogiri::HTML::ElementDescription[tag] ? excerpt : "&lt;#{capture}&gt;"
  end

  # Add highlighting back.
  highlighted_terms.uniq.each do |term|
    escaped_content.gsub!(/highlight-#{random_hex}:(#{term})/) do |match|
      "<b class=\"highlight\">#{$1}</b>"
    end
  end

  escaped_content
end

class HTMLTest < Test::Unit::TestCase
  def setup
    @raw_html = <<-EOS
      <html>
        <foo@bar.baz>
        <p><foo@<b class="highlight">bar</b>.baz></p>
        <p>
          <foo@<b class="highlight">bar</b>.baz>
        </p>
        <    don't erase this   >
      </html>
    EOS

    @filtered_html = <<-EOS
      <html>
        &lt;foo@bar.baz&gt;
        <p>&lt;foo@<b class="highlight">bar</b>.baz&gt;</p>
        <p>
          &lt;foo@<b class="highlight">bar</b>.baz&gt;
        </p>
        &lt;    don't erase this   &gt;
      </html>
    EOS
  end

  def test_filter_html
    assert_equal(@filtered_html, filter_html(@raw_html))
  end
end

# It passes!
Test::Unit::UI::Console::TestRunner.run(HTMLTest)
    • 0