Question

There are so many different parsers for HTML that it is hard to choose the rigth one.

My task is to read url and to find a <table> with specific id and then parse all <tr> rows of this table for content (text), but also <a> links and <img> images within <td> tags.

I also need to check class for each row element to sort data to categories.

What would be my best choice, which library and what methods should I use to get resutls quick?

---

Example of part of HTML code that I want to parse:

<table id="t1">
  <tr class="r1">
    <td class="c1"><a href="..."><img height="50" src="..." width="50" /></a></td>
    <td class="c2">
      <div class="d1">
        <ul class="u1">
          <li class="l1"><a href="..." rel='...'>text here</a></li>
          <li class="l2"><a href="..." rel='...'>text here</a></li>
        </ul>
      </div>
      <div class="d2">
        <a href="...">text here</a>
      </div>
    </td>
    <td class="c3">
      <div ...>...</div>
      <div class="d2">
        <a href="...">text here</a>
      </div>
    </td>
    <td class="c4">text here</td>
    <td class="c5">text here</td>
  </tr>
  ...
</table>

Answer 1

Use Web::Query. Use its methods find and text and attr.

use List::Gen qw(mapn);
use Web::Query 'wq';

sub classify {
    my ($l) = @_; my %r;
    mapn { push @{ $r{$_[0]} }, $_[1] } 2, @$l; return %r;
};

my $w = wq('file:///tmp/so11301348.html');
my %rows = classify $w
    # find a <table> with specific id
    ->find('table#t1')
    # parse all <tr> rows of this table for content (text)
    # check class for each row element to sort data to categories
    ->find('tr')->map(sub {
        my (undef, $tr) = @_;
        return $tr->attr('class') => $tr->text;
    });
# (
#     '' => [
#         ' ... '
#     ],
#     r1 => [
#         'text heretext heretext here...text heretext heretext here'
#     ]
# )

my $links_images = $w
# but also <a> links and <img> images within <td> tags
->find('td a, td img')
->map(sub {
    my (undef, $e) = @_;
    return $e->attr('src')
        ? [img => $e->attr('src') => $e->attr('alt')]
        : [a => $e->attr('href') => $e->text];
});
# [
#     ['a',   '...', ''],
#     ['img', '...', ''],
#     ['a',   '...', 'text here'],
#     ['a',   '...', 'text here'],
#     ['a',   '...', 'text here'],
#     ['a',   '...', 'text here']
# ]